Deep Stream takes too much time to create Engine file at startup

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Nano
• DeepStream Version 6
• JetPack Version (valid for Jetson only) 4.6
• TensorRT Version 8.0.1.6
• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)
I am using TAO’s TrafficCam Net in Python App.
The default engine file with TrafficCam is resnet18_trafficcamnet_pruned.etlt_b1_gpu0_int8.engine Whereas Jetson only accepts FP16. That’s why it always starts to create new engine file at startup, taking around 2 minutes.
Following is the config file and terminal output where the application stops for 2 mins.

What’s the solution?

As the build process builds a fp16 engine you should use this filename in model-engine-file. Then the correct engine will be picked up at start. The filename should be reset*…_gpu0_fp16.engine

You are suggesting to just renaming the .engine file?

Tried, same issue persists.

I looked into the trafficcamnet folder in tao_pretrained_models. There is no .engine file there.

No, not renaming. As I can see in your screenshot you selected the fp16 network mode. If the compilation to fp16 was successful there has to be a engine file with “fp16” in the name (usually in the same directory as the encoded model file). When you set this filename in model-engine-file (and try to comment out tlt-encoded-model, iam not sure if this is necessary) DS should pick up this engine file directly instead of compiling it again.

I am not sure where it creates the new .engine file with fp16. When running deepstream, terminal does say ‘Trying to create engine from model files’ but I can’t find any engine file created. Not in the same directory.

Yay, problem solved. It was actually trying to create the new engine file in the same directory but didn’t had the permission. So, I ran deepstream with sudo and it created new .engine file (fp16) in the same directory. This file can now be re-used and saves time.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.