Is there a way to speed up the start of an inference process?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson Nano
• DeepStream Version
5.0.1
• JetPack Version (valid for Jetson only)
4.4

• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I’m asking for a friend :)

Especially during development of DeepStream SDK apps it is a bit nightmarish to be forced to wait for 30-60 secs every time until everything is setup. I suppose the GPU preparations take that long, but why? I mean if I don’t reboot the machine, why is it required to feed the GPU again and again with the same stuff?

I would be fine if I would just have to wait that long for the very first time after a boot.

2

Have you saved the engine file?

60 seconds looks like its recompiling the model every time.

3

Hmm. Your question seems to reveal an interesting fact. But I don’t know what you mean. I basically launch one of the sample Python apps. There is a configuration file, which points to a model and label file. I wouldn’t know what to do else to prevent a new compilation. Could you please be a bit more specific on that?

4

“Trying to create engine from model files”

I think, this is a pointer to the “over and over again compilation”?

How to change that?

5

Cool. Figured it out

model-engine-file=…/…/…/…/samples/models/Primary_Detector_Nano/resnet10.caffemodel_b3_gpu0_fp16.engine

6

Thanks for the pointer, btw