Nvinfer to load an engine with multiple optimization profiles (without onnx)


My current setup is as follows:
• Hardware Platform (Jetson / GPU): GPU
• DeepStream Version: 6.0
• TensorRT Version: 8.2.3
• NVIDIA GPU Driver: 470.103.01
• Issue Type: question

Is it possible to configure nvinfer to load a dynamic engine with multiple optimization profiles without the onnx model? or I must provide an onnx and let nvinfer generate a new engine with a single optimization profile as mentioned here?

I am looking for way to reduce the down time of having to switch models together with the generation time needed by each new engine creation. On the Fly model update might do the trick but will it regenerate the engine as it did initially or I will need to regenerate the engine myself?

Best Regards,

Suppose build engine will be skipped if there is the TRT engine.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.