Lowering tensorrt memory usage

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi, could you provide guidance to lower tensorrt memory usage? I have compiled the engine without CUDNN etc. but it still uses ~500MB of ram for a 4MB engine. Perhaps smaller inference only libraries?

3

Hi,

The memory is used for loading the TensorRT library and it’s essential for inference.

Did you also run TensorRT without cuDNN when inference?
The library is by default loaded.

Thanks.

4

Yes I’m running without cuDNN, but it’s for example loading the 193MB libcufft file, all 193MB are required for inference? It also required cublas/cublaslt even if I don’t use cublas/cublaslt as a tactic source

5

I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks

Hi,

Which JetPack do you use?
Do you mind sharing the model with us so we can check it further?

Thanks.