Hi, could you provide guidance to lower tensorrt memory usage? I have compiled the engine without CUDNN etc. but it still uses ~500MB of ram for a 4MB engine. Perhaps smaller inference only libraries?
The memory is used for loading the TensorRT library and it’s essential for inference.
Did you also run TensorRT without cuDNN when inference?
The library is by default loaded.
Yes I’m running without cuDNN, but it’s for example loading the 193MB libcufft file, all 193MB are required for inference? It also required cublas/cublaslt even if I don’t use cublas/cublaslt as a tactic source
I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks
Which JetPack do you use?
Do you mind sharing the model with us so we can check it further?