My problem is simillar with cuDNN take up too much GPU memory , when I start two trtexec process, they occupy almost same memory, but according to the answer, tensorrt or cudnn dynamic library could share between processes, is there anything we should set?

How to share tensorrt between processes

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier

AastaLLL March 1, 2022, 6:33am 8

Hi,

Thanks for your patience.
We have got some information about this issue.

Please note that the CUDA context on Jetson is created per process.
Since cuDNN/TensorRT context is based on CUDA context, sharing context across different processes is not available.

On Jetson, it’s more recommended to use multi-thread (different model) or TensorRT batch (same model) instead.

Thanks.

GPU vs CPU deep learning memory usage

Topic		Replies	Views
cuDNN take up too much GPU memory cuDNN	7	2080	November 6, 2022
TensorRT used lots of memory when loading model files Jetson Orin NX tensorrt	6	1054	May 31, 2023
Why tensorRT occupy many memory ? Jetson TX2	9	3813	May 12, 2021
Multi-process running tensorRT Jetson AGX Xavier tensorrt	5	1558	October 18, 2021
Running two or more docker GPU using docker containers concurrently Jetson Nano docker , gpu , gpu-computing	5	1718	January 19, 2023
Running two neural networks simultaneously Jetson Xavier NX tensorrt , python , dla	4	1918	August 29, 2021
Sharing CUDA memory between processes Jetson AGX Xavier cuda	8	2203	October 18, 2021
Run time improvement of a few AI algorithms running together on a single GPU Jetson AGX Xavier gpu	3	423	October 18, 2021
[QST] Best practices of handling competence of gpu resource? TensorRT tensorrt , cuda , jetson-inference , performance	2	553	September 20, 2021
Tensorrt Threads affect each other during multithreaded inference TensorRT tensorrt	16	1314	September 6, 2024

How to share tensorrt between processes

Related topics