How to Limit GPU usage of Tensorrt Engine inference?

jazeel.jk · July 20, 2021, 11:38am

I have a trt engine converted from yolo v3 etlt model. I am inferring it with python in my RTX 3070 machine. How can i control the GPU memory fraction taken by the tensorrt engine ? Is there an option as in tensorflow, like

config.gpu_options.per_process_gpu_memory_fraction = 0.1

is there any such parameters to control that?

Morganh · July 20, 2021, 3:21pm

Please try to
export TF_FORCE_GPU_ALLOW_GROWTH=true

system · September 18, 2021, 3:22pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Whether the GPU's memory resources can be set in TensorRT ？ TensorRT	1	484	October 28, 2020
How to get total used gpu memory and set gpu memory limit when tensorrt inferring? TensorRT tensorrt	3	839	November 2, 2022
Manage TensorRT GPU memory conversion usage TensorRT tensorrt , tensorflow , ubuntu	3	2629	April 7, 2021
GPU memory difference between 1070 and 2070 for YOLOv3 TensorRT tensorrt	3	570	May 21, 2020
Minimize gpu usage with Tensorrt Jetson Nano	2	551	October 18, 2021
How to set limitation on GPU memory usage? Jetson TX2	2	5810	October 18, 2021
How to identify whether the memory used by the current graphics card comes from the memory module or from the graphics card itself Jetson Nano cuda , python	3	449	March 9, 2022
How to manage the GPU memory and Host Memory? Especially release to OS TensorRT	1	605	May 6, 2020
Tensorflow inference using TRT converted model TensorRT	10	1057	May 25, 2021
Is there any other way besides TensorRT to increase the GPU utilization while doing the inference? TensorRT	1	743	March 25, 2019

How to Limit GPU usage of Tensorrt Engine inference?

Related topics