Minimize gpu usage with Tensorrt

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

How to minimize gpu usage (e.g for tensorflow ==> gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.3) in Tensorrt?

my code,

		trt_graph = trt.create_inference_graph(
			input_graph_def=graph_def,
			outputs=outputs,
			max_batch_size=32,
			max_workspace_size_bytes=1 << 25,
			precision_mode="FP16")
2

Hi,

Do you mean the memory usage of TensorRT?
If yes, max_workspace_size_bytes indicates the maximal available memory for TensorRT.
You can update the parameter based on your use case.

Here is some relevant information for your reference:
[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#troubleshooting[/url]

Thanks.