How to minimize gpu usage (e.g for tensorflow ==> gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.3) in Tensorrt?
my code,
trt_graph = trt.create_inference_graph(
input_graph_def=graph_def,
outputs=outputs,
max_batch_size=32,
max_workspace_size_bytes=1 << 25,
precision_mode="FP16")
Hi,
Do you mean the memory usage of TensorRT?
If yes, max_workspace_size_bytes indicates the maximal available memory for TensorRT.
You can update the parameter based on your use case.
Here is some relevant information for your reference:
[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#troubleshooting[/url]
Thanks.