setMaxWorkspaceSize parameter clarification

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

“setMaxWorkspaceSize”, is this parameter only valid while building tensorrt engine? Or is it valid after the engine is built also?
“Increasing the limit may affect the number of applications that could share the GPU at the same time. Setting this limit too low may filter out several algorithms and thus create a sub-optimal engine”. https://developer.nvidia.com/blog/speed-up-inference-tensorrt

From the above reading, I found out that we need to allot a good amount of size while building the engine, but if that workspacesize I allot is too high then it might affect other processes which are running on the same gpu. But if that parameter is not valid after the engine is built(using the engine file for inference after it is built successfully), then it won’t affect performance, so there is no need to worry about it.
Also, what is the most optimal size we can chose?

2

Hi @bschandu67,

Hope following will help you. Please refer developer guide for more info.

Q: How do I choose the optimal workspace size?

A: Some TensorRT algorithms require additional workspace on the GPU. The method IBuilderConfig::setMaxWorkspaceSize() controls the maximum amount of workspace that may be allocated, and will prevent algorithms that require more workspace from being considered by the builder. At runtime, the space is allocated automatically when creating an IExecutionContext. The amount allocated will be no more than is required, even if the amount set in IBuilderConfig::setMaxWorkspaceSize() is much higher. Applications should therefore allow the TensorRT builder as much workspace as they can afford; at runtime TensorRT will allocate no more than this, and typically less.

Thank you.