“setMaxWorkspaceSize”, is this parameter only valid while building tensorrt engine? Or is it valid after the engine is built also?
“Increasing the limit may affect the number of applications that could share the GPU at the same time. Setting this limit too low may filter out several algorithms and thus create a sub-optimal engine”. https://developer.nvidia.com/blog/speed-up-inference-tensorrt
From the above reading, I found out that we need to allot a good amount of size while building the engine, but if that workspacesize I allot is too high then it might affect other processes which are running on the same gpu. But if that parameter is not valid after the engine is built(using the engine file for inference after it is built successfully), then it won’t affect performance, so there is no need to worry about it.
Also, what is the most optimal size we can chose?