Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

“INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.”

What does this means? I see it when starting deepstream-app sometimes. I see it on every run with the objectdetection_SSD sample config.

Is there some settings we need to change or is it safe to ignore this?

There is a post from Jan 31 asking the same question with no response: Is the INFO `[I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.` critical?

#3

Hi,

There is a “workspace” parameter that limit the maximal memory amount for TensorRT.
This error indicates that the workspace is not enough for TensorRT to reach the optimal performance.

You can update the workspace value like this:
config_infer_primary_ssd.txt

[property]
...
workspace-size=471859200   # default is set to 450Mb
...

To set a proper workspace value, please check this document for information:
https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#faq

Q: How do I choose the optimal workspace size?

A: Some TensorRT algorithms require additional workspace on the GPU. The method IBuilderConfig::setMaxWorkspaceSize() controls the maximum amount of workspace that may be allocated, and will prevent algorithms that require more workspace from being considered by the builder. At runtime, the space is allocated automatically when creating an IExecutionContext. The amount allocated will be no more than is required, even if the amount set in IBuilderConfig::setMaxWorkspaceSize() is much higher. Applications should therefore allow the TensorRT builder as much workspace as they can afford; at runtime TensorRT will allocate no more than this, and typically less.

Thanks.

#4

So if the workspace size is not large enough does that mean the model is running slower than it should?

#5

If I try a bigger number (on Jetson Xavier NX) I get this error:

ERROR: [TRT]: ../rtSafe/safeRuntime.cpp (25) - Cuda Error in allocate: 2 (out of memory)

Have tried setting it to just a little higher at: workspace-size=600000000. #600MB

Memory usage at the moment is: 1.6G/7.8GB.

Actually some more testing is showing me that the mobilenet_ssd config wont run at all if I have that parameter set. Maybe the units is wrong and it needs to be 600 and not 600000000 ?

#6

Found the problem. The units are in MB. So if you want to set 3GB for example you enter it as :

workspace-size=3000

and not

workspace-size=3000000000

I find that with the SSD sample you need it at 3GB before the warning foes away.

I don’t actually see any meaningful performance difference though so can probably just remove the parameter…

1 Like
Trtexec Yolov4: Some tactics do not have sufficient workspace memory to run
#7

Hi,

You are the right, the unit is MB.
Sorry that I check this source code directly:
/opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_context_impl.cpp

static const int WORKSPACE_SIZE = 450 * 1024 * 1024;

But missing that the parameter from configure file is updated to MB here:
/opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_model_builder.cpp

        params.workspaceSize =
            initParams.workspaceSize * UINT64_C(1024) * UINT64_C(1024);

Thanks.

1 Like
#10

I don’t notice this message when loading .engine model file directly. So probably, the limited performance is only for the caffe / onnx model to .engine conversion, which only needs to be done once.

2 Likes