Trtexec Yolov4: Some tactics do not have sufficient workspace memory to run

Description

I am building a runtime engine using tensorrt from a .onnx file - YoloV4. It is able to build successfully however, even when i give the workspace 3 GB (3000 in MB in the command), it prints a message while building saying

Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.

I suspect the problem is that there is some configuration file somewhere that puts a limit on the maximum space that tensorrt is able to use. Sadly i can’t find such a file :( . Any help would be much apreciated!

Environment

TensorRT Version: 7.1.3
GPU Type: Volta- Arch 7.2
Nvidia Driver Version: Whatever comes with Jetpack 4.4 – L4T 32.4.3 on the Jetson Xavier NX
CUDA Version: 10.2.89
CUDNN Version: 8.0.0.180
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): N/A
PyTorch Version (if applicable): 1.6.0

Relevant Files

test1.onnx file here:
https://drive.google.com/drive/folders/1aMiWU2l_wy2ssDV21KPPnjARXZAezMwz?usp=sharing

Steps To Reproduce

Please include:

  • Using after building trtexec from usr/src/tensorrt/samples/trtexec run this trtexec command to build an engine from test1.onnx

/usr/src/tensorrt/bin/./trtexec --onnx=test1.onnx --explicitBatch --saveEngine=Yolov4_DLA1.trt --useDLACore=1 --workspace=3000 --fp16 --allowGPUFallback

Hi @MostafaTheReal,

Please refer to the below link for setting up the workspace.

Thanks!

Hi @AakankshaS, yes i have read this thread, the flag for the trtexec command is workspace not workspace-size, which is also in MB and i have also given it 3GB as mentioned above. That discussion is about the DeepStream SDK. I am asking about TensorRT. The file they referenced to change that parameter is not in the TensorRT package.

Hi @MostafaTheReal,

I was having the same problem when I tried to port the yolov4 onnx model to TensorRT with trtexec.
Using onnx-tensorrt solved the problem for me. At least, that sort of warning message didn’t come up anymore.

Hi @MostafaTheReal,

Increasing workspace looks like the only solution.
Some TensorRT algorithms require additional workspace on the GPU. Applications should therefore allow the TensorRT builder as much workspace as they can afford; at runtime TensorRT will allocate no more than this, and typically less.
However this is a Info message conveying that this might not be the best optimized model based on workspaceSize availability.
Thanks!

1 Like

@AakankshaS Thanks for your feedback, I’ll update this post if I discover anything else.

@Htut Thanks for letting me know, I’ll give that a try when i have some time.