[TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

When I convert the onnx model using trtexec within the tar file with this command
./trtexec --explicitBatch --onnx=duke_onnx.onnx --minShapes=input:1x3x288x144 --optShapes=input:5x3x288x144 --maxShapes=input:30x3x288x144 --saveEngine=duke_more_space_output_16_2.engine

It gives me the mentioned error " [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output." and reduced the accuracy of the model , So I think the accuracy of the model is directly related to that error, how could I give the workspace more size with that conversion method ?

** I’ll set some sinppet from the terminal
• Hardware Platform (GPU)
• DeepStream Version: 5.0
• TensorRT Version: 7.0.0.11

image
image1901Ă—674 132 KB

image
image1826Ă—597 221 KB

3

Hi @ayanasser

There is --workspace option to give it more workspace memory in MB.

–workspace=N Set workspace size in megabytes (default = 16)

4

Where to find that option ?
I am using TRT through tarfile

5

–workspace=N is an option of trtexec. you can find it from “./trtexec --help”

6

Thank you, Is this helping with increasing the accuracy right ?

7

I don’t think it helps accuracy

8

Thank you ^^,
But do you have any ideas that helping with the model accuracy as it converted to engine ?

9

By increasing workspace size, did you still see the warning? And, how about the accuracy?

What difference causes the accuracy drop?

10

The warning is still here, even I made the workspace=2000 MB

11

could you share me a screenshot?

12

image
image1846Ă—823 183 KB

image
image1758Ă—875 327 KB

image
image1661Ă—438 149 KB

13

could you run trtexec with “–verbose” and check the verbose log?

1 Like
14

It’s a huge, The terminal didn’t fit the whole output, so the output is at this txt file

verbose.txt (869.2 KB)

15

I’ve used 2080 RTX super that has 12 GB RAM, I’ve gave it workspace of 8 GB for conversion with maximum output shape 2 streams (2 batch size), and here’s the command :
./trtexec --explicitBatch --onnx=duke_onnx.onnx --minShapes=input:1x3x288x144 --optShapes=input:1x3x288x144 --maxShapes=input:2x3x288x144 --saveEngine=duke_8g_fp16.engine --workspace=8192 --fp16

But it’s still give me those errors:
[08/09/2021-07:33:35] [W] [TRT] Calling isShapeTensor before the entire network is constructed may result in an inaccurate result.

16

Hi @ayanasser ,
Could you use DS 5.1 with TRT 7.2.X?
Seems we have some TRT fix for this issue.

Thanks!