TensorRT context consume mush memory on Orin

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
,
#1

Hi,
I modified a superGLue model for a image matching project and the tensorRT engine works well ,the superGlue trt engine’s result is correct. but the problem is that the engine counsume too much memory! my device is Orin with 32GB memory. the superGLue engine context consume 24G+ memory !!!. on my other x64 server with GPU A6000, the engine only need 9GB memory. that’s a big problem. please help me, thanks a lot.

Jetson AGX Orin enviroment:
CUDA 11.4
cudnn 8.3.2
TensorRT 8.4.0

x64 server with GPU A6000:
CUDA 11.1
cudann 8.2.1
TensorRT 8.2.1

#3

Hi,

Have you tried to set up the workspace parameter?
TensorRT will choose an algorithm based on the available memory that the parameter is allowed.

For example, with trtexec:

$ /usr/src/tensorrt/bin/trtexec --workspace=8192 ...

Thanks.

#4

Hi， I had tried many workspace value from 1<<20 to 1<<33, but it doesn’t work.

#5

Hi,

Could you share the TensorRT output log with us as well?
Thanks.