Container with different CUDA version on Jetson Orin NX

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
,
1

Hi,

I have an application developed with CUDA 12.1 and TensorRT 8.6.1.6.
Now I want to use it on my Jetson ORIN NX. On my Jetson I have
JetPack 5.1.1 / L4T 35.3.1
CUDA 11.4, TensorRT 8.5.2

Is it possible to use my application in a container with CUDA 12.1 and TensorRT 8.6.1.6 on that Jetson? On my host system I use nvcr.io/nvidia/tensorrt:23.06-py3 to run the application. Or do I need to change my application so that it runs with the version of CUDA and TensorRT that are installend on the Jetson?

Thank you

3

Hi,

We have CUDA 12.2 that can work on JetPack 5.
But unfortunately, TensorRT 8.6 is upgradable from JetPack 6.

So if you need TensorRT 8.6, please reflash your system into JetPack 6.

Thanks.

4

JetPack 6.0 uses Jetson Linux 36.3. Will nvcr.io/nvidia/l4t-base:r36.2.0 container work for my application? A 36.3 version is not yet released.

5

Hi,

l4t-base:r36.2.0 cannot work on r35 environment due to driver dependency.
Thanks.

6

I flashed Jetpack 6.0 on my jetson. Now I have CUDA 12.2 and TensorRT 8.6.2

I usually build my application on a workstation/pipeline inside an arm64 container. I can easily build everything using nvcr.io/nvidia/tensorrt:23.10-py3 as arm64. The output also runs on the Jetson. But there is one problem: nvcr.io/nvidia/tensorrt:23.10-py3 has TensorRT 8.6.1.6 and on my Jetson there is TensorRT 8.6.2. So when creating my TensoRT engine, I need 8.6.2 to run it. TensorRT 8.6.2 is not available, when I look at :: (nvidia.com)

So I am still facing the same problem. I can only build and deploy with TensorRT 8.6.1.6 and on my Jetson there is 8.6.2. When I use L4T tensorRT 8.6.2 container to build on my workstation I get these errors:

/usr/bin/ld: warning: libnvdla_compiler.so, needed by /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)                  
/usr/bin/ld: warning: libcudla.so.1, needed by /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)                                                                                                              
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaDeviceGetCount@libcudla.so.1'                                            
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setUseSoftMaxOptz(bool)'                                    
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'                                                                                                                                                                                    /usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaSubmitTask@libcudla.so.1'                                                
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'                                        
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaModuleGetAttributes@libcudla.so.1'                                       
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaModuleUnload@libcudla.so.1'                                              
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaModuleLoadFromMemory@libcudla.so.1'
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReshape(nvdla::ITensor*, nvdla::Dims4)'                   
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addSlice(nvdla::ITensor*, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::SliceLayerMode)'                                                                                                                                           
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'                 
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaCreateDevice@libcudla.so.1'                                              
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'                                      
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'                                                        
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaDestroyDevice@libcudla.so.1'                                             
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createNetwork()'                                                       
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'                 
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'                                                                                                                                                                                              /usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)'                                          
/usr/bin/ld: /usr/lib/gcc/aarch64-linux-gnu/11/../../../aarch64-linux-gnu/libnvinfer.so: undefined reference to `cudlaMemRegister@libcudla.so.1'                                               
collect2: error: ld returned 1 exit status