Running nvidia/tensorrt on Jetson

[BenButterworth]

Is it possible to run nvidia/tensorrt containers on Jetson machines? It should be possible, if you follow NVIDIA Container Runtime on Jetson.

I want to do this so that the development environment (Docker running on macOS, windows, linux) is similar to the deployment environment (Jetson). Unfortunately, my application is crashing with pycuda._driver.Error: cuInit failed: unknown error when my application (and the python repl) initializes cuda with:

import pycuda.driver as cuda
cuda.init()

In this case, I am not setting the runtime: nvidia. When I do that, I get a different error: Jetson: `libcudnn_adv_infer_static_v8.a: file exists: unknown` error · Issue #1598 · NVIDIA/nvidia-docker · GitHub

I’m wondering if there are fundamental reasons this is happening.

Unfortunately, I cannot run l4t-base, l4t-cuda or l4t-tensorrt on my development machine because they don’t support multiple architectures. See TensorRT L4T docker image Python version Issue - #17 by mdegans for good reasons why we should have this.

[AastaLLL]

Hi,

Sorry for the late reply.

May I know how do you set up the Jetson first?
Do you use JetPack4.6? If not, would mind giving it a try?

Thanks.

[BenButterworth]

Hey @AastaLLL,

I’m using Jetpack 4.5.1. I don’t think there is anything that would be solved by upgrading to Jetpack 4.6.

[AastaLLL]

Hi,

Please give it a try.
It will need JetPack4.6 for some compatibility issues.

The container is built on the top of l4t-cuda and you can find this constraint in its document as below:

https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-cuda

The image is tagged with the version corresponding to the CUDA release version. Based on this, the l4t-cuda:r10.2.460-runtime container is intended to be run on devices running JetPack 4.6 which supports CUDA version 10.2.460

Thanks.

[BenButterworth]

I just upgraded to 4.6, but unfortunately now docker fails to start, even after reinstalling it.

I get the same issue as: Docker error "failed to mount overlay: no such device" storage-driver=overlay2

Running sudo dockerd gets me:

failed to mount overlay: no such device       storage-driver=overlay2

The full output:

INFO[2022-02-23T16:33:53.976453799+01:00] Starting up
WARN[2022-02-23T16:33:53.976690154+01:00] Running experimental build
INFO[2022-02-23T16:33:53.980197780+01:00] parsed scheme: "unix"                         module=grpc
INFO[2022-02-23T16:33:53.980279381+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2022-02-23T16:33:53.980365590+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2022-02-23T16:33:53.980404663+01:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2022-02-23T16:33:53.983147128+01:00] parsed scheme: "unix"                         module=grpc
INFO[2022-02-23T16:33:53.983230809+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2022-02-23T16:33:53.983292538+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2022-02-23T16:33:53.983329338+01:00] ClientConn switching balancer to "pick_first"  module=grpc
ERRO[2022-02-23T16:33:53.990005739+01:00] failed to mount overlay: no such device       storage-driver=overlay2
ERRO[2022-02-23T16:33:53.990079980+01:00] [graphdriver] prior storage driver overlay2 failed: driver not supported
failed to start daemon: error initializing graphdriver: driver not supported