EGX - `nvidia` docker runtime on Nano

[mdegans]

The EGX Platform

@marketing droid

EGX starts with the tiny NVIDIA® Jetson Nano™

…

NVIDIA Edge Stack is an optimized software stack that includes NVIDIA drivers, a CUDA® Kubernetes plug-in, a CUDA Docker container runtime

This was announced yesterday. Bolded is the particular pony I have been wishing for. Is there a beta image or apt package available for the nvidia docker runtime on Nano?

[kayccc]

Hi mdegans,

it’s coming, and stay tuned to our announcements in the coming month. In the meantime, there are existing recipes for running GPU-accelerated workloads within standard Docker containers—it is a bit of additional work, but you can get started today using one of the community-supplied recipes such as this one: [url]https://blog.hypriot.com/post/nvidia-jetson-nano-intro/[/url]

Thanks

[mdegans]

Thanks for the timeline! I will wait for EGX and prototype any containers on x86 while I wait.

[prlawrence]

mdegans,

nvidia-docker will be included with JetPack 4.2.1, to be released first week of July. :- )

[mdegans]

@prlawrence

mdegans,
nvidia-docker will be included with JetPack 4.2.1, to be released first week of July. :- )

Thank you for the update! I appreciate it.

[tibo.durand]

Awesome ! If you want some beta test for JetPack 4.2.1 , I’m available ;-)

[HelmutHofferVonAnkershoff]

@prlawrence

mdegans,
nvidia-docker will be included with JetPack 4.2.1, to be released first week of July. :- )

Thank you for the update! I appreciate it.

Will Jetpack 4.2.1 arrive today as planned?

[snarky]

Earlier this week I saw them say “next week” in another post.
Note that yesterday was a major holiday in the US, and today a lot of people are taking the day off to make it a four-day holiday.

[mdegans]

Guessing last minute commits break everything. It happened to my spouse right before the 4th. He was stuck on his laptop half the day as a result. I’m fine with the delay for 4.2.1 personally. Stuff happens, and I would rather it be stable.

[Rusli]

I also want JetPack 4.2.1. Please :)

[HelmutHofferVonAnkershoff]

@prlawrence: Any update on the release date?

P.S.: While waiting for the Nvidia Docker runtime I automated provisioning Jetson devices to join Kubernetes clusters including automatically building a fitting kernel for weave networking and using the mount /dev/nv* approach to support cuda - see https://github.com/helmuthva/jetson

[prlawrence]

@mdegans

…I’m fine with the delay for 4.2.1 personally. Stuff happens, and I would rather it be stable.

@zaripov.r.z

I also want JetPack 4.2.1. Please :)

@helmuthva

@prlawrence: Any update on the release date?

We are fixing a couple last bugs, so as you can tell the schedule has unfortunately slipped. We’ll update again when the release is finalized – JetPack 4.2.1 has lots of great updates and we look forward to sharing it.
https://devtalk.nvidia.com/default/topic/1055628/jetson-nano/new-jetson-software-modules-and-pricing/post/5360588/#5360588

@helmuthva

P.S.: While waiting for the Nvidia Docker runtime I automated provisioning Jetson devices to join Kubernetes clusters including automatically building a fitting kernel for weave networking and using the mount /dev/nv* approach to support cuda - see GitHub - helmut-hoffer-von-ankershoffen/jetson: Helmut Hoffer von Ankershoffen experimenting with arm64 based NVIDIA Jetson (Nano and AGX Xavier) edge devices running Kubernetes (K8s) for machine learning (ML) including Jupyter Notebooks, TensorFlow Training and TensorFlow Serving using CUDA for smart IoT.

Nice! I’m sure many others will want to try something like this, thanks for taking the time to document your work. :- )

[HelmutHofferVonAnkershoff]

@prlawrence: By now https://github.com/helmuthva/jetson contains a kubernetes deployment of a Jupyter server supporting CUDA accelerated Tensorflow running on Jetson nodes (such as Nanos and Xaviers).

To achieve this the container has to mount closed source libraries such as cudnn from the Jetson host, see https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/kustomize/base/deployments.yaml and https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/src/Dockerfile

Question: Is this the assumed way or will base images containing the cuda and cudnn libraries be provided as part of the Nvidia Docker runtime in Jetpack 4.2.1?

P.S.: For the ones interested in getting Anaconda running on Jetson - it’s part of the deployment, see the Dockerfile.

[prlawrence]

@HelmutHofferVonAnkershoffen

…
To achieve this the container has to mount closed source libraries such as cudnn from the Jetson host, see https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/kustomize/base/deployments.yaml and https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/src/Dockerfile

Question: Is this the assumed way or will base images containing the cuda and cudnn libraries be provided as part of the Nvidia Docker runtime in Jetpack 4.2.1?

The “l4t-base” image will be very slim – libraries will be mounted from the host to the container. See:
https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-Container-Runtime-on-Jetson#mount-plugins

[mdegans]

@prlawrence

@HelmutHofferVonAnkershoffen

…
To achieve this the container has to mount closed source libraries such as cudnn from the Jetson host, see https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/kustomize/base/deployments.yaml and https://github.com/helmuthva/jetson/blob/master/workflow/deploy/jupyter/src/Dockerfile

Question: Is this the assumed way or will base images containing the cuda and cudnn libraries be provided as part of the Nvidia Docker runtime in Jetpack 4.2.1?

The “l4t-base” image will be very slim – libraries will be mounted from the host to the container. See:
https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-Container-Runtime-on-Jetson#mount-plugins

-v /usr/local/cuda:/usr/local/cuda

Is there a reason this isn’t this included in the image? I really don’t like to bind mount, in part because of container escapes, and in part becuase i’ve had issues using it with swarm.

Edit: Apparently there is a reason given in the link: image size, and the problem is being worked on

Edit, also:

The NVIDIA software stack, so that it can ultimately run GPU code, talks to the NVIDIA driver through a number of userland libraries (e.g: libcuda.so). Because the driver API is not stable, these libraries are shipped and installed by the NVIDIA driver.

In effect, what that means is that having a container which contains these libraries, ties it to the driver version it was built and ran against. Therefore moving that container to another machine becomes impossible. The approach we decided to take is to mount, at runtime, these libraries from your host filesystem into your container.

Other than that, thank you for the beta. Very cool!

[HelmutHofferVonAnkershoff]

@prlawrence, @mdegans - sounds promising, thx for the input.

Btw: The page referenced by you contains typos/inconsistencies and i guess the “l4t-base” image is not yet published, see NVIDIA Container Runtime on Jetson / Typos · Issue #1013 · NVIDIA/nvidia-docker · GitHub ?

[ralf_s]

I’m happy to report that GPU accelerated Docker works fine on my Nano running Jetpack 4.2.1. As Helmut mentioned, there’s an issue with pulling the image from the Nvidia cloud, so I created my own images. If you’re interested, see here: https://github.com/jetsistant/docker-cuda-jetpack. I tried almost all CUDA samples in the devel container, they are all running flawlessly. Thanks for this great release!

[HelmutHofferVonAnkershoff]

The nvcr.io/nvidia/l4t-base:r32.1 image was published, see NVIDIA L4T Base | NVIDIA NGC :-)

[HelmutHofferVonAnkershoff]

For the ones interested in running Jetson edge devices as part of a larger multi-platform Kubernetes cluster: I’m happy to report that JetPack 4.2.1 works with Kubernetes + Weave Networking just fine after minor tweaks. Thank you NVIDIA for this wonderful release :-)

Have a look at jetson/.config at master · helmut-hoffer-von-ankershoffen/jetson · GitHub for the necessary kernel configuration - or https://github.com/helmuthva/jetson for the bigger picture.