Xavier NX Deepstream Docker image on Azure IoT Edge works with nvidia-container but not with Moby

I have a system based on the Xavier NX platform (specifically, a Lenovo ThinkEdge SE70 running Ubuntu 20.04 and JetPack 5.0.1) and a Docker image built on top of the nvcr.io/nvidia/deepstream-l4t:6.1.1-base image.

When I install the nvidia-container meta package (per the instructions here) from the https://repo.download.nvidia.com/jetson/common r35.1 and https://repo.download.nvidia.com/jetson/t194 r35.1 repos, running the module with the following deployment manifest in IoT Hub works:

"inference": {
    "version": "0.0.1",
    "imagePullPolicy": "on-create",
    "restartPolicy": "always",
    "settings": {
        "image": "reg.azurecr.io/inference:latest",
        "createOptions": {
            "HostConfig": {
                "runtime": "nvidia",
                "Binds": [
    "status": "running",
    "type": "docker"

However, Azure IoT Edge production checklist explicitly states that: “Only moby-engine is supported in production.”

With that in mind, on a fresh install if, instead of using the nvidia-container package I use the recommended moby-engine package (and install nvidia-docker2 as well since it would appear that this is needed), the container starts, but seems to have no access to the GPU.

I can find no instructions for getting a Deepstream 6.1.1-based container, running in a container/module via IoT Edge Hub on moby-engine. The sample from Azure shows getting a Deepstream 5.1 app running, and uses the old method of nvidia-docker which mounts a bunch of stuff off the host, which I’m trying to avoid and wouldn’t work anyway without me installing Deepstream on the devices.

Other relevant reading:

  • This issue in the moby repo provides essentially no help.
  • https://docs.nvidia.com/metropolis/deepstream/DeepStream_6.1.1_Release_Notes.pdf states that: “For the Jetson platform, omit installation of the Moby packages. Moby is
    currently incompatible with NVIDIA Container Runtime.” Is this true? Are we relegated to using an unsupported runtime?
  • Neither this nor this recommendation on passing extra commands to the module/container startup either results in a container that doesn’t start at all and throws errors in the edgeAgent logs, or the same behaviour as above where the container starts but seems to have no access to the GPU so it crashes and restarts.
  • This issue talks about much the same thing and recommends e.g., sudo apt-get remove docker docker-engine docker.io containerd runc but that doesn’t work.
  • This repo seems to be the closest, but adding all these extra lines in deployment.template.json doesn’t work either.

Any help here would be appreciated.


Moving this topic to the DeepStream SDK forum for visibility.

We have taken the Moby Engine support requirement in our docker container roadmap.

That’s good to know. Do you know when that might be coming along?

The latest NVIDIA Container Toolkit v1.11.0 have supported Moby Engine. Please follow Installation Guide — NVIDIA Cloud Native Technologies documentation for installation.

That linked site does not mention moby engine support and does not provide information on configuring it or running containers with it.

Note that currently, installing nvidia-container metapackage from e.g., https://repo.download.nvidia.com/jetson/common r35.1 InRelease or https://repo.download.nvidia.com/jetson/t194 r35.1 InRelease installs version 5.0.2-b231 per apt-cache show nvidia-container, and provides the following dependencies which in my case were automatically installed: nvidia-docker2 (= 2.11.0-1), libnvidia-container0 (= 0.11.0+jetpack), libnvidia-container-tools (= 1.10.0-1), nvidia-container-toolkit (= 1.11.0~rc.1-1), libnvidia-container1 (= 1.10.0-1), nvidia-container-runtime (= 3.9.0-1). Note nvidia-container-toolkit (= 1.11.0~rc.1-1).

The latest NVIDIA Container Toolkit v1.11.0 have supported Moby Engine.

Please consult Azure.

The latest NVIDIA Container Toolkit v1.11.0 have supported Moby Engine.

I understood this from your previous post. However, the link you provided in the same post:

Installation Guide — NVIDIA Cloud Native Technologies documentation

does not provide instructions on setting up for moby engine, only docker.

Please update the documentation, or provide similar setup instructions for moby engine to those that exist there for docker here in this forum.

You can install the Moby Engine as it is.

After installing Moby Engine, you can install NVIDIA Container Runtime package succesfully since the latest v1.11.0 is compatible to Moby Engine.

After following the documentation linked here Installation Guide — NVIDIA Cloud Native Technologies documentation for docker (but with moby-engine), containers are now working as expected.

Request that the linked document be updated to indicate moby-engine support, and the pdf https://docs.nvidia.com/metropolis/deepstream/DeepStream_6.1.1_Release_Notes.pdf similarly.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.