Docker build error "failed to create shim: OCI runtime create failed..."

I am trying to create a container based off of Jetson-Inference for a project I’m doing with the Jetson Nano. I have been able to successfully run the container following the instructions in the walk through and have already run several examples, including imagenet, detectnet, and segnet.

Now I’m trying to build a container based off of this one to begin developing the code for my own project, and I’ve started by making the following docker file:

# Specify the parent image from which we build
FROM dustynv/jetson-inference:r32.4.3

# setup dependancies needed inside the container
RUN set -ex;                                                \
    apt-get update;                                           

# Set the working directory inside the container
WORKDIR /app

# Copy files from your host to the working directory in the container
COPY ./container_src container

# Build the application with cmake inside the container
RUN set -ex;                                                \
    cd /app/container/build;                                \
    cmake ..;                                               \
    make;

when I execute my build command

sudo docker build . -t <dockeruser>/<repo>:tag

I get the following error message

failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: mount error: file creation failed: /var/lib/docker/overlay2/b54f3cd6ae01d2cfe046d4f330bbac6f125e1ccd438e08f6a45d39632228fbb3/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-fatbinaryloader.so.440.18: file exists: unknown

For the record, if my docker file only contained “FROM dustynv/jetson-inference:r32.4.3” and nothing else I would have a successful build, but I would get a similar error if I then tried to run that container.

As I looked at other posts that were shown as similar, but none of them helped me resolve my issue. I’m tagging @dusty_nv as well since it is his repo that I’m basing my container off of. Thank you for taking time to read this!

Hi @crose72, are you indeed running L4T R32.4.3? (in /etc/nv_tegra_release)

Are you able to run other containers like l4t-base, l4t-pytorch, and jetson-inference still?

@dusty_nv
According to that file I’m running r32.7.2. As far as other containers, I have successfully run a container where the program was literally just a basic print to terminal the words “Hello world”. It was made using

FROM nvcr.io/nvidia/l4t-base:r32.7.1

And as far as jetson-inference, I followed the github tutorial and ran

sudo docker/build.sh

but then it had me manually specify a base container for L4T R32.7.2. So I then used the command

sudo docker/build.sh nvcr.io/nvidia/l4t-pytorch:r32.7.2

and got a “manifest unknown” error (I also tried specifying r32.7.1 and same thing).

EDIT:
I have flashed a new image onto a different sd card with r32.4.3, and using that image I cannot execute the run or build scripts in jetson-inference.

Hi @crose72, try using this tag for r32.7.* instead: nvcr.io/nvidia/l4t-pytorch:r32.7.1-pth1.10-py3

What’s the error you get when you try running docker/run.sh? There is a container image for r32.4.3 here: dustynv/jetson-inference:r32.4.3

Hi @dusty_nv, so for your first suggestion, and for the sd card with r32.7.2, I ran sudo docker/build.sh nvcr.io/nvidia/l4t-pytorch:r32.7.1-pth1.10-py3 and saw the following error on step 6/21

W: GPG error: https://apt.kitware.com/ubuntu bionic InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 42D5A192B819C5DA
E: The repository 'https://apt.kitware.com/ubuntu bionic InRelease' is not signed.

For the sd card with the r32.4.3 image, I received a “unary operator expected on lines 168 and 179” (phrased slightly differently) error after running the build script, and I believe it also was on the run script as well.

You shouldn’t need to build the jetson-inference container yourself (unless you want to). For L4T r32.7.2, use the dusty-nv/jetson-inference:r32.7.1 image

It would seem this error is related to this:

Try adding this to your docker file before apt update is run:

RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null \
    && echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ focal main' | tee /etc/apt/sources.list.d/kitware.list >/dev/null

Hi @dusty_nv, as far as building the container goes, I’m just trying to build my own container based off of yours, so I wanted to make sure I could successfully build yours first.

Alright so I tried running your suggested line before my apt-get update but still got the same error. I also tried to execute the other two commands in the post you linked
sudo rm /usr/share/keyrings/kitware-archive-keyring.gpg
sudo apt-get install kitware-archive-keyring
but to no avail.

So the conclusion I draw from that is if I want to build a container based off of yours (and also your container), I just can’t use apt-get update. I wonder if that will be a problem in the future.

OK, sorry about that I’ve since changed to upgrading cmake with pip instead of apt (so this isn’t an issue in more recent versions of the containers), and unfortunately I don’t have my L4T R32.7 system handy at the moment to cross-check this. The next thing I would try is checking the /etc/apt/sources.list and /etc/apt/sources.list.d in the container and removing the kitware repo (or adding [trusted=yes] to it).

This thread below seems to have some relevant suggestions and in a pinch you may be able to try apt-get update --allow-unauthenticated or apt-get update --allow-insecure-repositories

Hi @dusty_nv,

I’ll try that, how do I actually edit something like /etc/apt/sources.list in the container for each time I want to run the container? I’m guessing I’d want that file to be permanently changed in the container (unless I decide to manually change it back).

I’ll also try the forced update from an unsigned repository and see how that goes.

The easiest way would probably be to make the edits by hand (i.e. by running the nano editor inside container) and then doing a docker commit to save the state of the container image. Or you could write your own dockerfile that procedurally made the changes. Or you could recompile l4t-pytorch from the sources @ jetson-containers repo which have been updated.

I wasn’t able to get these forced updates to work, but running those commands from this post did work for me. Regardless, this GPG key issue has been fixed in the Dockerfiles in commit #e8a040, so you could rebuild the container if you wanted to.

@crose72 You need to replace “focal” with “bionic” in Dusty’s command if you are using a Jetpack based on Ubuntu 18.04, such as JP 4.6.2.

Woohoo! That worked! At least, I was able to run my docker build script, to the point where it made it past the apt-get update step. For all you folks watching at home, my docker file (not including additional commands specific to my project): ends up looking like this:

# Specify the parent image from which we build
FROM dustynv/jetson-inference:r32.7.1

RUN wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null \
    && echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ bionic main' | tee /etc/apt/sources.list.d/kitware.list >/dev/null

# setup dependancies needed inside the container (not apt-get update)
RUN set -ex;                                                \
    apt-get update;                                           

@jonl8t6c can you provide some explanation for the reason behind replacing “focal” with “bionic”?

Bionic and Focal are the short names of the Ubuntu 18.04 and 20.04 releases. You need to match the release used by your JetPack when instructing apt to use a source’s packages.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.