Running deepstream sample apps in docker container requires host networking and root user

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I have a Xavier NX dev board with jetpack 4.6.1 (deepstream 6.0.1) I’ve pulled the nvcr.io/nvidia/deepstream-l4t:6.0.1-samples and built a dev environment on top of it (installed gcc, cmake, cuda enabled opencv, ros2 and a few other libraries). I don’t have a keyboard / monitor connected to the jetson. Typically there are two ways I connect to the container–with vscode (via ssh) to build code or over VNC (to run programs or otherwise interact with the desktop). In the dev container I’ve also added a non-root user with sudo privileges. I start the container via

docker run -it -d --restart unless-stopped \
    --name=dev_container \
    --group-add video \
    -v /tmp/.X11-unix:/tmp/.X11-unix:ro \
    -v ${HOME}/dev:/workspaces/vscode \
    -v ~/.ssh/id_rsa:/home/vscode/.ssh/id_rsa:ro \
    -v ~/.gitconfig:/etc/gitconfig:ro  \
    --env="DISPLAY" \
    -p 8022:22 -p 9001:9001 \
    my_dev_image:latest /bin/bash

(it’s configured to start in /workspaces/vscode with the non-root user using nvidia as the default runtime). The non-root user is also configured to have the same uid / gid as me or whoever is using the container (so all the permissions work out how they should). I can run the program xeyes fine to show that X11 forwarding from the container and gui apps at least appear to be working, however when I run the deepstream sample app (from a terminal on the vnc desktop):

vscode@dd9f3c74e486:/workspaces/vscode ()
$ rm -rf ~/.cache/gstreamer-1.0/
vscode@dd9f3c74e486:/workspaces/vscode ()
$ deepstream-app -c /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/source30_1080p_dec_infer-resnet_tiled_display_int8.txt
nvbuf_utils: Could not get EGL display connection
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 205)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:1790): GStreamer-WARNING **: 08:15:21.222: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:1791): GStreamer-WARNING **: 08:15:21.267: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:1792): GStreamer-WARNING **: 08:15:21.323: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
nvbufsurftransform: Could not get EGL display connection
** ERROR: <create_multi_source_bin:1424>: Failed to create element 'src_bin_muxer'
** ERROR: <create_multi_source_bin:1517>: create_multi_source_bin failed
** ERROR: <create_pipeline:1326>: create_pipeline failed
** ERROR: <main:688>: Failed to create pipeline
Quitting
App run failed

If I run the same command outside of the dev container, the app runs as expected. I’ve been through

and

as well as a few others which seem very similar. If I run the base container directly in host mode, i.e.

vscode@nx02:~ ()
$ xhost +
access control disabled, clients can connect from any host
vscode@nx02:~ ()
$ docker run -it --rm --net=host --runtime=nvidia -w /opt/nvidia/deepstream/deepstream-6.0 -v /tmp/.X11-unix:/tmp/.X11-unix:ro --env="DISPLAY"  nvcr.io/nvidia/deepstream-l4t:6.0.1-samples /bin/bash
root@nx02:/opt/nvidia/deepstream/deepstream-6.0# echo $DISPLAY
:0
root@nx02:/opt/nvidia/deepstream/deepstream-6.0# deepstream-app -c /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/source30_1080p_dec_infer-resnet_tiled_display_int8.txt

It also works. Narrowing things down, I have also gotten it to work with my development container if two conditions are met–I’m in host networking mode and I’m the root user, both of which are not ideal (host networking makes it so I can’t ssh into the container, so I can’t get to the container from vscode, running as root is generally not good practice and also messes with the permissions of the mounted dev directory).

Is there a way to run the apps (and still see the annotated output) while not in host mode or running as root?

2

I also tried just setting sink0 to fake_sink just to see if I could run it without the display in my normal dev container (not host networking and not root user) and I get a batch of other errors:

deepstream-app -c /opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/source30_1080p_dec_infer-resnet_tiled_display_int8.txt
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
Error: Can't initialize nvrm channel
Error: Can't initialize nvrm channel
Couldn't create ddkvic Session: Cannot allocate memory
nvbuf_utils: Could not create Default NvBufferSession
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 

(gst-plugin-scanner:246): GStreamer-WARNING **: 09:42:38.011: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
nvbufsurftransform:cuInit failed : 100 

(gst-plugin-scanner:247): GStreamer-WARNING **: 09:42:38.036: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
nvbufsurftransform:cuInit failed : 100 

(gst-plugin-scanner:248): GStreamer-WARNING **: 09:42:38.073: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
nvbufsurftransform:cuInit failed : 100 
** ERROR: <create_multi_source_bin:1424>: Failed to create element 'src_bin_muxer'
** ERROR: <create_multi_source_bin:1517>: create_multi_source_bin failed
** ERROR: <create_pipeline:1326>: create_pipeline failed
** ERROR: <main:688>: Failed to create pipeline
Quitting
App run failed
3

I just happened upon this post: Running accelerated gstreamer in non-privileged docker container - #4 by arnim.balzer. When I add the devices listed on that post to my docker command it appears I no longer need to be root. I haven’t tried to see if that addresses having to be on host networking, but my guess is it doesn’t. I guess that is the final part I’m trying to solve–how to not have to be in host networking mode.

5

You can add nvidia user into docker group to not use root user.
sudo usermod -aG docker nvidia

6

Hi @Amycao, sorry I should’ve clarified, I had to be root inside the container. I have a non-root user inside my container with the same uid/gid as the user outside the container so the files in the mounted development directory all have the correct permissions when accessed outside the container. When I originally posted, I was unable to use the non-root user inside the container, but adding the devices to my docker run command appears to have made it so I can. You are correct though, outside the container my user is a member of the docker group.

The last issue I’d like to try to fix is the issue of having to be in host networking mode. My container is set up so I can use normal X11 apps just fine (xeyes, gitk, opencv gui elements) without being in host networking, but apparently that’s not still not sufficient for EGL graphics (which to be honest I’m not sure what is)

7

Did your device in headless mode or connected with monitor?
Running a DeepStream application over SSH (via putty) with X11 forwarding does not work.

8

I don’t have a monitor or keyboard connected directly to the Jetsons, but rather I have them setup so on startup they launch a vnc server (x11vnc).

Depending on what I’m doing I connect to the Jetson in a couple of different ways–either ssh with X-forwarding or in this case via VNC so I can see the entire desktop (I had tried running over ssh with X-forwarding and as you mention it didn’t work…bummer, but not too big of a deal). My question here involves the second case–connecting to the Jetson via a VNC connection.

9

I think you need a monitor to make the X11 server working on Jetson. for headless mode, suggest you use rtsp streaming for output.

10

The VNC server functions kind of like a monitor–i.e it provides a desktop environment and everything as if a monitor were connected. When the docker container is in host mode I can run the sample pipelines with displays over VNC no problem, so I don’t think having a VNC server desktop as opposed to a physical monitor hooked up is the issue. My guess is that if I hooked up a physical monitor (which I guess I can try) I would have the exact same issue.

12

sorry for the late reply, Is this still an issue to support? Thanks

13

Hi @fanzh–yes I’m still trying to resolve this issue (i.e. why the need to be in host networking mode to get the output display to work–I would rather not be in host networking mode.)

Thanks

14

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

  1. please hook up a physical monitor because deepstream will initialize failed if no physical monitor.
  2. if having physical monitor, you can use nomachine to see the output display, we have verified.
16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.