Question about DISPLAY and docker

[tom_s]

jetson@jetson:/data/src/jetsonUtilities$ ./jetsonInfo.py 
NVIDIA Jetson Nano (Developer Kit Version)
 L4T 32.7.1 [ JetPack UNKNOWN ]
   Ubuntu 18.04.6 LTS
   Kernel Version: 4.9.253-tegra
 CUDA 10.2.300
   CUDA Architecture: 5.3
 OpenCV version: 4.1.1
   OpenCV Cuda: NO
 CUDNN: 8.2.1.32
 TensorRT: 8.2.1.8
 Vision Works: 1.6.0.501
 VPI: ii libnvvpi1 1.2.3 arm64 NVIDIA Vision Programming Interface library
 Vulcan: 1.2.70

Deepstream 6.0.1
question

On the base install of the jetson i have a DISPLAY=:0
and when i run

 gst-launch-1.0 -v videotestsrc ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd  ! appsink

it works alright ( i know it doesn’t do anything interesting, but it works)

I then startup a docker container like:

 docker run --rm -it --name=test1 --cap-add=SYS_PTRACE --cap-add=sys_nice --runtime=nvidia --privileged --net=host -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v $HOME/.Xauthority:/root/.Xauthority nvcr.io/nvidia/deepstream-l4t:6.0.1-samples 

and i run the same pipeline it complains about

No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified

and
WARNING: erroneous pipeline: no element "nvvideoconvert"

full log:

jetson@jetson:/data/src/dockers$ docker run --rm -it --name=test1 --cap-add=SYS_PTRACE --cap-add=sys_nice --runtime=nvidia --privileged --net=host -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v $HOME/.Xauthority:/root/.Xauthority nvcr.io/nvidia/deepstream-l4t:6.0.1-samples 
root@jetson:/opt/nvidia/deepstream/deepstream-6.0# clear

root@jetson:/opt/nvidia/deepstream/deepstream-6.0# gst-launch-1.0 -v videotestsrc ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd  ! appsink
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:20): GStreamer-WARNING **: 03:10:51.877: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:21): GStreamer-WARNING **: 03:10:51.930: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection

(gst-plugin-scanner:32): GStreamer-WARNING **: 03:10:53.035: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection
No protocol specified
nvbuf_utils: Could not get EGL display connection
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 205)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)
WARNING: erroneous pipeline: no element "nvvideoconvert"
root@jetson:/opt/nvidia/deepstream/deepstream-6.0# printenv
DS_REL_PKG_DIR=deepstream_sdk_v6.0.1_jetson
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64
DISPLAY=:0
TAG=gmc1
HOSTNAME=jetson
DS_REL_PKG=deepstream_sdk_v6.0.1_jetson.tbz2
NVIDIA_VISIBLE_DEVICES=all
DS_VERSION=6.0.1
PWD=/opt/nvidia/deepstream/deepstream-6.0
HOME=/root
TERM=xterm
PLATFORM=jetson
NVIDIA_DRIVER_CAPABILITIES=all
SHLVL=1
PATH=/usr/local/cuda-10.2/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/printenv
root@jetson:/opt/nvidia/deepstream/deepstream-6.0# 

can anyone see something wrong here ?

[tom_s]

i changed nvvideoconvert with nvvidconv (is this the one that is compatible with deepstream?) and it still doesn’t like it :

root@jetson:/opt/nvidia/deepstream/deepstream-6.0# gst-launch-1.0 -v videotestsrc ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvdsosd  ! appsink
No protocol specified
nvbuf_utils: Could not get EGL display connection
WARNING: erroneous pipeline: no element "nvdsosd"

the library is in the docker container :

root@jetson:/opt/nvidia/deepstream/deepstream-6.0#  ls /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_osd.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_osd.so

but it can’t load:

root@jetson:/opt/nvidia/deepstream/deepstream-6.0# gst-inspect-1.0 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_osd.so
No protocol specified
No EGL Display 
nvbufsurftransform: Could not get EGL display connection

if I then remove unset the DISPLAY environment variable, it detects it

root@jetson:/opt/nvidia/deepstream/deepstream-6.0#  gst-inspect-1.0 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_osd.so
Plugin Details:
  Name                     nvdsgst_osd
  Description              Gstreamer plugin to draw rectangles and text
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_osd.so
  Version                  6.0.1
  License                  Proprietary
  Source module            nvdsosd
  Binary package           GStreamer nvosd Plugin
  Origin URL               http://nvidia.com/

  nvdsosd: NvDsOsd plugin

  1 features:
  +-- 1 elements

but

root@jetson:/opt/nvidia/deepstream/deepstream-6.0# gst-inspect-1.0 nvdsosd    
No such element or plugin 'nvdsosd'

update
i needed to clear the cache after unsetting DISPLAY

[tom_s]

I’m still confused why nvvideoconvert doesn’t work as

nvvidconv vs nvvideoconvert (DS 4.0 Nano)

What is the difference between the two? Also, using nvvideoconvert in a pipeline causes this: (gst-launch-1.0:16102): GLib-GObject-WARNING **: 10:05:56.928: cannot register existing type ‘GstInterpolationMethod’ (gst-launch-1.0:16102): GLib-GObject-CRITICAL **: 10:05:56.928: g_param_spec_enum: assertion ‘G_TYPE_IS_ENUM (enum_type)’ failed (gst-launch-1.0:16102): GLib-GObject-CRITICAL **: 10:05:56.928: validate_pspec_to_install: assertion ‘G_IS_PARAM_SPEC (pspec)’ failed

it says that nvvideoconvert is the ‘unified’ convert plugin that should be used in case of a deepstream pipeline. Isn’t my example a deepstream pipeline ?
but in this case only nvvidconv works.

How do i decide which convert plugin to use ?

[Amycao]

@tom_s

On the base install of the jetson i have a DISPLAY=:0
and when i run

 gst-launch-1.0 -v videotestsrc ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd  ! appsink

it works alright ( i know it doesn’t do anything interesting, but it works)

Seems nvvideoconvert works in device environment, but not work in container? nvvidconv works for your example within container? did you find the osd plugin?

[tomschuring]

@Amycao

Seems nvvideoconvert works in device environment, but not work in container? nvvidconv works for your example within container? did you find the osd plugin?

yes i’ve found nvoverlaysink

how would i decide if I should use nvvideoconvert and when nvvidconv ?
Is there a difference between the 2, how to choose ?

can one (which one) only be used if there is any deepstream component in the pipeline ?
if so, is there a list of deepstream components that i can check against ?

currently I read a lot of advice like ‘why don’t you try the other one’ …which makes it feel like it is a black-art…

[mchi]

@tomschuring

how would i decide if I should use nvvideoconvert and when nvvidconv ?
Is there a difference between the 2, how to choose ?

Hi @tomschuring , please find the DS plugin in GStreamer Plugin Overview — DeepStream 6.3 Release documentation and also a FAQ - DeepStream SDK FAQ - #15 by Fiona.Chen
nvvidconv is imcompatilble with DS plugin because the buffer type used in nvvidconv is different from the buffer type used on DS plugin.