Jetson xavier, docker nvidia:l4t-pytorch image, usb_camera, trt_pose

Hi,

I am new to embedded systems/nvidia gpu computing/docker/this forum so I apologise up-front for any generally accepted comments regarded standard.

Simplified statement of the issue: I am trying to run trt_pose demo jupyter notebook on jetson xavier. Setup and run when installed natively on the jetson is quite straightforward. However when I try to replicate the demo within a docker container on the device I am running into problems where I’m simply unable to recognise the usb_camera when executing within the container.

Details:
JETSON OS/VERSION:

cat /etc/nv_tegra_release;
R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: t186ref, EABI: aarch64, DATE: Fri Oct 16 19:37:08 UTC 2020

Nvidia Pytorch/Pytorchvision Image pulled an installed:

https://ngc.nvidia.com/catalog/containers/nvidia:l4t-pytorch

trt_pose demo attempting to run in docker container hosted on github:

https://github.com/NVIDIA-AI-IOT/trt_pose

usb_camera:

Logitech, Inc. Webcam C270

1tb ssd mounted to ~/ssd:
970 EVO PLUS 1TB M.2 2280 PCI-E 3.0 X4 NVME SOLID STATE DRIVE

Originally I pulled the above docker image nvidia:l4t-pytorch. Activating and then running the image in new container I realised that the container is quite a stripped down version of the native jetapck flashed host device. Nevertheless I persisted with the container, installing all dependencies apparently absent. In terms of the dependencies required by the trt_pose demo there has been little or no issue, requiring

  • pytorch/pytorchvision (acquired by the prepackaged docker image)
  • torch2trt
  • jetcam

Both the trt_pose jupyter notebook live_demo.ipynb and the jetcam usb_camera.ipynb demo fall over when trying to construct USBCamera() giving the following error trace:

RuntimeError Traceback (most recent call last)
in
1 from jetcam.usb_camera import USBCamera
2
----> 3 camera = USBCamera(width=224, height=224, capture_width=640, capture_height=480, capture_device=1)

/usr/local/lib/python3.6/dist-packages/jetcam-0.0.0-py3.6.egg/jetcam/usb_camera.py in init(self, *args, **kwargs)
26 except:
27 raise RuntimeError(
—> 28 ‘Could not initialize camera. Please see error trace.’)
29
30 atexit.register(self.cap.release)

RuntimeError: Could not initialize camera. Please see error trace.

Attempts to Resolve/Diagnosis:
After a lot of googling I have tried many things to resolve this issue.

docker run variants
In terms of docker run flags I have tried, amongst many, attaching the device:

docker run -it --runtime nvidia --network host -p 8888:8888 --device=/dev/video0:/dev/video0 --name trtpose-l4t-pytorch nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

I have also tried bind-mounting other potential necessary drives/device and display:

docker run --privileged -p 8888:8888 --net=host --runtime nvidia --rm --ipc=host --device=/dev/video0:/dev/video0 -v /tmp/.X11-unix/:/tmp/.X11-unix/ -v /tmp/argus_socket:/tmp/argus_socket --cap-add SYS_PTRACE -e DISPLAY=$DISPLAY -it backup/trtpose-l4t-pytorch:backup_version

note trtpose-l4t-pytorch:backup_version is docker backup of pytorch docker image with some dependencies resolved in base image l4t-pytorch:r32.4.4-pth1.6-py3.

note also I access the notebook on host from container by launching jupyter lab from inside the contianer;

jupyter lab --ip 0.0.0.0 --no-browser --allow-root

diagnosis within the container
In almost all the attempts above (following other google searches and discussions) I have tried the following command to test the webcam within the container, which works outside the container:

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! xvimagesink;

however this returns the following error:

no element “xvimagesink”

Note when I run this command the light on the usb_camera comes on momentarily…

potential missing fundamental dependencies in container
I have then tried to manually install any other missing dependencies. For example in the container there seems to be an incomplete installation of gstreamer, which I have tried to resolve the issue with:

https://gstreamer.freedesktop.org/documentation/installing/on-linux.html?gi-language=c

another docker image?
I have also tried to run the trt_demo starting with another base docker image which I hoped would fill the gap on missing dependencies:

https://ngc.nvidia.com/catalog/containers/nvidia:l4t-ml

Unfortunately ending at the same road block.

services/pipelines
many attempts here trying to resolve such as restarting/running services/daemons and pipelines…

other posts:
Although my issue resembles other discussions online, such as the following link relating to Jetson Nanno which seems unresolved?:

Jupyter jetcam in docker, 'Could not initialize camera'

Bottom Line

The bottom line is, no matter what I have tried independently (highlighted above) I simply can’t access/run the usb_camera from within the docker container and am in need of guidance as this should in my mind be resolvable.

Please help!

Many thanks in advance

Amit

Hi,

Could you launch the container with the following comands:

$ export DISPLAY=:0
$ xhost +
$ sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

And run the gst-launch-1.0 with the instruction shared below first?

Thanks.

Hi @AastaLLL, thank you for getting back to me.

So if I understand correctly, before running the docker command, in terminal when I type

I get the following response in terminal:

Invalid MIT-MAGIC-COOKIE-1 keyxhost: unable to open display “:0”

Wild guess, but I’m not sure this is what we were expecting? Previously I noticed the default value of DISPLAY in jetson is :1

$ echo $DISPLAY
:1

As an experiment, I reset DISPLAY to oringinal :1 and then just ran in terminal

$ xhost +
access control disabled, clients can connect from any host

Then I ran the docker “run” command

sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

in this attached docker container when I run the following command I get

v4l2-ctl -d /dev/video0 --list-formats-ext
bash: v4l2-ctl: command not found

Hi,

It seems that your display is mounting at :1.
So you will need to use export DISPLAY=:1.

For the v4l2-ctl app, please install it with the following command:

apt-get install v4l-utils

Thanks.

Hi @AastaLLL, thanks for your response,

seems that the docker run command advised doesn’t specify “–device” so when I’m in container I have no “video0” under “/dev”. So I have altered the command above to add the device. I have tried the following (steps below verbatim). Please note I have previously created “docker” group on device so currently I don’t need sudo to use docker run. Again apologies up front if I’m doing something wrong as this is the first time I am using these tech/devices.

ATTEMPT 1, docker image: nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

When working with the nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3 image, in terminal a I run:

export DISPLAY=:1
xhost +
docker run --device=/dev/video0:/dev/video0 -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

then following advice at link
Logitech C930e on Jetson TX1 very slow and choppy video

in container when I run

apt-get install v4l-utils
errors over not having package in repo “Unable to locate package v4l-utils”

I have another docker image named “trtpose-l4t-pytorch:backup_version” which I created when trying to run the jetcam demo (what I’m trying to achieve) this is built off “nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3” image with dependencies added, so I tried again with this image (this has the repo missing above)

ATTEMPT 2, docker image: trtpose-l4t-pytorch:backup_version
working with this image gets me a bit further as it has the missing repo, now in terminal running docker run again with “device”

export DISPLAY=:1
xhost +
docker run --device=/dev/video0:/dev/video0 -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix backup/trtpose-l4t-pytorch:backup_version

again as advised following “Logitech C930e on Jetson TX1 very slow and choppy video” in container running :

apt-get install v4l-utils
v4l2-ctl -d /dev/video0 --list-formats-ext

does now recognise the device and lists (I believe) the formats available to the webcam

Blockquote

Hi @AastaLLL, thanks for your response,

seems that the docker run command advised doesn’t specify “–device” so when I’m in container I have no “video0” under “/dev”. So I have altered the command above to add the device. I have tried the following (steps below verbatim) also note I have previously created “docker” group so currently I don’t need sudo to use docker run. Again apologies up front if I’m doing something wrong as this is the first time I am using these tech/devices.

ATTEMPT 1, docker image: nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

When working with the nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3 image, in terminal a I run:

export DISPLAY=:1
xhost +
docker run --device=/dev/video0:/dev/video0 -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3

then following advice at link
Logitech C930e on Jetson TX1 very slow and choppy video

in container when I run

apt-get install v4l-utils
errors over not having package in repo “Unable to locate package v4l-utils”

I have another docker image named “trtpose-l4t-pytorch:backup_version” which I created when trying to run the jetcam demo (what I’m trying to achieve) this is built off “nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3” with dependencies added, so I tried again with this image (this has the repo missing above"

ATTEMPT 2, docker image: trtpose-l4t-pytorch:backup_version
working with this image gets me a bit further as it has the missing repo, now in terminal running docker run again with “device”

export DISPLAY=:1
xhost +
docker run --device=/dev/video0:/dev/video0 -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix backup/trtpose-l4t-pytorch:backup_version

again as advised following “Logitech C930e on Jetson TX1 very slow and choppy video” in container running :

apt-get install v4l-utils
v4l2-ctl -d /dev/video0 --list-formats-ext

does have the necessary links in repo able to install “install v4l-utils” after which does now recognise the device/fomats and lists (I believe) the formats available to the webcam:

Index       : 0
Type        : Video Capture
Pixel Format: 'YUYV'
Name        : YUYV 4:2:2
	Size: Discrete 640x480
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 160x120
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 176x144
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x176
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 352x288
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 432x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 544x288
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 752x416
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x448
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 864x480
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 960x544
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 960x720
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1024x576
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1184x656
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x960
		Interval: Discrete 0.133s (7.500 fps)
		Interval: Discrete 0.200s (5.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'MJPG' (compressed)
Name        : Motion-JPEG
	Size: Discrete 640x480
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 160x120
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 176x144
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x176
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 320x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 352x288
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 432x240
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 544x288
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 640x360
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 752x416
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x448
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 800x600
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 864x480
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 960x544
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 960x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1024x576
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1184x656
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x720
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)
	Size: Discrete 1280x960
		Interval: Discrete 0.033s (30.000 fps)
		Interval: Discrete 0.040s (25.000 fps)
		Interval: Discrete 0.050s (20.000 fps)
		Interval: Discrete 0.067s (15.000 fps)
		Interval: Discrete 0.100s (10.000 fps)
		Interval: Discrete 0.200s (5.000 fps)

now I am trying to modify the following command to pass one of these available formats, using the first block (index0) from formats, passing the following comand:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUYV,width=640,height=480,framerate=30/1! nvvidconv ! ‘video/x-raw(memory:NVMM),format=NV12’ ! nvoverlaysink

gives the following warning:

WARNING: erroneous pipeline: could not link v4l2src0 to nvvconv0, neither element can handle caps video/x-raw, format=(string)YUYV, width=(int)640, height=(int)480, framerate=(fraction)30/1

now taking the second block (index1) from formats I modified to the following

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=MJPG,width=640,height=480,framerate=30/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=NV12’ ! nvoverlaysink

gave a similar warning:

WARNING: erroneous pipeline: could not link v4l2src0 to nvvconv0, neither element can handle caps video/x-raw, format=(string)MJPG, width=(int)640, height=(int)480, framerate=(fraction)30/1

As as side note the original gstreamer command

gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! xvimagesink;

also returns erroneous pipeline warning

WARNING: erroneous pipeline: no element “xvimagesink”

not sure what I’m doing wrong here…help appreciated in advance

thanks

Hi,

I’m still unable to resolve this issue and would be grateful if someone could help me resolve

many thanks