Access to Raspberry Cam ( nvargus-daemon ) from docker container

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello,

I’m working on running YOLO darknet with raspberry cam module on the jetson nano and it works great. (with OpenCV with gstreamer support)

I’m successfully feeding the cam input into darknet with this gstreamer pipeline:

nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12 ! nvvidconv ! video/x-raw, format=BGRx, width=640, height=360 ! videoconvert ! video/x-raw, format=BGR ! appsink

I’d like this same app to be able to run from a docker container for easy install, I’m very aware that Jetson platform isn’t supported yet by nvidia-docker, but I’ve successfully managed to give the Docker container access to the GPU stuff by mounting severals volumes from the jetson host system into the docker container, here is the script I use: https://github.com/moovel/lab-opendatacam/blob/v2/docker/run-jetson/run-docker.sh

In this script I run the docker container as privileged, which gives me access to an USB Webcam without problem, for example from the docker container I can run this gstreamer pipeline successfully:

v4l2src ! video/x-raw, framerate=30/1, width=640, height=360 ! videoconvert ! appsink

But I’m unable to have access to the rasbberry cam module, when I run this pipeline:

nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1280, height=720, framerate=30/1, format=NV12 ! nvvidconv ! video/x-raw, format=BGRx, width=640, height=360 ! videoconvert ! video/x-raw, format=BGR ! appsink

I get this error:

(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 201)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:515 Failed to create CameraProvider

I think I need to give the docker container some other volumes so it has access to the nvarguscamerasrc process, for now I’ve tried:

-v /usr/sbin:/usr/sbin:ro 
-v /usr/sbin/nvargus-daemon:/usr/sbin/nvargus-daemon:ro

But without success.

Could you give me any pointers of which volumes are needed in order to use nvarguscamerasrc ?

Thanks,

Thibault

2

It looks like libargus depends on a daemon running. That can’t happen in a container so it looks like what you are trying to do is not possible (easily) if at all.

You can, however, see up a design where some process outside the container is streaming data from the camera into the container (no privileged!). The problem then is: CUDA cannot currently run in Docker on Nano (but support is planned). You could process video, but only with the CPU, and the CPU on the nano is not very strong.

Tl;Dr? Containers are not very useful right now on Nano. If you want to plan for future Nano, maybe sending streams into an Nvidia-docker container on x86.

3

@mdegans Thanks a lot for your answer, so no technical solution to give the process inside the docker container access to this daemon.

Yes the solution would be to stream the video into the container. BTW in my post I explain I succeeded to use CUDA inside the container so this is not a pb for me. (will be better when docker-nvidia will support it)

Container are really usefull already for me on Jetson, I can offer a one line setup to install all the dependencies of my app… Just a shame I can’t get easy access to the raspberry cam module, I will only support usb camera with docker for now.

4

You may be able to get CUDA to work without privileged by passing necessary devices into the container with --device. I have never tested this, however. Please see this:

You will still have to get the CUDA libraries and such into the container, however. Your linked script seems to do that fine.

5

Thanks for the pointer, but running privileged isn’t an issue for me… The people running this app can give full access to the host machine…

6

Just is you are aware of the security implications. Docker Hub is kind of a dumpster fire.

7

what you may do is to mount the camera via v4l2 and mount is as if it is usb camera

sudo su
cd /usr/src/linux-headers-4.9.140-tegra-ubuntu18.04_aarch64/kernel-4.9
make modules_prepare
mkdir v4l2loopback
git clone https://github.com/umlaeute/v4l2loopback.git v4l2loopback
cd v4l2loopback
make
make install
apt-get install -y v4l2loopback-dkms v4l2loopback-utils
modprobe v4l2loopback devices=1 video_nr=2 exclusive_caps=1
echo options v4l2loopback devices=1 video_nr=2 exclusive_caps=1 > /etc/modprobe.d/v4l2loopback.conf
echo v4l2loopback > /etc/modules
update-initramfs -u
TERMINAL 1:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1' ! nvvidconv ! 'video/x-raw, width=640, height=480, format=I420, framerate=30/1' ! videoconvert ! identity drop-allocation=1 ! 'video/x-raw, width=640, height=480, format=RGB, framerate=30/1' ! v4l2sink device=/dev/video2

it wont help you accessing the CSI directly, highly likely, but it will work as /dev/video2 camera that way

8

Hello @Andrey1984 ,

Many thanks for this. I’ll try this week.

9

Not at all. Moreover, I can see that the sensor should be represented as per the output below:

jetbot@jetbot:~$ ls /dev/video*
/dev/video0

Probably you could map it as well somehow as /dev/video0. But I would try doing it as /dev/video2 as per the approach above.

10

Yes, map it to /dev/video2 is OK for me . I’ll get my hands back on this project only later this week, I’ll keep you posted on how it goes.

11

Hi @Andrey1984

I just tried (sorry for delay I didn’t have access to the nano before)

After running your installations steps and launching you gstreamer pipeline I get this:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3, format=(string)RGB, colorimetry=(string)sRGB
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)4/3
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21,000000 fps Duration = 47619048 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28,000001 fps Duration = 35714284 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29,999999 
GST_ARGUS: PowerService: requested_clock_Hz=13608000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

But I don’t see any video frame, it seems to be stuck

When I run:

ls /dev/video*

I see that /dev/video2 is mapped

/dev/video0  /dev/video2

I tried to open cheese to see if I can see the output of video2 but it doesn’t detect it

Here is the full output

nano@nano-desktop:~$ v4l2-ctl --list-devices
vi-output, imx219 6-0010 (platform:54080000.vi:0):
	/dev/video0

Dummy video device (0x0000) (platform:v4l2loopback-000):
	/dev/video2

Any idea what I might be doing wrong ?

12 
gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=1920, height=1080, framerate=30/1' ! nvvidconv ! 'video/x-raw, width=640, height=480, format=I420, framerate=30/1' ! videoconvert ! identity drop-allocation=1 ! 'video/x-raw, width=640, height=480, format=RGB, framerate=30/1' ! v4l2sink device=/dev/video2

run that code in a separate terminal and try again
that should generate the stream that you could read e.g with:

gst-launch-1.0 v4l2src device=/dev/video2 ! 'video/x-raw, format=RGB, width=640, height=480, framerate=30/1' ! queue ! videoconvert ! xvimagesink
13

Great ! This works, sorry I didn’t understand that one was proxying the cam and needed to run while the other was the consumer…

I can access from the docker container this way !

Last question, do you have a way to clean up the gst-launch “server” , because after a few times stopping and restarting the command I get:

Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0: Device '/dev/video2' is not a output device.
Additional debug info:
v4l2_calls.c(646): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0:
Capabilities: 0x85208001
Setting pipeline to NULL ...
Freeing pipeline ...

If I restart the nano it goes away, but would be nice to have a better way.

Thanks you again !

14

it was said that “may be adding -e in producer would help for cleaner closing.”

15

@tibo.durand I somehow made it working in the Docker container. Check these links:

DISCLAIMER - not sure if all nvargus-daemon features do work, but at least my JetBot is properly wandering & avoiding collisions. It’s still WIP, but looks promising.

Output from the Docker container:

root@19c80a8:/usr/src/app# python3 jetbot/jetbot/apps/wander.py ./best_model.pth 
Loading model...
Initializing camera...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 0 
   Output Stream W = 3280 H = 2464 
   seconds to Run    = 0 
   Frame Rate = 21.000000 
GST_ARGUS: PowerService: requested_clock_Hz=37126320
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
Running...

nvargus-daemon is filling console with the following errors …

SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)
SCF: Error BadParameter: Vector index out of bounds (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/Vector.h, function get(), line 337)

… but they seem harmless, at least for my use case. (I maybe wrong with the harmless statement)

16

Here’s the short video of wandering JetBot - [url]https://twitter.com/robertvojta/status/1143081208294850560[/url]. Everything’s running in a container.

17

@zrzka Many thanks for sharing this ! well done ! For our use case right now I do not have time to try this (we are releasing our project soon ( GitHub - opendatacam/opendatacam: An open source tool to quantify the world ) but definitely ping us if you make any improvement on this.

18

I met the same problem as you. Could you tell me how you solved it？

Thanks a lot!

19

will the nvargus daemon run without privileged mode?
I can run it from container with

docker run -it --rm --net=host --runtime nvidia -v /usr/sbin/nvargus-daemon:/usr/sbin/nvargus-daemon --device /dev/video0:/dev/video0 -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-base:r32.3.1

20

I solved this here: Access to Raspberry Cam v2 fails from docker container using L4T 32.4.2 (using nvarguscamerasrc) , thanks !

1 Like