How to access camera with Jetson Nano

I’m trying to receive picture from camera on Jetson Nano. To do this I’m using code like this:

import cv2
device = cv2.VideoCapture()
pipeline = """ \
                nvarguscamerasrc sensor-id=0\
                sensor-mode=3 exposuretimerange="100000 80000000" ! \
                video/x-raw(memory:NVMM), width=640, height=480, format=NV12, framerate=30/1 ! \
                nvvidconv ! 
                video/x-raw, format=BGRx ! 
                videoconvert ! \
                appsink \
            """
device.open(pipeline, cv2.CAP_GSTREAMER)

But last line doesn’t execute. It just freeze, after pressing ctrl+c I receive this message:

(python3:45): GStreamer-CRITICAL **: 14:40:05.666: 
Trying to dispose element pipeline0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline

(python3:45): GStreamer-CRITICAL **: 14:40:05.666: 
Trying to dispose element videoconvert0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

(python3:45): GStreamer-CRITICAL **: 14:40:05.666: 
Trying to dispose element appsink0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

All of this was executed on Duckiebot DB21M in docker container. Before starting the container I’ve checked, that no process was using /dev/video0 by executing lsof /dev/video0
Docker container was started with flags --privileged -v /data:/data -v /dev:/dev so it should have access to all devices including camera

Hello @lelikk2002,

What happens if you run the pipeline outside python, using gst-launch-1.0?

Do you get the same result ?

regards,
Andrew
support@proventusnova.com

I’m not quite familiar with gstreamer, so I’m not sure how to execute it properly, so I tried two variants:
gst-launch-1.0 ! nvarguscamerasrc sensor-id=0 sensor-mode=3 exposuretimerange="100000 80000000" ! "video/x-raw(memmory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1" ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! appsink
This provide me with this result:
WARNING: erroneous pipeline: syntax error
And this try:
gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=3 exposuretimerange="100000 80000000" ! "video/x-raw(memmory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1" ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! appsink
Provided me with this result:
WARNING: erroneous pipeline: no element "nvarguscamerasrc"

Ok, got it. No worries, we can try to fix this together.

Its weird that it is saying that you don’t have the nvarguscamerasrc element, since it should already be installed in your Jetson board.

Also, just for you know. The appsink element will only work to get data from GStreamer into an application, so if we are going to run a pipeline outside an app, we should use a different sink, such as a videosink to watch the camera stream.

Lets try simplifying the pipeline to see what do we get.

Could you please run:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! ximagesink

I’ve run the pipeline you provided, but still receive error “no element”:

WARNING: erroneous pipeline: no element "nvarguscamerasrc"

Important to mention, that all of this happens in docker container, however in my docker file I install gstreamer as it was written in this guide. So I have line in my docker file:
RUN apt-get install -y gstreamer1.0-alsa gstreamer1.0-libav gstreamer1.0-plugins-bad gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-tools libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev
And in Docker container I have Gstreamer version 1.20.3, but without nvarguscamerasrc

Oh ok got it.

Could you please share the output of the following command ?

gst-inspect-1.0

regards,
Andrew
support@proventusnova.com

out.txt (73.8 KB)
Output has about 1,5K lines, so here’s file with output of line you’ve provided.
There is no anything that contains nvargus

Yes, you are right.

You are not only missing nvargus, but all NVIDIA proprietary elements.

I will do some research.

regards,
Andrew
support@proventusnova.com

@lelikk2002

What happens if you run this inside and outside the host container:

gst-inspect-1.0 | grep nv

I was reading this other forum and it seems like the NVIDIA accelerated plugins should be installed on the host device and the Docker container should have access to them:

regards,
Andrew
support@proventusnova.com

audioconvert:  audioconvert: Audio converter
audiofx:  audioinvert: Audio inversion
autoconvert:  autoconvert: Select converter based on caps
autoconvert:  autovideoconvert: Select color space converter based on caps
bayer:  rgb2bayer: RGB to Bayer converter
closedcaption:  ccconverter: Closed Caption Converter
debugutilsbad:  errorignore: Convert some GstFlowReturn types into others
ivtc:  ivtc: Inverse Telecine
libav:  avdec_dsicinvideo: libav Delphine Software International CIN video decoder
libav:  avdec_idcinvideo: libav id Quake II CIN video decoder
libav:  avdec_twinvq: libav VQF TwinVQ decoder
libav:  avdec_wnv1: libav Winnov WNV1 decoder
opengl:  glcolorconvert: OpenGL color converter
opengl:  gleffects_blur: Blur with 9x9 separable convolution Effect
opengl:  gleffects_laplacian: Laplacian Convolution Demo Effect
opengl:  glviewconvert: OpenGL Multiview/3D conversion filter
rtponvif:  rtponvifparse: ONVIF NTP timestamps RTP extension
rtponvif:  rtponviftimestamp: ONVIF NTP timestamps RTP extension
videoconvert:  videoconvert: Colorspace converter

It doesn’t look like there is any NVidia soft inside the container

Outside I have NVidia soft:

omx:  nvoverlaysink: OpenMax Video Sink
nvvideocuda:  videocuda: CUDA Post processor
debugutilsbad:  errorignore: Convert some GstFlowReturn types into others
nvdrmvideosink:  nvdrmvideosink: Nvidia Drm Video Sink
nvvideosink:  nvvideosink: nVidia Video Sink
nveglstreamsrc:  nveglstreamsrc: nVidia EGL Stream
nvarguscamerasrc:  nvarguscamerasrc: NvArgusCameraSrc
nvvidconv:  nvvidconv: NvVidConv Plugin
nvvideosinks:  nv3dsink: Nvidia 3D sink
nvegltransform:  nvegltransform: NvEGLTransform
nvjpeg:  nvjpegdec: JPEG image decoder
nvjpeg:  nvjpegenc: JPEG image encoder
libav:  avdec_wnv1: libav Winnov WNV1 decoder
libav:  avdec_idcinvideo: libav id Quake II CIN video decoder
libav:  avdec_dsicinvideo: libav Delphine Software International CIN video decoder
libav:  avdec_twinvq: libav VQF TwinVQ decoder
audiofx:  audioinvert: Audio inversion
ivtc:  ivtc: Inverse Telecine
nvvideo4linux2:  nvv4l2vp9enc: V4L2 VP9 Encoder
nvvideo4linux2:  nvv4l2vp8enc: V4L2 VP8 Encoder
nvvideo4linux2:  nvv4l2h265enc: V4L2 H.265 Encoder
nvvideo4linux2:  nvv4l2h264enc: V4L2 H.264 Encoder
nvvideo4linux2:  nvv4l2decoder: NVIDIA v4l2 video decoder
nvtee:  nvtee: NvTee
bayer:  rgb2bayer: RGB to Bayer converter
nvv4l2camerasrc:  nvv4l2camerasrc: NvV4l2CameraSrc
autoconvert:  autovideoconvert: Select color space convertor based on caps
autoconvert:  autoconvert: Select convertor based on caps
audioconvert:  audioconvert: Audio converter
nveglglessink:  nveglglessink: EGL/GLES vout Sink
videoconvert:  videoconvert: Colorspace converter
rtponvif:  rtponvifparse: ONVIF NTP timestamps RTP extension
rtponvif:  rtponviftimestamp: ONVIF NTP timestamps RTP extension
opengl:  glviewconvert: OpenGL Multiview/3D conversion filter
opengl:  gleffects_laplacian: Laplacian Convolution Demo Effect
opengl:  gleffects_blur: Blur with 9x9 separable convolution Effect
opengl:  glcolorconvert: OpenGL color converter
nvivafilter:  nvivafilter: NvIVAFilter Plugin
nvcompositor:  nvcompositor: NvCompositor

Unfortunately solution from topic you’ve mentioned doesn’t help me.
If I run export LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0:/lib/aarch64-linux-gnu/libgomp.so.1 in container and after that run gst-inspect-1.0 | grep nv I’ll receive the same output as I sent before

Thank you for your message.
I am using agx orin 64gb jetpack 6 L4T 36.3.0
running the following command

jetson-containers run --env HUGGINGFACE_TOKEN=xxxxxxxxxxx
$(autotag nano_llm)
using my HF TOKEN

INSIDE THE CONTAINER

gst-inspect-1.0 | grep nv

(gst-plugin-scanner:23): GStreamer-WARNING **: 11:19:46.756: adding type GstEvent multiple times

(gst-plugin-scanner:23): GLib-GObject-WARNING **: 11:19:47.422: cannot register existing type ‘GstRtpSrc’

(gst-plugin-scanner:23): GLib-GObject-CRITICAL **: 11:19:47.422: g_type_add_interface_static: assertion ‘G_TYPE_IS_INSTANTIATABLE (instance_type)’ failed

(gst-plugin-scanner:23): GLib-CRITICAL **: 11:19:47.422: g_once_init_leave: assertion ‘result != 0’ failed

(gst-plugin-scanner:23): GStreamer-CRITICAL **: 11:19:47.422: gst_element_register: assertion ‘g_type_is_a (type, GST_TYPE_ELEMENT)’ failed

(gst-plugin-scanner:23): GLib-GObject-WARNING **: 11:19:47.423: cannot register existing type ‘GstRtpSink’

(gst-plugin-scanner:23): GLib-GObject-CRITICAL **: 11:19:47.423: g_type_add_interface_static: assertion ‘G_TYPE_IS_INSTANTIATABLE (instance_type)’ failed

(gst-plugin-scanner:23): GLib-CRITICAL **: 11:19:47.423: g_once_init_leave: assertion ‘result != 0’ failed

(gst-plugin-scanner:23): GStreamer-CRITICAL **: 11:19:47.423: gst_element_register: assertion ‘g_type_is_a (type, GST_TYPE_ELEMENT)’ failed
sh: 1: lsmod: not found
sh: 1: modprobe: not found
audioconvert: audioconvert: Audio converter
audiofx: audioinvert: Audio inversion
autoconvert: autoconvert: Select converter based on caps
autoconvert: autovideoconvert: Select color space converter based on caps
bayer: rgb2bayer: RGB to Bayer converter
closedcaption: ccconverter: Closed Caption Converter
debugutilsbad: errorignore: Convert some GstFlowReturn types into others
ivtc: ivtc: Inverse Telecine
libav: avdec_dsicinvideo: libav Delphine Software International CIN video decoder
libav: avdec_idcinvideo: libav id Quake II CIN video decoder
libav: avdec_twinvq: libav VQF TwinVQ decoder
libav: avdec_wnv1: libav Winnov WNV1 decoder
nvarguscamerasrc: nvarguscamerasrc: NvArgusCameraSrc
nvcompositor: nvcompositor: NvCompositor
nvdrmvideosink: nvdrmvideosink: Nvidia Drm Video Sink
nveglglessink: nveglglessink: EGL/GLES vout Sink
nveglstreamsrc: nveglstreamsrc: nVidia EGL Stream
nvegltransform: nvegltransform: NvEGLTransform
nvipcpipeline: nvipcpipelinesink: Inter-process Pipeline Sink
nvipcpipeline: nvipcpipelinesrc: Inter-process Pipeline Source
nvipcpipeline: nvipcslavepipeline: Inter-process slave pipeline
nvivafilter: nvivafilter: NvIVAFilter Plugin
nvjpeg: nvjpegdec: JPEG image decoder
nvjpeg: nvjpegenc: JPEG image encoder
nvtee: nvtee: NvTee
nvv4l2camerasrc: nvv4l2camerasrc: NvV4l2CameraSrc
nvvidconv: nvvidconv: NvVidConv Plugin
nvvideo4linux2: nvv4l2av1enc: V4L2 AV1 Encoder
nvvideo4linux2: nvv4l2decoder: NVIDIA v4l2 video decoder
nvvideo4linux2: nvv4l2h264enc: V4L2 H.264 Encoder
nvvideo4linux2: nvv4l2h265enc: V4L2 H.265 Encoder
nvvideo4linux2: nvv4l2vp9enc: V4L2 VP9 Encoder
nvvideosink: nvvideosink: nVidia Video Sink
nvvideosinks: nv3dsink: Nvidia 3D sink
rtponvif: rtponvifparse: ONVIF NTP timestamps RTP extension
rtponvif: rtponviftimestamp: ONVIF NTP timestamps RTP extension
videoconvert: videoconvert: Colorspace converter
root@ubuntu:/#

OUTSIDE THE CONTAINER
st-inspect-1.0 | grep nv
audioconvert: audioconvert: Audio converter
audiofx: audioinvert: Audio inversion
autoconvert: autoconvert: Select converter based on caps
autoconvert: autovideoconvert: Select color space converter based on caps
bayer: rgb2bayer: RGB to Bayer converter
closedcaption: ccconverter: Closed Caption Converter
debugutilsbad: errorignore: Convert some GstFlowReturn types into others
ivtc: ivtc: Inverse Telecine
libav: avdec_dsicinvideo: libav Delphine Software International CIN video decoder
libav: avdec_idcinvideo: libav id Quake II CIN video decoder
libav: avdec_twinvq: libav VQF TwinVQ decoder
libav: avdec_wnv1: libav Winnov WNV1 decoder
nvarguscamerasrc: nvarguscamerasrc: NvArgusCameraSrc
nvcompositor: nvcompositor: NvCompositor
nvdrmvideosink: nvdrmvideosink: Nvidia Drm Video Sink
nveglglessink: nveglglessink: EGL/GLES vout Sink
nveglstreamsrc: nveglstreamsrc: nVidia EGL Stream
nvegltransform: nvegltransform: NvEGLTransform
nvipcpipeline: nvipcpipelinesink: Inter-process Pipeline Sink
nvipcpipeline: nvipcpipelinesrc: Inter-process Pipeline Source
nvipcpipeline: nvipcslavepipeline: Inter-process slave pipeline
nvivafilter: nvivafilter: NvIVAFilter Plugin
nvjpeg: nvjpegdec: JPEG image decoder
nvjpeg: nvjpegenc: JPEG image encoder
nvtee: nvtee: NvTee
nvv4l2camerasrc: nvv4l2camerasrc: NvV4l2CameraSrc
nvvidconv: nvvidconv: NvVidConv Plugin
nvvideo4linux2: nvv4l2av1enc: V4L2 AV1 Encoder
nvvideo4linux2: nvv4l2decoder: NVIDIA v4l2 video decoder
nvvideo4linux2: nvv4l2h264enc: V4L2 H.264 Encoder
nvvideo4linux2: nvv4l2h265enc: V4L2 H.265 Encoder
nvvideo4linux2: nvv4l2vp9enc: V4L2 VP9 Encoder
nvvideosink: nvvideosink: nVidia Video Sink
nvvideosinks: nv3dsink: Nvidia 3D sink
opengl: glcolorconvert: OpenGL color converter
opengl: gleffects_blur: Blur with 9x9 separable convolution Effect
opengl: gleffects_laplacian: Laplacian Convolution Demo Effect
opengl: glviewconvert: OpenGL Multiview/3D conversion filter
rtponvif: rtponvifparse: ONVIF NTP timestamps RTP extension
rtponvif: rtponviftimestamp: ONVIF NTP timestamps RTP extension
videoconvert: videoconvert: Colorspace converter

I’ve just figured it out. Can it be a problem if I’m using Ubuntu 22.04 as base image in my docker container?

Hello @lelikk2002,

That might be an issue if the Ubuntu version on the host device is different, however, I am not sure.

Have you tried using a different ubuntu version?

Here are the NVIDIA official Docker instructions: Your First Jetson Container | NVIDIA Developer

regards,
Andrew
support@proventusnova.com

I tried to use Ubuntu 22, 20 and 18. In all variants I didn’t have any NVidia plugins for gstreamer

I’ve tried to pull image from your link:
docker run -it --rm --net=host --runtime nvidia -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-base:r35.1.0 (I’ve removed flag for display, because I don’t have one)
In container I’ve installed opencv and executed the same lines of code, as I wrote in the start of this topic.
Right now execution isn’t blocked, but I can’t connect to camera.
I’ve also executed export LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0:/lib/aarch64-linux-gnu/libgomp.so.1 as it was mentioned in other topic you’ve sent here.
Also I tried to execute these two lines:
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! ximagesink I’ve received warning:
WARNING: erroneous pipeline: no element "ximagesink"
And for this line:
gst-launch-1.0 nvarguscamerasrc ! nvvidconvI have this output:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 201)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:551 Failed to create CameraProvider

Hello @lelikk2002,

Seems like the nvargus issue is no longer showing.

What happens if you run the following:

gst-launch-1.0 nvarguscamerasrc ! fakesink

regards,
Andrew
support@proventusnova.com

docker run -it --rm --net=host --runtime nvidia -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/l4t-base:r35.1.0
root@duckie06:/# export LD_PRELOAD=/lib/aarch64-linux-gnu/libGLdispatch.so.0:/lib/aarch64-linux-gnu/libgomp.so.1
root@duckie06:/# gst-launch-1.0 nvarguscamerasrc ! fakesink

(gst-plugin-scanner:9): GStreamer-WARNING **: 14:51:57.497: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so': libgstbadvideo-1.0.so.0: cannot open shared object file: No such file or directory
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 201)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:551 Failed to create CameraProvider
Got EOS from element "pipeline0".
Execution ended after 0:00:00.011416446
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...