Unable to stream imx334 with v4l2 driver on Nvidia Jetson Xavier NX where the same dts works for tegra driver(nvargs)

@proventusnova ,
I could only get rid of CHANSEL_NOMATCH error 00249 when I don’t use --stream-to = test.raw but still I could not see the streaming window
I am using gstlaunch command and I am facing issue.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! xvimagesink

The use case is I would like to stream the camera with imx334 sensor using v4l2 driver .

@spavan,

I don’t think that pipe is going to work, given that the camera provides bayer and if you use v4l2src it won’t go thought the ISP for delayering.

Could you please try the following pipe?

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! xvimagesink

Please let me know how it goes.

regards,
Andrew
Embedded Software Engineer at ProventusNova

Hi @proventusnova ,

Thank you ,now I could see the streaming window with the command , but I need to get the stream with the v4l2src pipeline which is failing as shown below.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000308676
Setting pipeline to NULL ...
Freeing pipeline ...

@ShaneCCC & @proventusnova
Few questions from my side

  1. Is this command working because my tegra driver is working well , is there any connection to it?
  2. Can I now confirm that the driver is working fine ?
  3. What changes should I do in order to stream through v4l2src?

Hello @spavan,

Its great to hear that you are now able to see the stream using GStreamer.

To answer your questions:

  1. From the GStreamer perspective, it seems like the driver is working as expected. We would suggest you run an endurance test just to make sure that the CHANCEL_NOMATCH error does not show.
  2. This confirms that the driver works fine for video capture with nvargus. However, it is probable that the error with v4l2-ctl still persists when storing video into a file.
  3. The problem with v4l2src is that you are capturing raw bayer data from the camera, and then pushing that into a videosink. That is not going to work for a couple of reasons:

a. When you capture raw bayer data (without doing debayer), the other elements in your pipe are not able to understand the data format given that most elements require a debayered image. Therefore, you must have a very specific reason for using v4l2src instead of nvarguscamerasrc, given that you will need to do a lot more processing to use the captured frames.

b. Even if you were ok with capturing bayer buffers, in order to use v4l2src with your driver you would need to patch the element, this is because v4l2src only supports raw8, and you are trying to capture raw12. But patching v4l2src is not a complex tasks, we could provide you with instructions.

All that being said. We would first like to understand why is it important for you to use v4l2src for streaming, there might be a better solution to your use case. Therefore, would you please answer the following 2 questions:

  1. Why is it that you need to capture with v4l2src specifically ?
  2. Could you please describe your use case ?

regards,
Andrew
Embedded Software Engineer at ProventusNova

Thank you @proventusnova
Can you explain more about the endurance test to get clarity on the error.
Also can you suggest the changes to be done to collect raw12 data?
Regarding the exact use case, I’ll let you know after a while, need some clarifications

@spavan,

What I mean by performing an endurance test, is to run video capture with GStreamer using nvarguscamerasrc for a long period of time, so that you can make sure that you wont get the CHANCEL_NOMATCH error.

Now, going back to finding a solution that works for you, could you please help me answer the following questions ?

  1. Why is it that you need to capture with v4l2src specifically ?
  2. Could you please describe your use case ?

regards,
Andrew
Embedded Software Engineer at ProventusNova

For the additional information which may help us,



I have tried streaming with qv4l2 (got to know about this through a forum) and below are the observations

qv4l2 -d /dev/video0(qv4l2_1.png)
qv4l2 -R -d /dev/video0qv4l2_2.png)

@spavan,

I am sorry, I still don’t understand what is your end goal.

Also, quick question, when you run this pipeline:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! xvimagesink

Were you seeing a green image or the actual camera stream ?

regards,
Andrew
Embedded Software Engineer at ProventusNova

My end goal is to stream the camera with the v4l2src pipeline instead of nvargus.

No I am not seeing the green screen with nvargus command, the images are the outputs of qv4l2 utility with the v4l2 driver

@spavan,

Sorry for the persistence, but I need to be in sync with you to provide the best support I can.

Why is it that your goal is using v4l2src instead of nvargus?

regards,
Andrew
Embedded Software Engineer at ProventusNova

Hello @proventusnova ,

  1. a) Just curious about the v4l2src in jetson xavier board,I want to explore more about the raw data captured by v4l2 since gst-launch is compressed.
    b) Just want to check bayer2rgb with that raw data irrespective of the clarity.
gst-launch-1.0 v4l2src -ev ! bayer2rgb ! videoconvert ! fakesink -ev

I need this command to work properly.

  1. From the imx334 datasheet I could not find any register settings with RAW8 , I have only RAW10 ,RAW12.

@spavan,

Thanks for clarifying.

If you need to capture raw bayer.
Then, we would advice you to patch the v4l2src element.

Here you can find instructions on how to patch the element for 10bit pixel depth.

Please give that a try, and if you still continue facing issues, just let us know.

regards,
Andrew
Embedded Software Engineer at ProventusNova

Hi @proventusnova ,
I have tried after applying this patch ,I could not find any difference and when I change the pixel format to RG10 I don’t see any data when I use below command

v4l2-ctl -d /dev/video0 --stream-mmap --stream-count=5 --stream-to=test.raw
gst-launch-1.0 -v v4l2src ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Hello @spavan,

What happens if you run the following command?

GST_DEBUG=2,GST_CAPS:5 gst-launch-1.0 -v v4l2src ! fakesink

regards,
Andrew
Embedded Software Engineer at ProventusNova

I will run and let you know the output.

I have changed the bit depth to 10 in dts file and changed data format to RG10 in the driver as well,why do you think the raw data is not being captured (my file size is zero) do I need to change anything else?

That might be the case.

You can try running the command that previously worked for you to verify your pixel depth changes:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! xvimagesink

You might need to change other registers.

regards,
Andrew
Embedded Software Engineer at ProventusNova

Yes, I have tried this command actually after going through the process mentioned in the link you have shared, this command says there is no element "nvargus… "
Without those changes(installing the plugins related to 1.16.3)I could not apply the patch.

after changing to RG10 I could not find nvargus element

So i could not test that command

  1. a)I have checked RG10 with nvargus it is working fine.
    b) When I run the below command
GST_DEBUG=2,GST_CAPS:5 gst-launch-1.0 -v v4l2src ! fakesink

The output I saw is below

GST_DEBUG=2,GST_CAPS:5 gst-launch-1.0 -v v4l2src ! fakesink
0:00:00.064943647  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3057:gst_pad_query_caps:<v4l2src0:src> get pad caps with filter (NULL)
0:00:00.065697282  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3064:gst_pad_query_caps:<v4l2src0:src> query returned image/jpeg; video/mpeg, mpegversion=(int)4, systemstream=(boolean)false; video/mpeg, mpegversion=(int){ 1, 2 }; video/mpegts, systemstream=(boolean)true; video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-dv, systemstream=(boolean)true; video/x-fwht; video/x-h263, variant=(string)itu; video/x-h264, stream-format=(string){ byte-stream, avc }, alignment=(string)au; video/x-h265, stream-format=(string)byte-stream, alignment=(string)au; video/x-pwc1, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-pwc2, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-sonix, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-vp8; video/x-vp9; video/x-wmv, wmvversion=(int)3, format=(string)WVC1
0:00:00.065839331  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3057:gst_pad_query_caps:<fakesink0:sink> get pad caps with filter (NULL)
0:00:00.065902947  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3064:gst_pad_query_caps:<fakesink0:sink> query returned ANY
0:00:00.065992803  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3057:gst_pad_query_caps:<v4l2src0:src> get pad caps with filter (NULL)
0:00:00.066107620  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3064:gst_pad_query_caps:<v4l2src0:src> query returned image/jpeg; video/mpeg, mpegversion=(int)4, systemstream=(boolean)false; video/mpeg, mpegversion=(int){ 1, 2 }; video/mpegts, systemstream=(boolean)true; video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-dv, systemstream=(boolean)true; video/x-fwht; video/x-h263, variant=(string)itu; video/x-h264, stream-format=(string){ byte-stream, avc }, alignment=(string)au; video/x-h265, stream-format=(string)byte-stream, alignment=(string)au; video/x-pwc1, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-pwc2, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-sonix, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-vp8; video/x-vp9; video/x-wmv, wmvversion=(int)3, format=(string)WVC1
0:00:00.066227524  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3057:gst_pad_query_caps:<fakesink0:sink> get pad caps with filter (NULL)
0:00:00.066277317  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstutils.c:3064:gst_pad_query_caps:<fakesink0:sink> query returned ANY
0:00:00.066412581  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstpad.c:2277:gst_pad_link_check_compatible_unlocked:<v4l2src0:src> src caps image/jpeg; video/mpeg, mpegversion=(int)4, systemstream=(boolean)false; video/mpeg, mpegversion=(int){ 1, 2 }; video/mpegts, systemstream=(boolean)true; video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-dv, systemstream=(boolean)true; video/x-fwht; video/x-h263, variant=(string)itu; video/x-h264, stream-format=(string){ byte-stream, avc }, alignment=(string)au; video/x-h265, stream-format=(string)byte-stream, alignment=(string)au; video/x-pwc1, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-pwc2, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-sonix, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-vp8; video/x-vp9; video/x-wmv, wmvversion=(int)3, format=(string)WVC1
0:00:00.066560998  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstpad.c:2279:gst_pad_link_check_compatible_unlocked:<fakesink0:sink> sink caps ANY
0:00:00.066604198  1874 0xaaaad5d61c40 DEBUG               GST_CAPS gstpad.c:2297:gst_pad_link_check_compatible_unlocked: caps are compatible
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.068163789  1874 0xaaaad5d54d20 DEBUG               GST_CAPS gstutils.c:3057:gst_pad_query_caps:<v4l2src0:src> get pad caps with filter (NULL)
0:00:00.068300014  1874 0xaaaad5d54d20 DEBUG               GST_CAPS gstutils.c:3064:gst_pad_query_caps:<v4l2src0:src> query returned EMPTY
0:00:00.068345070  1874 0xaaaad5d54d20 DEBUG               GST_CAPS gstutils.c:3103:gst_pad_peer_query_caps:<v4l2src0:src> get pad peer caps with filter (NULL)
0:00:00.068384974  1874 0xaaaad5d54d20 DEBUG               GST_CAPS gstutils.c:3110:gst_pad_peer_query_caps:<v4l2src0:src> peer query returned ANY
0:00:00.068459950  1874 0xaaaad5d54d20 WARN                 basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:00.068495631  1874 0xaaaad5d54d20 WARN                 basesrc gstbasesrc.c:3072:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

2)How to check whether the patch is effective or not ?
Currently I have made the changes in the source and replaced the Image in my target followed by reboot ,then gave the above command.

Hello @spavan,

You can use a filesink to store your buffers into a raw file and check integrity as we discussed before.

You can even share the raw file with me and I will give it a quick look for you.

regards,
Andrew
Embedded Software Engineer at ProventusNova

Yes i have looked at the size of the raw file, it seems fine.
I have not checked the data inside.
Give me sometime I will share the raw file.