GStreamer pipeline occasionally fails with "Stream format not found, dropping the frame"

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Xavier NX
• DeepStream Version 6.3.0
• JetPack Version (valid for Jetson only) 5.1.3
• TensorRT Version 8.5.2
• NVIDIA GPU Driver Version (valid for GPU only) N/A
• Issue Type( questions, new requirements, bugs) Bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

The bug occurs when I run a simple GStreamer pipeline which reads data from a USB web camera and writes the stream to disk.

gst-launch-1.0 v4l2src device=/dev/video0 !  nvv4l2decoder mjpeg=1 ! nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=out.mp4

Around half the time I run this pipeline, I see the message Stream format not found, dropping the frame. When this happens, the pipeline stops and no data is written to disk. Here is a sample output where the pipeline fails:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
0:00:00.551220603 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d9dc0 Failed to determine interlace mode
0:00:00.551507841 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d9dc0 Failed to determine interlace mode
0:00:00.551614211 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d9dc0 Failed to determine interlace mode
0:00:00.551708229 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d9dc0 Failed to determine interlace mode
0:00:00.551940426 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
0:00:00.615291650 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.615430949 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:00.615532455 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:00.615625641 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:00.615720427 10325 0xaaaabf4fc690 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277
0:00:00.663234189 10325 0xaaaabf3d2cc0 WARN          v4l2bufferpool gstv4l2bufferpool.c:809:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
Stream format not found, dropping the frame
Stream format not found, dropping the frame
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:09.779189593
Setting pipeline to NULL ...
0:00:10.405151010 10325 0xaaaabf3d2cc0 WARN                    v4l2 gstv4l2object.c:4561:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:10.405318534 10325 0xaaaabf3d2cc0 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:10.405527082 10325 0xaaaabf3d2cc0 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:10.405629324 10325 0xaaaabf3d2cc0 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
0:00:10.405751439 10325 0xaaaabf3d2cc0 WARN                    v4l2 gstv4l2object.c:2420:gst_v4l2_object_add_interlace_mode:0xaaaabf4d5020 Failed to determine interlace mode
Freeing pipeline ...

I have tried two different USB cameras and both produce the same error. This includes an ELP Synchronized Stereo Camera and an ELP USB 3.0 Low Light Camera.

It seems like the issue is caused by the DeepStream plugins because when I replace nvv4l2decoder with jpegdec and nvv4l2h264enc with x264enc, the pipeline always works correctly. The problem is that the CPU is not fast enough to transcode the stream in realtime, so I need the hardware accelerated versions to work correctly 100% of the time.

Any ideas as to what is causing this issue? Thank you.

2

I have done further investigation, and it seems the problem is caused by the nvv4l2decoder plugin. When I replace this plugin with jpegdec followed by nvvideoconvert, the pipeline always works as expected.

I would still like to use the hardware accelerated JPEG decoding for my MJPEG stream. I tried using nvjpegdec instead of jpegdec, but the output video is static (i.e. it only shows the first frame for the full duration) even when I set mjpegdecode=1. Is there a way I can get either nvv4l2decoder or nvjpegdec to work?

4

Could you refer to this FAQ and check if that helps?

5

Hi yuweiw,

Thank you for sharing the FAQ. Unfortunately, adding an explicit caps filter does not resolve the issue. I still get the same error about half the time.

Here is a minimal pipeline that reproduces the problem:

gst-launch-1.0 v4l2src device=/dev/video0 ! nvv4l2decoder mjpeg=true ! fakesink

Here is the same pipeline with a capsfilter, which still produces the problem:

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,height=720,width=1280,format=MJPG,framerate=60/1 ! nvv4l2decoder mjpeg=true ! fakesink

Here is standard output:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277
Stream format not found, dropping the frame
Stream format not found, dropping the frame

It only occurs around half the time, but when I see the message Stream format not found, dropping the frame, the pipeline hangs. When this occurs, jtop also shows that the hardware accelerated JPEG decoder is off. Are you able to reproduce this, or is this issue isolated to the ELP cameras I am using?

6

How long does it take?

No. I use ST200-ARV01 model for about 3 hours, that issue doesn’t occur.

This is possible.

You can also try running that with uridecoderbin source plugin.
When you run that, please use jtop to observe whether the loading and memory use are abnormal.

7

It happens as the pipeline is starting, so within a couple seconds of running the gst-launch-1.0 command. If the pipeline starts successfully, then I experience no issues for any duration of recording. The problem is that it only starts correctly around half the time.

Does the pipeline above always launch correctly with your camera? If so, the issue must be originating in the ELP cameras.

The plugin uridecodebin does work, but it isn’t using the JPEG hardware decoder so it seems equivalent to the pipeline using jpegdec.

8

Could you refer to our FAQ to dump the whole pipeline when you use the uridecodebin?

9

Sure, here is the pipeline with uridecodebin:

playing
playing4961×537 202 KB

10

As you can see from the pipeline, there is a nvvidconv between the v4l2src and nvv4l2decoder. Could you try to add that in your pipeline and try?

11

Adding nvvidconv betwen v4l2src and nvv4l2decoder does not work since the v4l2src is a source for image/jpeg which is not included in the nvvidconv sink caps:

  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw(memory:NVMM)
                 format: { (string)I420, (string)I420_10LE, (string)P010_10LE, (string)I420_12LE, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV
12, (string)NV16, (string)NV24, (string)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { (string)I420, (string)UYVY, (string)YUY2, (string)YVYU, (string)NV12, (string)NV16, (string)NV24, (string)P010_10LE, (strin
g)GRAY8, (string)BGRx, (string)RGBA, (string)Y42B, (string)Y444 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Specifically, I get this warning:

WARNING: erroneous pipeline: could not link v4l2src0 to nvvconv0, nvvconv0 can't handle caps image/jpeg, width=(int)1280, height=(int)720
12

There is a GstDecoderBin plugin between the v4l2src and nvvidconv plugin too.
Also the format of the source stream is YUY2 1920x1080 60fps from your pipeline graph, not jpeg. So it may be that the jpeg streams from your camera cannot be supported by our hardware decoder.

13

The issue seems isolated to my cameras, so I’ll just stick with the software decoder.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.