I'm trying to capture image in ultra-low latency mode

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi,

I am trying to decode via UDP in H265 with ultra-low-latency mode, for encoding I am using a VITEC encoder.
I launch the following code:

gst-launch-1.0 udpsrc address=225.1.1.1 multicast-iface=eth0 port=30120 ! typefind ! tsdemux ! multiqueue ! h265parse ! capsfilter caps='video/x-h265, stream-format=byte-stream' ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),format=(string)NV12' ! videoconvert ! autovideosink sync=false

I have the following output

0:00:09.074792416 11134   0x7f90056d40 WARN               h265parse gsth265parse.c:1063:gst_h265_parse_handle_frame:<h265parse0> broken/invalid nal Type: 38 FD, Size: 4 will be dropped
NvMMLiteOpen : Block : BlockType = 279 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 279 
0:00:09.184232512 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:3300:gst_v4l2_object_save_format:<nvv4l2decoder0:src> Driver bug detected, stride (640) is too small for the width (1280)
0:00:09.184340128 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:09.184389824 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:3038:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.184425152 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:2924:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat PM10
0:00:09.184452512 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:3038:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.184557792 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:2930:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat PM10
0:00:09.184873824 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x55a1874cc0 Failed to determine interlace mode
0:00:09.184965472 11134   0x7f90056d40 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x55a1874cc0 Failed to determine interlace mode
0:00:09.199535456 11134   0x7f90056d40 WARN            v4l2videodec gstv4l2videodec.c:1673:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:09.200177312 11134   0x7f90056d40 WARN          v4l2bufferpool gstv4l2bufferpool.c:1066:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:09.381094464 11134   0x55a1879140 WARN                 tsdemux tsdemux.c:2359:gst_ts_demux_queue_data: CONTINUITY: Mismatch packet 7, stream 15

I cannot see the video, I only receive the data packet of the last line.

This happens when I try to capture the video with the encoder broadcasting. If I start the gstreamer process before the encoder starts broadcasting, once I start receiving, I play the video without problem.

The same goes for the DeepStream, if I launch it with the encoder emitting, I get a black screen.

This problem only happens to me with the ultra-low-latency mode activated. With ultra low latency turned off I can watch the video anyway.

Any solution?

Thanks a lot

3

Hi,
It looks like the stream cannot be decoded correctly. Please help check if you can dump the stream as a file source and see if it can be decoded:

gst-launch-1.0 udpsrc address=225.1.1.1 multicast-iface=eth0 port=30120 ! typefind ! tsdemux ! multiqueue ! h265parse ! capsfilter caps='video/x-h265, stream-format=byte-stream' ! filesink location=a.h265
gst-launch-1.0 filesrc location=a.h265 ! h265parse ! nvv4l2decoder ! nvoverlaysink
4

Hi, thanks for your response.
I can dump the image into a file and then decode, but with the same problem as before, that is:
If the dump process is running before the encoder casts, I can decode and display the image correctly.
But if once the decoder is broadcasting, I lift the dump process, then in decoding I have no image. It only happens to me with ultra-low-latency UDP.

5

Hi,
Please check if it helps by setting the property:
Deepstream Decode Acceleration - #3 by DaneLLL