Raw data video not stable by gstreamer rtpvrawpay

frank_cf_weng · August 30, 2023, 6:42am

Hi

I am trying to play a live video from the camera capture by gstreamer command with rtpvrawpay parameters.
The hardware architecture is such as below diagram.
Screenshot from 2023-08-30 13-59-31

I have been tried to change the ‘client’ to different platform, such like as Xavier, Orin and Intel i7.
The result is only i7 can display the normal video (1080P), but the video display from Xavier and Orin would be broken.

The gstreamer command on the video sender is :
gst-launch-1.0 -v v4l2src device=/dev/video0 ! “video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080” ! rtpvrawpay ! udpsink clients=192.168.55.100:5001 sync=false

The gstreamer command on the video receiver is :
gst-launch-1.0 -v udpsrc port=5001 caps=“application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)SMPTE240M, palyload=(int)96, a-framerate=(string)30” ! rtpvrawdepay ! videoconvert ! autovideosink sync=false

I google related information for rtpvrawpay and found someone met the similar issue on the other platform.
UDP-Raw Stream Issue On Nvidia Jetson Devices

Can you give me more details to explain what kind of power, features or functions impact a CPU to display a raw data video?

DaneLLL · August 30, 2023, 7:33am

Hi
Please run sudo nvpmodel -m 0 and sudo jetson_clocks to run system in maximum performance. And we would suggest encode frame date to h264/h265 stream to use less network bandwidth.

frank_cf_weng · August 30, 2023, 7:42am

Hi @DaneLLL

I have forced the system to maximum performance by nvpmodel and jetson_clocks commands already.
And also, I confirmed the network bandwidth does not limit the video quality on my case because I can see the perfect 1080P video on Intel i7 platform.

Just like I mentioned, Nvidia replied someone this situation may hit system limitation in other ticket, then I want to know what kind limitation of CPU impact this use case.

Can you share me more details?

DaneLLL · August 30, 2023, 8:13am

Hi,
The capability of CPU cores may dominate the performance. Please run sudo tegrastats to get system status.

frank_cf_weng · August 30, 2023, 8:47am

Hi @DaneLLL

Below attachment is the log, which client displays a raw data video.
dump_tegrastats_0830.log (4.6 KB)

DaneLLL · August 30, 2023, 11:07am

Hi,
It looks like the loading is on single CPU cores so capability of one core dominates the performance. It looks expected since i7 has better CPU capability. Xavier is an embedded system and we would suggest encode to h264/h265 stream.

Honey_Patouceul · August 30, 2023, 5:52pm

On receiver Orin, you may try:

# Increase max kernel socket buffer size to 1M
sudo sysctl -w net.core.rmem_max=1000000

# Receive with setting buffer-size property of udpsrc, and use nvvidconv into a NVMM memory displaysink, so that you avoid videoconvert:
gst-launch-1.0 -v udpsrc port=5001 buffer-size=1000000 ! 'application/x-rtp,media=video,clock-rate=90000,encoding-name=RAW,sampling=YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)SMPTE240M, palyload=(int)96, a-framerate=30' ! rtpvrawdepay ! queue ! nvvidconv ! nv3dsink

frank_cf_weng · August 31, 2023, 3:28am

Hi @Honey_Patouceul

Thanks for your suggestion, and it works well on Orin at my side.
Then, do you know how to fix this issue on Xavier?

Honey_Patouceul · August 31, 2023, 8:55pm

I’d expect the same to work on Xavier as well. Doesn’t it work ? If your Xavier is running JetPack4 and doesn’t have nv3dsink, you may try nvoverlaysink instead.

frank_cf_weng · September 1, 2023, 1:03am

Hi @Honey_Patouceul

No, Xavier still did not work, and I tried this case on JetPack5 with L4T 35.2.1.

Hi @DaneLLL

Do you any ideas?

DaneLLL · September 1, 2023, 1:17am

Hi,
As suggested before, we think it is better to encode to h264/h265 stream.

One more thought is you may try tcp. Honey Patouceul have the examples in:
gstreamer: gst-launch-1.0 to read from a tcp source - #2 by Honey_Patouceul

frank_cf_weng · September 1, 2023, 9:05am

Hi @DaneLLL

Can you suggest me how to create gstreamer command to play raw data video by a TCP server?

I tried some parameters but not work at all.
Here are my setup:

[sender]

gst-launch-1.0 -v v4l2src device=/dev/video0 ! “video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080” ! rtpvrawpay ! queue ! tcpserversink port=7001 host=0.0.0.0 recover-policy=keyframe sync-method=latest-keyframe

[receiver]

GST_DEBUG=3 gst-launch-1.0 tcpclientsrc port=7001 host=192.168.55.1 ! “application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)SMPTE240M, payload=(int)96, a-framerate=(string)30” ! rtpvrawdepay ! queue ! nvvidconv ! nv3dsink -v

After I run the command on receiver, system shows some error messages such like as below…

Setting pipeline to PAUSED …
Pipeline is PREROLLING …
0:00:00.145079136 7383 0xaaaae941fc00 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:tcpclientsrc0:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)SMPTE240M, payload=(int)96, ssrc=(uint)1632689860, timestamp-offset=(uint)831650250, seqnum-offset=(uint)12622, a-framerate=(string)30
/GstPipeline:pipeline0/GstRtpVRawDepay:rtpvrawdepay0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA
/GstPipeline:pipeline0/GstNv3dSink:nv3dsink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)RGBA
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
0:00:00.150519168 7383 0xaaaae941fc00 ERROR rtpbasedepayload gstrtpbasedepayload.c:623:gst_rtp_base_depayload_handle_event: Segment with non-TIME format not supported
0:00:00.150602240 7383 0xaaaae941fc00 ERROR rtpbasedepayload gstrtpbasedepayload.c:623:gst_rtp_base_depayload_handle_event: Segment with non-TIME format not supported
0:00:00.150669472 7383 0xaaaae941fc00 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop: error: Internal data stream error.
0:00:00.150701056 7383 0xaaaae941fc00 WARN basesrc gstbasesrc.c:3072:gst_base_src_loop: error: streaming stopped, reason error (-5)
/GstPipeline:pipeline0/GstRtpVRawDepay:rtpvrawdepay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)1920, height=(string)1080, colorimetry=(string)SMPTE240M, payload=(int)96, ssrc=(uint)1632689860, timestamp-offset=(uint)831650250, seqnum-offset=(uint)12622, a-framerate=(string)30
0:00:00.150818368 7383 0xaaaae941fc00 ERROR rtpbasedepayload gstrtpbasedepayload.c:623:gst_rtp_base_depayload_handle_event: Segment with non-TIME format not supported
0:00:00.150853376 7383 0xaaaae941fc00 ERROR rtpbasedepayload gstrtpbasedepayload.c:623:gst_rtp_base_depayload_handle_event: Segment with non-TIME format not supported
0:00:00.150883648 7383 0xaaaae941fc00 ERROR rtpbasedepayload gstrtpbasedepayload.c:623:gst_rtp_base_depayload_handle_event: Segment with non-TIME format not supported
ERROR: from element /GstPipeline:pipeline0/GstTCPClientSrc:tcpclientsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstTCPClientSrc:tcpclientsrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

DaneLLL · September 1, 2023, 9:19am

Hi,
We don’t have experience in the setup. Would see if other users can suggest next.

Honey_Patouceul · September 1, 2023, 11:19am

RTP is designed for UDP packetization.
For streaming raw video over TCP, you may try:

Sender:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! queue ! matroskamux streamable=1 ! tcpserversink port=7001 host=<Sender_IP>

Receiver:

gst-launch-1.0 tcpclientsrc host=<Sender_IP> port=7001 ! matroskademux ! xvimagesink

# Or (tested on AGX Xavier R32):
gst-launch-1.0 tcpclientsrc host=<Sender_IP> port=7001 ! queue ! matroskademux ! videoconvert ! xvimagesink sync=0

system · September 25, 2023, 6:26am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.