Agx Dynamic object video 60fps rtp send and recv some question?

happy886 · January 16, 2023, 8:44am

agx version
nvidia@localhost:~$ head -n 1 /etc/nv_tegra_release
# R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020
60 fps fpdlink camera

send agx and recv agx is connect directly ，60fps camera video dynamic object.

send pipeline :
gst-launch-1.0 nvv4l2camerasrc device=/dev/video1 num-buffers=600 ! ‘video/x-raw(memory:NVMM), width=(int)1920,height=(int)1080,format=(string)UYVY, framerate=(fraction)60/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=(int)1920,height=(int)1080,format=(string)I420, framerate=(fraction)60/1’ ! omxh264enc bitrate=10000000 control-rate=2 iframeinterval=30 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! tee name=streams streams. ! filesink location=88.264 streams. ! rtph264pay config-interval=1 ! udpsink host=192.168.0.112 port=30006 sync=false async=false

recv pipeline 1:
gst-launch-1.0 -ev udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! ‘video/x-h264,stream-format=(string)byte-stream’ ! filesink location=112.264

recv pipeline 2:
gst-launch-1.0 -ev udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! tee name=streams streams. ! ‘video/x-h264,stream-format=(string)byte-stream’ ! filesink location=112-tee.264 streams. ! omxh264dec disable-dpb=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080’ ! nvdrmvideosink offset-x=0 offset-y=0 plane-id=1 sync=false async=false

send pipeline and recv pipeline1 ,the save file is 88.264 and 112.264 ,no frame lost!
send pipeline and recv pipelien 2,the save file is 88-1.264 and 112-tee.264 ,there is frame lost! I don’t know what caused the frame loss, could you please help check the cause？

i test nvoverlaysink,nvdrmvideosink,nveglglessink and nv3dsink,and also nvv4l2h264enc,nvv4l2h265enc,omxh264enc and omxh265enc ，Especially when the recv pipeline was starting up.
88.264 (11.9 MB)
112.264 (11.3 MB)
88-1.264 (11.9 MB)
112-tee.264 (11.1 MB)

happy886 · January 16, 2023, 9:44am

Especially in the case of running both recv pipeline and jtop command, the image dragging situation is serious. Close the jtop command and the image dragging disappears after a while

happy886 · January 17, 2023, 1:19am

If you add queue or rtpjitterbuffer, there will be no drag caused by frame drops at recv pipeline startup
We need low latency for sending and receiving, but the queue and rtpjitterbuffer add latency，so we don’t want to use it
queue pipeline:
gst-launch-1.0 -ev udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! queue ! omxh264dec disable-dpb=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080’ ! nvdrmvideosink offset-x=0 offset-y=0 plane-id=1 sync=false async=false

rtpjitterbuffer pipeline:
gst-launch-1.0 -ev udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtpjitterbuffer latency=0 ! rtph264depay ! h264parse ! omxh264dec disable-dpb=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080’ ! nvdrmvideosink offset-x=0 offset-y=0 plane-id=1 sync=false async=false

happy886 · January 17, 2023, 5:56am

I did another test with fpsdisplaysink, testing nvv4l2decoder and omxh264dec
test pipeline1,average: 22,23,Why can’t we get to 60 ？
gst-launch-1.0 udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! omxh264dec ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
test pipeline2,average: 60
gst-launch-1.0 udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

nvidia@localhost:~$ gst-launch-1.0 udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! omxh264dec ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, encoding-name=(string)H264, payload=(int)96, media=(string)video, clock-rate=(int)90000
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, encoding-name=(string)H264, payload=(int)96, media=(string)video, clock-rate=(int)90000
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)4.2

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.858: gst_caps_is_empty: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.859: gst_caps_truncate: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.859: gst_caps_fixate: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.859: gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.859: gst_structure_get_string: assertion ‘structure != NULL’ failed

(gst-launch-1.0:16111): GStreamer-CRITICAL **: 21:57:43.859: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)4.2
Allocating new output: 1920x1088 (x 11), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstOMXH264Dec-omxh264dec:omxh264dec-omxh264dec0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 12, dropped: 0, current: 22.06, average: 22.06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 24, dropped: 0, current: 22.48, average: 22.27
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 36, dropped: 0, current: 23.25, average: 22.58
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 48, dropped: 0, current: 23.28, average: 22.75
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 60, dropped: 0, current: 23.24, average: 22.85
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 72, dropped: 0, current: 23.06, average: 22.88
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 84, dropped: 0, current: 23.46, average: 22.96
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 96, dropped: 0, current: 23.35, average: 23.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 108, dropped: 0, current: 23.40, average: 23.05
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 120, dropped: 0, current: 23.30, average: 23.08
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 132, dropped: 0, current: 23.37, average: 23.11
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 144, dropped: 0, current: 23.06, average: 23.10
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 156, dropped: 0, current: 23.35, average: 23.12
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 168, dropped: 0, current: 23.37, average: 23.14
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 180, dropped: 0, current: 23.48, average: 23.16
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 192, dropped: 0, current: 23.44, average: 23.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 204, dropped: 0, current: 23.45, average: 23.19
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 216, dropped: 0, current: 23.04, average: 23.19
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 228, dropped: 0, current: 23.46, average: 23.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 240, dropped: 0, current: 23.27, average: 23.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 252, dropped: 0, current: 23.24, average: 23.20
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 264, dropped: 0, current: 23.35, average: 23.21
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 276, dropped: 0, current: 23.21, average: 23.21
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 288, dropped: 0, current: 23.40, average: 23.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 300, dropped: 0, current: 23.19, average: 23.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 312, dropped: 0, current: 23.33, average: 23.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 324, dropped: 0, current: 23.23, average: 23.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 336, dropped: 0, current: 23.42, average: 23.23
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:14.746134752
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
nvidia@localhost:~$ gst-launch-1.0 udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! nvv4l2decoder ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, encoding-name=(string)H264, payload=(int)96, media=(string)video, clock-rate=(int)90000
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, parsed=(boolean)true
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, encoding-name=(string)H264, payload=(int)96, media=(string)video, clock-rate=(int)90000
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)4.2

(gst-launch-1.0:16451): GStreamer-CRITICAL **: 21:58:02.857: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, width=(int)1920, height=(int)1080, framerate=(fraction)0/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, profile=(string)constrained-baseline, level=(string)4.2
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 37, dropped: 0, current: 72.61, average: 72.61
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 67, dropped: 0, current: 60.00, average: 66.37
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 98, dropped: 0, current: 60.02, average: 64.22
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 128, dropped: 0, current: 60.00, average: 63.18
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 158, dropped: 0, current: 59.98, average: 62.54
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 188, dropped: 0, current: 59.98, average: 62.12
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 219, dropped: 0, current: 60.06, average: 61.82
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 249, dropped: 0, current: 59.95, average: 61.59
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 280, dropped: 0, current: 60.02, average: 61.41
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 311, dropped: 0, current: 59.97, average: 61.26
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 341, dropped: 0, current: 60.00, average: 61.15
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 372, dropped: 0, current: 60.04, average: 61.06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 402, dropped: 0, current: 59.95, average: 60.97
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 432, dropped: 0, current: 59.99, average: 60.90
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 462, dropped: 0, current: 59.99, average: 60.84
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 493, dropped: 0, current: 60.04, average: 60.79
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 523, dropped: 0, current: 59.97, average: 60.74
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 554, dropped: 0, current: 59.98, average: 60.70
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 585, dropped: 0, current: 60.01, average: 60.66
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 616, dropped: 0, current: 60.12, average: 60.64
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 647, dropped: 0, current: 59.96, average: 60.60
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 677, dropped: 0, current: 59.96, average: 60.57
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 708, dropped: 0, current: 60.08, average: 60.55
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 738, dropped: 0, current: 59.99, average: 60.53
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 768, dropped: 0, current: 59.95, average: 60.51
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 799, dropped: 0, current: 59.99, average: 60.49
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 829, dropped: 0, current: 59.93, average: 60.47
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 860, dropped: 0, current: 60.11, average: 60.45
^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:14.748805280
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

DaneLLL · January 17, 2023, 6:49am

Hi,
We have deprecated omx plugins, so please use v4l2 plugins.
Please execute the steps on all Jetson platforms and check again:

Execute $sudo nvpmodel -m 0 and sudo jetson_clocks
Enable the property to nvv4l2decoder:

  enable-max-performance: Set to enable max performance
                        flags: readable, writable
                        Boolean. Default: false

Enable the property to encoder:

  maxperf-enable      : Enable or Disable Max Performance mode
                        flags: readable, writable, changeable only in NULL or READY state
                        Boolean. Default: false

4 Enable VIC at maximum clock:
Nvvideoconvert issue, nvvideoconvert in DS4 is better than Ds5? - #3 by DaneLLL

After the steps, Jetson platforms are in maximum performance mode. Please give it a try and check if there is improvement.

happy886 · January 17, 2023, 7:05am

Yes,it is maxn mode and jetson_clocks and enabled VIC at maximum clock
And They have also been tested, and they are all the same. When recv pipeline starts up, the moving object will have folded shadow trailing, and when recv pipeline runs jtop at the same time, the moving object will have folded shadow trailing immediately.

Moving object, 60fps, 1920x1080 camera, h264 codec shouldn’t be a problem? Normal mode and Best Performance mode should be fine, right?

test send pipeline
gst-launch-1.0 nvv4l2camerasrc device=/dev/video1 ! ‘video/x-raw(memory:NVMM), width=(int)1920,height=(int)1080,format=(string)UYVY, framerate=(fraction)60/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM),width=(int)1920,height=(int)1080,format=(string)I420, framerate=(fraction)60/1’ ! nvv4l2h264enc bitrate=10000000 control-rate=1 iframeinterval=30 preset-level=1 maxperf-enable=1 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! rtph264pay config-interval=1 ! udpsink host=192.168.0.112 port=30006 sync=false async=false

test recv pipeline
gst-launch-1.0 -ev udpsrc port=30006 ! ‘application/x-rtp,encoding-name=(string)H264,payload=(int)96’ ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 disable-dpb=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080’ ! nvoverlaysink overlay-x=0 overlay-y=0 overlay-w=1920 overlay-h=1080 sync=false async=false

happy886 · January 17, 2023, 7:32am

filesrc pipeline
gst-launch-1.0 -e filesrc location=88.264 ! h264parse ! nvv4l2decoder enable-max-performance=1 disable-dpb=1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080’ ! nv3dsink window-x=0 window-y=0 window-width=1920 window-height=1080 sync=false async=false

Using udpsrc video stream decoding[ test recv pipeline], running jtop at the same time, the overlap and trailing situation is obvious;

Using filesrc to decode files directly[ filesrc pipeline], while running jtop, the video is obviously stuck;

happy886 · January 29, 2023, 9:29am

@DaneLLL Is there any reply?

DaneLLL · January 30, 2023, 2:07am

Hi,
Please set bitrate tp 2Mbps or 4Mbps for a try. Probably the bottleneck is in network bandwidth.

And please try to configure CBR + virtual buffer size:
Random blockiness in the picture RTSP server-client -Jetson TX2 - #5 by DaneLLL
This can avoid burst bitrate in fast scene change.

happy886 · January 30, 2023, 5:43am

Modifying different bit rates has been tested, it has nothing to do with bit rates;
It looks like a 60fps live motion video decoding problem. It’s the receiving side, but the sending side should be fine
omxh264enc is already set cbr mode！
control-rate : Bitrate control method
flags: readable, writable, changeable only in NULL or READY state
Enum “GstOMXVideoEncControlRate” Default: 1, “variable”
(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): variable-skip-frames - Variable Skip Frames
(4): constant-skip-frames - Constant Skip Frames
The two agx devices are directly connected without a router, so there should be no network bottleneck

happy886 · January 30, 2023, 5:51am

“jetson_multimedia_api\samples\00_video_decode” ,I also tested on this demo, if it is 30fps camera data, the decoder can process the data in time, rtp receiving thread queue without data accumulation; If it is 60fps camera data, there will be data accumulation in the rtp receiving thread queue, and it feels that the decoder cannot process the data in time.

The NvVideoDecoder also didn’t find the place to set fps;

happy886 · January 30, 2023, 6:09am

This post,This local file test, excluding network and sender issues, can see the decoder issues, why is this?
The file is in the first post, you can try it.@DaneLLL
Is this a 1920x1080 60fps decoding performance problem?

DaneLLL · January 30, 2023, 1:42pm

Hi,
Please remove disable-dpb=1 and try. The property works for specific stream. Certain h264 stream cannot be well decoded when setting the property.

happy886 · January 31, 2023, 5:29am

thank you! I tried to remove disable-dpb, but no effect！
omxh264enc plugin vbv-size default value is 10
nvv4l2h264enc plugin vbv-size default value is 4000000
i tried omxh264enc plugin vbv-size maximum value 30,but no effect!
I did another test！
Keep sender and receiver unchanged
send： 1920x1080 60fps gst pipeline
recv: jetson_multimedia_api/00_video_decode ，The demo made small changes
rtp thread：Receive rtp packets and parse h264
NvVideoDecoder output_plane thread：Decode input
NvVideoDecoder capture_plane thread： Decode output → g_que.push(fd) //std::queue g_que;
NvEglRenderer thread: Refresh fd and display thread. Clear the queue before display to ensure that the latest fd is displayed
NvEglRenderer .setFPS(60)

The only variable of the test process：
① xrandr --output HDMI-0 --mode 1920x1080 --rate 60.00
② xrandr --output HDMI-0 --mode 1920x1080 --rate 120.00
In the case of rate 60, the accumulation of g_que is artificially caused by jtop, and the image is stuck. After closing jtop, the fd accumulated by g_que that is not flushed cannot be eliminated in time，there are always old fd in the queue
In the case of rate 120, the accumulation of g_que is artificially caused by jtop, and the image is stuck. After closing jtop, the fd accumulated by g_que that is not flushed can be eliminated in time，only the latest fd is in the queue, and the old fd has been refreshed and displayed

happy886 · January 31, 2023, 5:39am

happy886 · January 31, 2023, 5:40am

DaneLLL · February 2, 2023, 1:53am

Hi,
We try the pipelines with videotestsrc plugin and it looks working fine. Please try the pipelines and check if you still hit the issue. We use one Jetson device as sender and the other as receiver:
[sender]

gst-launch-1.0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! tee name=t ! nvv4l2h264enc insert-sps-pps=1 idrinterval=15 maxperf-enable=1 ! h264parse ! rtph264pay ! udpsink host=10.19.115.105 port=5000 sync=0

[receiver without rendering]

gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,encoding-name=H264,payload=96' ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v

[receiver with rendering]

gst-launch-1.0 udpsrc port=5000 ! 'application/x-rtp,encoding-name=H264,payload=96' ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! fpsdisplaysink text-overlay=0 video-sink=nv3dsink sync=0 -v