Gstreamer stream on xavier stuck

gadi.didi85 · May 16, 2021, 8:35am

hi , i have Xavier release 32 and i have some c code with gstreamer lib

my pipline is :

> appsrc  name=mysrc  ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! video/x-raw(memory:NVMM), width=%d, height=%d, format=NV12 ! nvv4l2h264enc control-rate=1 bitrate=%d peak-bitrate=%d ! video/x-h264, profile=baseline  ! rtph264pay pt=96 name=pay0 "
>                     ")"

i connect my xavier to dji drone.
the drone has 2 camera. fpv ( built in camera )
and H20T camera.

the fpv stream works well by the pipline above…
But, the second camera doesnot work. i try to connect with this client:
gst-launch-1.0 -vvv rtspsrc location=rtsp://192.168.1.225:1234/drone ! rtph264depay ! decodebin ! videoconvert ! autovideosink sync=false

and client stuck on line :

...
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info

and the streamer (server) stuck on lines:

NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4

link for the details about the cameras :
https://developer.dji.com/onboard-sdk/documentation/tutorial/advanced-sensing.html

( by the way , i connected the H20T camera to my computer and it works well …)
UPDATE:
I CAN DO (WITH THE H20T CAMERA):
" appsrc name=mysrc ! h264parse ! rtph264pay pt=96 name=pay0 "

(i have to encode my stream)
So i guess that decoding does not work well on this camera ?
but i can do decode\encode in tx2 nvidia(release 28) and on my pc (amd64 architecture)
maybe those cameras have different format ?

tnkx

DaneLLL · May 17, 2021, 2:41am

Hi.
Please set

$ export GST_DEBUG=*FACTORY*:4

And share the prints. Would like to know what elements are picked by decodebin.

gadi.didi85 · May 18, 2021, 9:54am

i used GST_DEBUG and i get ( its still stuck on H20T CAMERA…but work on another drone camera):

> 0:00:03.342207104  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "appsrc"
> 0:00:03.352402592  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "h264parse"
> 0:00:03.435545120  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvv4l2decoder"
> 0:00:03.443327776  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvvidconv"
> 0:00:03.444307328  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvv4l2h264enc"
> 0:00:03.448281280  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "rtph264pay"
> 0:00:03.448917952  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "bin"
> 0:00:03.452373824  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "capsfilter"
> 0:00:03.453710304  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "capsfilter"
> 0:00:03.454850560  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "pipeline" named "media-pipeline"
> 0:00:03.456792160  9925   0x7f50200190 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "rtpbin"
> 0:00:03.458379264  9925   0x7edc009ad0 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "rtpsession"
> 0:00:03.460305152  9925   0x7edc009ad0 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "rtpssrcdemux"
> 0:00:03.460749856  9925   0x7edc009ad0 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "rtpstorage"
> 0:00:03.461367232  9925   0x7edc009ad0 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "funnel"
> 0:00:03.461708832  9925   0x7edc009ad0 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "funnel"
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4

and stuck on this line …

i compare with the camera that works.
the camera that works has more lines:

===== NVMEDIA: NVENC =====

NvMMLiteBlockCreate : Block : BlockType = 4 
H264: Profile = 66, Level = 0 
0:00:02.741530336 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "funnel"
0:00:02.743477824 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "udpsrc"
0:00:02.745410656 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "tee"
0:00:02.745824960 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "multiudpsink"
0:00:02.747016928 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "tee"
0:00:02.747182752 11281   0x7f541ffd90 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "multiudpsink"

Honey_Patouceul · May 18, 2021, 10:46am

Does it improve with the following ?

appsrc name=mysrc ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! video/x-raw(memory:NVMM), width=%d, height=%d, format=NV12 ! nvv4l2h264enc control-rate=1 bitrate=%d peak-bitrate=%d profile=0 insert-vui=true insert-sps-pps=1 insert-aud=1 ! h264parse ! video/x-h264, stream-format=byte-stream ! rtph264pay pt=96 name=pay0

gadi.didi85 · May 18, 2021, 10:58am

Hi ,
still doesnt works…

its stuck on:

NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4

maybe is it depend on camera format?
it not works on jetson tx2 and xavier both …
it work only on my PC ( without nvidia gpu…)

in jetson:

===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
TVMR: FrameRate = 31
TVMR: NVDEC LowCorner Freq = (186000 * 1024)

and stuck…

DaneLLL · May 19, 2021, 1:46am

Hi,
It looks like it decode h264 stream and re-encode into another h264 stream. Maybe you can try to send out the h264 stream without re-encoding. Please try

appsrc name=mysrc ! h264parse config-interval=1 ! video/x-h264,stream-format=byte-stream ! rtph264pay pt=96 name=pay0

gadi.didi85 · May 19, 2021, 4:33am

Hi,
It works , but without encode the bandwidh is ~ 12 Mb per seci have to encode…
Is there reason why some camera’s stream could not be encoded ?

gadi.didi85 · May 19, 2021, 7:48am

i try this with -vvv:

gst-launch-1.0 -vvv rtspsrc location=rtsp://192.168.1.225:1234/drone ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! 'video/x-raw(memory:NVMM),width=480, height=360, format=(string)NV12' ! nvv4l2h264enc ! h264parse ! rtspclientsink latency=0 location=rtsp://localhost:$RTSP_PORT/$RTSP_PATH

and i get:

/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)30/1
0:00:03.031857341 17116   0x558eaf3680 WARN            v4l2videodec gstv4l2videodec.c:1673:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:03.032502975 17116   0x558eaf3680 WARN          v4l2bufferpool gstv4l2bufferpool.c:1065:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:03.034338467 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:848:gst_h264_parser_parse_buffering_period: failed to read UE
0:00:03.034410723 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:892:gst_h264_parser_parse_buffering_period: error parsing "Buffering period"
0:00:03.034481187 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:1188:gst_h264_parser_parse_sei_message: Bit non equal to one.
0:00:03.034540451 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:1194:gst_h264_parser_parse_sei_message: Bit non equal to zero.
0:00:03.034642436 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:1194:gst_h264_parser_parse_sei_message: Bit non equal to zero.
0:00:03.034775236 17116   0x558eaf3680 WARN               h264parse gsth264parse.c:531:gst_h264_parse_process_sei:<h264parse0> failed to parse one or more SEI message
0:00:03.036555752 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:848:gst_h264_parser_parse_buffering_period: failed to read UE
0:00:03.036595784 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:892:gst_h264_parser_parse_buffering_period: error parsing "Buffering period"
0:00:03.036646472 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:1188:gst_h264_parser_parse_sei_message: Bit non equal to one.
0:00:03.036675272 17116   0x558eaf3680 WARN       codecparsers_h264 gsth264parser.c:1194:gst_h264_parser_parse_sei_message: Bit non equal to zero.
...

i seems like the NVIDIA xavier cannot deal with decode this camera ?
because with another drone camera it do it well

Honey_Patouceul · May 19, 2021, 6:56pm

I cannot tell much more, but the error seems rather related to your camera stream format and how gstreamer h264 parser handles it than related to nvidia. I would expect any other arm64 platform with same gstreamer version and olugins to do the same.
Also be sure you don’t have custom plugin in gst plugins path. I’d suggest first trying with test-launch for RTSP stream testing.

A few things that you might try just decoding and displaying to monitor:

# Be sure to get only H264 video 
gst-launch-1.0 -v rtspsrc location=rtsp://192.168.1.225:1234/drone ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM), width=640,height=480,pixel-aspect-ratio=1/1' ! nvoverlaysink

# Try decodebin metaplugin (should use nvv4l2decoder on Jetson)
gst-launch-1.0 -v rtspsrc location=rtsp://192.168.1.225:1234/drone ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! decodebin ! autovideosink

# Try cpu decoder (may be slow) and displaying to X:
gst-launch-1.0 -v rtspsrc location=rtsp://192.168.1.225:1234/drone ! application/x-rtp,media=video,encoding-name=H264 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! xvimagesink

If none of these work, you may analyze your h264 stream (there are some free softwares for that) and check if there is something wrong or special within this h264 stream.

If you see one these working, then it is ok for decoding and next step would be encoding and streaming.

Topic		Replies	Views
installing missing plugin ffmpeg for gstreamer DeepStream SDK	8	5621	October 12, 2021
Gstreamer pipeline(nvv4l2decoder) for .264 file Jetson Xavier NX gstreamer	9	2102	November 1, 2021
JX2, ZedMINI GStreamer, RTSP server using Android phone Jetson TX2	8	2090	August 7, 2018
gStreamer - high delay, low fps Jetson AGX Orin gstreamer	12	949	March 26, 2024
Deepstream 3.0 C API Jetson AGX Xavier	10	2491	May 30, 2019
GStreamer - stream video to IP Jetson Nano gstreamer	12	5554	October 15, 2021
RTSP stream not work "Stream format not found, dropping the frame" DeepStream SDK rtsp , gstreamer , nvbugs	24	1609	January 3, 2024
Streaming desktop with RTSP Gstreamer server Jetson AGX Xavier rtsp	22	8389	October 18, 2021
TX2 H264 RTSP Stream decoding issues Jetson TX2	27	11566	October 18, 2021
revising streaming with Xavier Jetson AGX Xavier	12	1252	October 18, 2021

Gstreamer stream on xavier stuck

Related topics