Nvv4l2decoder + RTSP can not display

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

This command no display:
gst-launch-1.0 rtspsrc location=rtsp://10.86.62.111:8554/ ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! videoconvert ! nveglglessink

But no problem with commands:
gst-launch-1.0 rtspsrc location=rtsp://10.86.62.111:8554/ ! rtph264depay ! h264parse ! avdec_h264 ! nveglglessink
gst-launch-1.0 filesrc location=/home/tu/106.mkv ! matroskademux ! h264parse ! nvv4l2decoder ! nvvidconv ! nveglglessink

logs:

Setting pipeline to PAUSED …
Opening in BLOCKING MODE
0:00:00.102491611 22532 0x55850f9240 WARN v4l2 gstv4l2object.c:4410:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.102554875 22532 0x55850f9240 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x55850ceaf0 Failed to determine interlace mode
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://10.86.62.111:8554/
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
0:00:00.303410957 22532 0x7f68059230 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:fakesrc0:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.338775833 22532 0x7f50004b70 FIXME basesink gstbasesink.c:3145:gst_base_sink_default_event: stream-start event without group-id. Consider implementing group-id handling in the upstream elements
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
0:00:03.190380251 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.004789541 too big, reset skew
0:00:05.003170423 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.311291202 too big, reset skew
0:00:18.635903299 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.057937285 too big, reset skew
0:00:18.648481805 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:746:rtp_jitter_buffer_calculate_pts: backward timestamps at server, schedule resync
0:00:23.194267095 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.009495420 too big, reset skew
0:00:26.187187221 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.013801594 too big, reset skew
0:00:28.187596107 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.179902597 too big, reset skew
0:00:31.287480891 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.776832162 too big, reset skew
0:00:31.301572859 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:746:rtp_jitter_buffer_calculate_pts: backward timestamps at server, schedule resync
0:00:31.329553500 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:746:rtp_jitter_buffer_calculate_pts: backward timestamps at server, schedule resync
0:00:44.752878508 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.008483562 too big, reset skew
0:00:44.770066103 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:746:rtp_jitter_buffer_calculate_pts: backward timestamps at server, schedule resync
0:00:57.853467704 22532 0x7f68059000 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.188387872 too big, reset skew

I use jetpack 4.4 on TX2

3

Hi,
Please try the pipeline with uridecodebin:

gst-launch-1.0 uridecodebin uri=rtsp://10.86.62.111:8554/ ! nvoverlaysink
4

I want find pipeline for appsink use “hardware-based decoder” (like NVDEC, it not use cpu and cuda core)

5

Hi,
Please set

$ export GST_DEBUG=*FACTORY*:4

You should see hardware decoder nvv4l2decoder is picked by uridecodebin

1 Like
7

@DaneLLL

I tried this pipeline (gst-launch-1.0 uridecodebin uri=rtsp://10.86.62.111:8554/ ! nvoverlaysink) and it works, but how do I play the audio at the same time?

The pipeline below did not work?

gst-launch-1.0 uridecodebin uri=$RTSP_PATH ! qtdemux name=demux
demux.video_0 ! queue ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 overlay=2
demux.audio_0 ! queue ! avdec_aac ! audioconvert ! alsasink device= “hw:0,3”

8

Hi,
Please try

gst-launch-1.0 rtspsrc location=rtsp://10.86.62.111:8554/ name=src src. ! 'application/x-rtp, media=(string)audio' ! decodebin ! audioconvert ! fakesink silent=false src. ! 'application/x-rtp, media=(string)video' ! decodebin ! videoconvert ! fakesink silent=false

Reference link:

9

tried, did not work.
no error, no audio and video

$ gst-launch-1.0 rtspsrc location=rtsp://192.168.1.168:8554/0 name=src src. ! ‘application/x-rtp, media=(string)audio’ ! decodebin ! audioconvert ! fakesink silent=false src. ! ‘application/x-rtp, media=(string)video’ ! decodebin ! videoconvert ! fakesink silent=false

Setting pipeline to PAUSED …

Pipeline is live and does not need PREROLL …

Progress: (open) Opening Stream

Progress: (connect) Connecting to rtsp://192.168.1.168:8554/0

Progress: (open) Retrieving server options

Progress: (open) Retrieving media info

Progress: (request) SETUP stream 0

Progress: (request) SETUP stream 1

Progress: (open) Opened Stream

Setting pipeline to PLAYING …

New clock: GstSystemClock

Progress: (request) Sending PLAY request

Progress: (request) Sending PLAY request

Progress: (request) Sent PLAY request

(gst-launch-1.0:17113): GLib-GObject- WARNING **: 14:22:05.716: …/…/…/…/gobject/gsignal.c:2641: instance ‘0x55984bfe40’ has no handler with id ‘4’

(gst-launch-1.0:17113): GLib-GObject- WARNING **: 14:22:05.717: …/…/…/…/gobject/gsignal.c:2641: instance ‘0x55984bfe40’ has no handler with id ‘3’

10

[quote=“james.ee.developer, post:9, topic:121901”]
I tried the following video pipeline, and it is not even working… no video output

$ gst-launch-1.0 rtspsrc location=$RTSP_PATH ! ‘application/x-rtp, media=(string)video’ ! decodebin ! videoconvert ! fakesink silent=false

Setting pipeline to PAUSED …

Pipeline is live and does not need PREROLL …

Progress: (open) Opening Stream

Progress: (connect) Connecting to rtsp://192.168.1.168:8554/0

Progress: (open) Retrieving server options

Progress: (open) Retrieving media info

Progress: (request) SETUP stream 0

Progress: (request) SETUP stream 1

Progress: (open) Opened Stream

Setting pipeline to PLAYING …

New clock: GstSystemClock

Progress: (request) Sending PLAY request

Progress: (request) Sending PLAY request

Progress: (request) Sent PLAY request

Opening in BLOCKING MODE

NvMMLiteOpen : Block : BlockType = 261

NVMEDIA: Reading vendor.tegra.display-size : status: 6

NvMMLiteBlockCreate : Block : BlockType = 261

reference in DPB was never decoded

11

Similarly, the audio pipeline is also not working… no audio

$ gst-launch-1.0 rtspsrc location=$RTSP_PATH name=src src. ! ‘application/x-rtp, media=(string)audio’ ! decodebin ! audioconvert ! fakesink silent=false

Setting pipeline to PAUSED …

Pipeline is live and does not need PREROLL …

Progress: (open) Opening Stream

Progress: (connect) Connecting to rtsp://192.168.1.168:8554/0

Progress: (open) Retrieving server options

Progress: (open) Retrieving media info

Progress: (request) SETUP stream 0

Progress: (request) SETUP stream 1

Progress: (open) Opened Stream

Setting pipeline to PLAYING …

New clock: GstSystemClock

Progress: (request) Sending PLAY request

Progress: (request) Sending PLAY request

Progress: (request) Sent PLAY request

12

Hi,
For video, please replace fakesink with nvoverlaysink. For audio, please replace with alsasink. Yo can run aplay -l to get information about audio output, and set device to alsasink.

13

@DaneLLL

Thanks. It works.

Just to share, this works too …

gst-launch-1.0 rtspsrc location= $RTSP_PATH name=src src. ! 'application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96’ ! rtph264depay ! nvv4l2decoder ! nvvidconv flip-method=0 ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 src. ! ‘application/x-rtp, media=(string)audio’ ! decodebin ! audioconvert ! alsasink device= “hw:0,3”

14

@DaneLLL

I have a multicast group stream at udp://@238.0.0.1:1234. I read Gstreamer udp decoding pipeline in Jetson TX2 - #11 by Kaladin forum, and tried the following client pipeline but it did not work. Appreciate it if you can give some guidance…

gst-launch-1.0 udpsrc multicast-group=238.0.0.1 auto-multicast=true port=1234 name=src
src. ! ‘application/x-rtp,encoding-name=H264,payload=96’ ! rtph264depay ! nvv4l2decoder ! nvvidconv flip-method=0 ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 overlay=2
src. ! ‘application/x-rtp, media=(string)audio’ ! decodebin ! audioconvert ! alsasink device=“hw:0,3”

Output messages :
Setting pipeline to PAUSED …

Opening in BLOCKING MODE

Pipeline is live and does not need PREROLL …

Setting pipeline to PLAYING …

New clock: GstSystemClock

NvMMLiteOpen : Block : BlockType = 261

NVMEDIA: Reading vendor.tegra.display-size : status: 6

NvMMLiteBlockCreate : Block : BlockType = 261

WARNING: from element /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0: Could not decode stream.

Additional debug info:

gstrtpbasedepayload.c(466): gst_rtp_base_depayload_handle_buffer (): /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0:

Received invalid RTP payload, dropping

WARNING: from element /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0: Could not decode stream.

Additional debug info:

gstrtpbasedepayload.c(466): gst_rtp_base_depayload_handle_buffer (): /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0:

Received invalid RTP payload, dropping