"Internal data stream error" : when decoding IP camera's rtsp stream with Jetson H264/HAL driver (omxh264dec)

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

According to the NVidia L4T Accelerated GStreamer User Guide, I have used the following three pipelines to decode My IP Camera stream but all failed at the same error message. Can someone shed some light here ?

IP Camera: Amcrest(IP8M-2493EW)/H264H

//Decodes stream from rtsp source
gst-launch-1.0 rtspsrc debug=1 location=rtsp://IP:554 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e

//Decode stream from IP camera “Save” video files
gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e

//Decode for jetson nano via low-memory filter, as recommended by User guide
gst-launch-1.0 filesrc location=<filename.mp4> ! qtdemux ! h264parse ! omxh264dec enable-low-outbuffer=1 ! ‘video/x-raw(memory:NVMM), format=(string)NV12’ ! fakesink sync=1 -e

**************** Paste Error message here: ***********************************
GST_DEBUG=3 gst-launch-1.0 rtspsrc debug=1 location=rtsp://192.168.1.93:554 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e
0:00:00.637570112 9595 0x55b9816470 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/user/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …

Using winsys: x11
Pipeline is live and does not need PREROLL …
0:00:00.682060317 9595 0x55b9816470 WARN structure gststructure.c:1832:priv_gst_structure_append_to_gstring: No value transform to serialize field ‘display’ of type ‘GstEGLDisplay’
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.1.93:554
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
0:00:00.929713096 9595 0x7f8403c770 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:fakesrc0:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.929891954 9595 0x7f8403c800 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:fakesrc1:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:01.030681115 9595 0x7f64003280 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error.
0:00:01.030772575 9595 0x7f64003280 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason not-linked (-1)
0:00:01.150813153 9595 0x7f580028a0 WARN default grammar.y:510:gst_parse_no_more_pads: warning: Delayed linking failed.
0:00:01.150969719 9595 0x7f580028a0 WARN default grammar.y:510:gst_parse_no_more_pads: warning: failed delayed linking some pad of GstRTSPSrc named rtspsrc0 to some pad of GstQTDemux named demux
WARNING: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
failed delayed linking some pad of GstRTSPSrc named rtspsrc0 to some pad of GstQTDemux named demux
0:00:01.152145942 9595 0x7f8403c8f0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: Internal data stream error.
0:00:01.152310946 9595 0x7f8403c8f0 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop: error: streaming stopped, reason not-linked (-1)
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1:
streaming stopped, reason not-linked (-1)
EOS on shutdown enabled – waiting for EOS after Error
Waiting for EOS…

2

Hi ssu,

I tested the following pipeline to decode and display a MP4 video file:

gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nvvidconv  ! nveglglessink -e

A quick question, are you running the pipelines through a SSH conection?

If yes, you need to define the display where to render the video, example:

DISPLAY=:0 gst-launch-1.0 -v filesrc location=test.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nvvidconv  ! nveglglessink -e

Regards,
Greivin

3

Hi,

Thx, not through SSH but HDMI console display. Is there any FTP site ? I could upload my 1-min clip video file (*.mp4) here. Then you can see what I saw here. (the video file was being captured by the following ffmpeg command from my IP camera:
ffmpeg -v verbose -i rtsp://:554 -vcodec h264 -acodec aac -f tee -map 0:v -map 0:a -timeout -1 -listen 1 “[f=mpegts]udp://:port|[f=segment:segment_time=60:segment_format=mp4:strftime=1]%Y-%m-%d_%H-%M-%S_saved.mp4”

Thanks,
ssu

4

Hi ssu,

You can share me a dropbox or drive link to download the file and test.

Regards,
Greivin F.

5

If I ran it without parameters

GST_DEBUG=3 gst-launch-1.0 rtspsrc debug=1 user-id= user-pw= location=rtsp://:554 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e
Invalid MIT-MAGIC-COOKIE-1 keynvbuf_utils: Could not get EGL display connection
0:00:00.110680623 10123 0x559fe4ce60 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home//.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …

Using winsys: x11
Invalid MIT-MAGIC-COOKIE-1 key0:00:00.114566703 10123 0x559fe4ce60 ERROR egladaption gstegladaptation_egl.c:155:gst_egl_adaptation_init_display: Could not get EGL display connection
0:00:00.114596651 10123 0x559fe4ce60 ERROR egladaption gstegladaptation_egl.c:196:gst_egl_adaptation_init_display: Couldn’t setup window/surface from handle
0:00:00.114619099 10123 0x559fe4ce60 ERROR nveglglessink gsteglglessink.c:555:egl_init: Couldn’t init EGL display
0:00:00.114641443 10123 0x559fe4ce60 ERROR nveglglessink gsteglglessink.c:581:egl_init: Failed to perform EGL init
ERROR: Pipeline doesn’t want to pause.

6

If I added Display , as you recommended.

DISPLAY=:0 GST_DEBUG=3 gst-launch-1.0 rtspsrc debug=1 user-id= user-pw= location=rtsp://:554 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e
Invalid MIT-MAGIC-COOKIE-1 keynvbuf_utils: Could not get EGL display connection
0:00:00.271867588 11190 0x55817d3e60 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home//.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …

Using winsys: x11
Invalid MIT-MAGIC-COOKIE-1 key0:00:00.281089215 11190 0x55817d3e60 ERROR egladaption gstegladaptation_egl.c:155:gst_egl_adaptation_init_display: Could not get EGL display connection
0:00:00.281341770 11190 0x55817d3e60 ERROR egladaption gstegladaptation_egl.c:196:gst_egl_adaptation_init_display: Couldn’t setup window/surface from handle
0:00:00.283125543 11190 0x55817d3e60 ERROR nveglglessink gsteglglessink.c:555:egl_init: Couldn’t init EGL display
0:00:00.283395286 11190 0x55817d3e60 ERROR nveglglessink gsteglglessink.c:581:egl_init: Failed to perform EGL init
ERROR: Pipeline doesn’t want to pause.

7

I have also attached a clip of my IP Camera Video file here. It can play with playbin ONLY, NOT GST-Launch-1.0

$ gst-launch-1.0 filesrc location=h264-2019-07-03_11-14-55_saved.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! nveglglessink -e
Setting pipeline to PAUSED …

Using winsys: x11
Pipeline is PREROLLING …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
ERROR: from element /GstPipeline:pipeline0/GstQTDemux:demux: Internal data stream error.
Additional debug info:
qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstQTDemux:demux:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …

8

https://gokeuslab-my.sharepoint.com/:v:/p/vincent_su/ERAte9_CffdMnnpvzWWlzkgBnWB6UY6CUXGnCejtZKXLHQ?e=QCcigH

You could download the file from here. Again can play with playbin But NOT “gst-launch-1.0 filesrc location…”

9

Hi ssu,

I am seeing similar issues while trying to play your file, i believe that the file is not properly saved.

MP4 requires an EOS signal at the end of the file, if you are saving those videos using Gstreamer then add a " -e " at the end of the pipeline.

Since i could not play your video with Ubuntu, i expected an internal error with Gstreamer pipeline.

Can you share the recording pipeline? (if you are using Gstreamer)

From personal notes i have this pipeline to record MP4 videos, i hope this helps.

gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxvp8enc bitrate=8000000 ! qtmux ! perf ! filesink location=imx230.mp4 -e

Regards,
Greivin F.

10

Can you try the following pipelines?

Record MP4

gst-launch-1.0 videotestsrc is-live=true pattern=ball ! nvvidconv  ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxvp8enc bitrate=8000000 ! qtmux  ! filesink location=test.mp4 -e

Play MP4

gst-launch-1.0 -v filesrc location=test.mp4 ! decodebin   ! nvvidconv  ! nveglglessink -e
11

The video file was being captured by the following ffmpeg command from my IP camera:
ffmpeg -v verbose -i rtsp://:554 -vcodec h264 -acodec aac -f tee -map 0:v -map 0:a -timeout -1 -listen 1 “[f=mpegts]udp://:port|[f=segment:segment_time=60:segment_format=mp4:strftime=1]%Y-%m-%d_%H-%M-%S_saved.mp4”

As I mentioned in this thread earlier, and it’s also the same ffmpeg command I grab from ip camera to my stream server.

There is NO gstreamer pipeline at this moment. I guess my question for you is that how you grab it with GStreamer pipelines from IP camera and decode it SUCCESSFULLY. – That’s My original tasks to port all we have under Jestson and that’s the reason I raised this whole thread at the first place.

…ssu

ps: BTW, the video file you have, it can can be played with “nvgstplayer-1.0 -i *.mp4” Did you try it ?

12

gst-launch-1.0 videotestsrc is-live=true pattern=ball ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! omxvp8enc bitrate=8000000 ! qtmux ! filesink location=test.mp4 -e

How did you convert the above commands to record from IP camera video rtsp source? and fit into my video http server (such as nvhtttpsink to my servert and port number, my guess) or let me try …

Thanks,
…ssu

13

Hi,
Suggest you go to http://gstreamer-devel.966125.n4.nabble.com/ and get working pipeline with rtspsrc and avdec_h264. Once you have a pipeline working with avdec_h264, it should also work with omxh264dec.

14

OK, Now I’m converting all my streaming server into GStreamer-based, instead of ffmpeg. However I still cannot get it to work on “a pipeline working with avdec_h264, it should also work with omxh264dec”, not to mention the HAL decoderr/encoder on Jestson. (it crashed as well on Jestson via VLC as well. But it works on Ubuntu/VLC)

//info

The following command is working on desktop/ubuntu18.04 BUT NOT on JESTSON (Nothing show) .

DISPLAY=:0 gst-launch-1.0 --gst-debug=3 -v rtspsrc location=rtsp://:port ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink sync=false

Please advice.

The end of the output string is on Jestson:

rb-jitter=(uint)0,\ rb-lsr=(uint)0,\ rb-dlsr=(uint)0,\ rb-round-trip=(uint)0;", “application/x-rtp-source-stats,\ ssrc=(uint)1933072347,\ internal=(boolean)true,\ validated=(boolean)true,\ received-bye=(boolean)false,\ is-csrc=(boolean)false,\ is-sender=(boolean)false,\ seqnum-base=(int)-1,\ clock-rate=(int)-1,\ octets-sent=(guint64)0,\ packets-sent=(guint64)0,\ octets-received=(guint64)0,\ packets-received=(guint64)0,\ bitrate=(guint64)0,\ packets-lost=(int)0,\ jitter=(uint)0,\ sent-pli-count=(uint)0,\ recv-pli-count=(uint)0,\ sent-fir-count=(uint)0,\ recv-fir-count=(uint)0,\ sent-nack-count=(uint)0,\ recv-nack-count=(uint)0,\ have-sr=(boolean)false,\ sr-ntptime=(guint64)0,\ sr-rtptime=(uint)0,\ sr-octet-count=(uint)0,\ sr-packet-count=(uint)0;” >, rtx-count=(uint)0;

15

Hi,
Please try to dump and inspect the decoded YUVs.

gst-launch-1.0 --gst-debug=3 -v rtspsrc location=rtsp://<NVR IP>:port ! rtph264depay ! h264parse ! avdec_h264 ! multifilesink location=dump%5d.yuv
16

OK, I have run the command, as expected no show screen but only some console output text.

Here is 1 frame YUV file.
https://gokeuslab-my.sharepoint.com/:u:/p/vincent_su/EbwRJbagBMhJlKcWWRLJccMB0ijcu7hZMJDk4m0KSQ2hPA?e=Iso9yg

Here are 50 frames YUV files.
https://gokeuslab-my.sharepoint.com/:u:/p/vincent_su/EYcz9jf6FXNAqLdfZsXdfwgBKzvWTu9SILEAe4z5aDD3oQ?e=xhoCHe

Please advice

17

Hi,
The 4K YUVs look good. Please try

gst-launch-1.0 --gst-debug=3 -v rtspsrc location=rtsp://<NVR IP>:port ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw,format=I420 ! multifilesink location=dump%05d.yuv
18

This is the file generated by omxh264, instead of avdec_h264.

https://gokeuslab-my.sharepoint.com/:u:/p/vincent_su/EYJCWyRskdlKtkIoa8jsV5sBWGma6OfEM62Ya9mfjSSNYg?e=5HFbv5

Although it looks good on the YUVs, it has caused Jeston GPU to freeze easily, if there’s too many GUI activities in the mean time, such as browsing Net.

19

Hi,
Please try

gst-launch-1.0 --gst-debug=3 -v rtspsrc location=rtsp://<NVR IP>:port ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12,width=1920,height=1080' ! nvoverlaysink