Problems with reading and streaming rtsp with an h264 camera

Hi all.
I’m trying to stream an h264 stream from my jetson nano.
Right now I’m using this camera:
http://www.webcamerausb.com/elp-2mp-h264-sony-imx322-001lux-usb-camera-low-illumination-elpusbfhd06hl36-p-187.html

Using this command:
./test-launch “v4l2src device=/dev/video1 ! video/x-h264,width=1920,height=1080,framerate=15/1 ! rtph264pay name=pay0 pt=96”

And stream is mixed up.
I get this stream:

Furthermore, when tryin to read the rtsp from FFMPEG and VLC I get:
errors in P frame
MV errors in I frame
error while decoding

You can see the at the lower part the video doesn’t work good.
Is there anything I can change in the pipeline I can change to make it work better?

Thanks.

Not sure, but you may try adding h264parse:

./test-launch "v4l2src device=/dev/video1 ! video/x-h264,width=1920,height=1080,framerate=15/1 ! h264parse ! rtph264pay name=pay0 pt=96"

If this doesn’t improve, you would try to rule out rtp and network issues decoding and displaying on Jetson if you have a monitor connected:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=15/1 ! h264parse ! nvv4l2decoder ! nvoverlaysink

Hi, thanks.
I’ve tried to use h264 parse but unfortunately it is even worse - I get a lot of “lost frames detected: count = 1”.
Furthermore, when tryin to read the rtsp from FFMPEG and VLC I get:
errors in P frame
MV errors in I frame
error while decoding

Hi,
Please check if the stream is good in local video playback:

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-h264,width=1920,height=1080,framerate=15/1 ! h264parse ! nvv4l2decoder ! nvoverlaysink

Hi thanks.
I solved it by

sudo apt upgrade

The instillation of connectech board says not to do so, but after doing it everything works.

Thank you

Hi sorry.
I thought I solved it, but I only managed to read the stream right on ffmpeg, but on vlc or gst it still does it.
Unfortunately I don’t have a screen connected, it is a server. Is there anything else I can check?

Thanks.

Hi,
Maybe you can dump decoded YUVs and check with YUV viewer. Please try

gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-h264,width=1920,height=1080,framerate=15/1 ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw ! multifilesink location=dump%05d.yuv

Hi, I couldn’t manage to read the yuv files.
But when I used this pipe:
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-h264, width=1920, height=1080, framerate=15/1 ! filesink location=output1.mp4
Again I manage to see very good video on my computer using ffmpeg, but the vlc and gst still doesn’t work.
I wonder what can it be?

Looking at the link you’ve provided in first post, it seems the native mode for 1080p is @30 fps.
You may further check with v4l2-ctl (provided by apt package v4l-utils):

v4l2-ctl -d1 --list-formats-ext

I’d suggest to first try with 30 fps.
You may also embed your h264 stream into a container, such as mp4 for file or rtph264 for streaming.
Saving to file :

gst-launch-1.0 -e v4l2src device=/dev/video1 ! video/x-h264,width=1920,height=1080,framerate=30/1 ! h264parse ! qtmux ! filesink location=test.mp4

You should be able to see this from ffmpeg, VLC (note that you may have to remove a plugin), or from gstreamer:

gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvoverlaysink

If this doesn’t work, you may try to get profile and level of your camera h264 encoding and add these into caps after v4l2src.

If it works, for streaming with test-launch, you would try:

test-launch "v4l2src device=/dev/video1 do-timestamp=true ! video/x-h264,width=1920,height=1080,framerate=30/1 ! h264parse ! rtph264pay pt=96 name=pay0 " 

Hi! thanks for the answer!
I’ve tried the recording, and when I try to play it I get this error:

ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: This file contains no playable streams.
Additional debug info:
qtdemux.c(701): gst_qtdemux_post_no_playable_stream_error (): /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
no known streams found

When I try to stream like suggested I get a stream only with ffplay - but with a lot more smudge and green screens., and on the streaming side I get:

lost frames detected: count = 1

When I remove the h264parse I get the same output as before - ffplay works good, vlc works but very smudgy with grey screen and publishing gst to kvs doesn’t work as well.

I use the same script and the same camera on UpBoard and everything works good, I was wondering maybe the cameras get any kind of init params that need to be changed before using them, but I can’t tell the difference.

Thanks again.

Hi,
Please set up a RTSP server with videotestsrc on Jetson Nano and check if this works. You may refer to
Jetson Nano FAQ
Q: Is there any example of running RTSP streaming?

Sorry, I missed the EOS flag that may be required for this case. Edited my previous post for correcting that.

Another possible cause might be that test-launch mainly supports baseline profile for H264. Does it work if you decode and re-encode with baseline profile ?

test-launch "v4l2src device=/dev/video1 do-timestamp=true ! video/x-h264,width=1920,height=1080,framerate=30/1 ! nvv4l2decoder ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 ! video/x-h264, stream-format=byte-stream, profile=baseline ! rtph264pay pt=96 name=pay0 "

Thanks a lot!
The re-encoding actually works!
Even if I change the profile to be “main” it works,
what does it mean? that the the encoding on camera doesn’t work good?

Thanks again!

I was not correct. This was omxh264enc related.
I think that for each profile, not any level may be expected, and maybe your camera does’t use an expected level…can you configure the camera H264 encoding ?

Unfortunatly I could not configure the camera H264 encoding.
I have decided to use the method you suggested, but there is another problem.

If I try to play the stream, then stop the playing and start it again, it just won’t start again and I need to kill the process and start again.

With debug 3 (on the second time) I get:

Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:13.857662305 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x7f400166d0 Failed to determine interlace mode
0:00:13.857732410 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x7f400166d0 Failed to determine interlace mode
0:00:13.857833192 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x7f400166d0 Failed to determine interlace mode
0:00:13.857944965 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:nvv4l2h264enc1:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Opening in BLOCKING MODE
Opening in BLOCKING MODE
0:00:13.901356836 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:nvv4l2decoder1:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:13.901416889 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:<(null)@0x7f40013bf0> Failed to determine interlace mode
0:00:13.901463504 1282 0x7f4c003e30 WARN v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:<(null)@0x7f40013bf0> Failed to determine interlace mode
Opening in BLOCKING MODE
0:00:33.808334587 1282 0x7f502fbc50 WARN rtspmedia rtsp-media.c:2994:wait_preroll: failed to preroll pipeline
0:00:33.808378858 1282 0x7f502fbc50 WARN rtspmedia rtsp-media.c:3298:gst_rtsp_media_prepare: failed to preroll pipeline

Edit:
When I use omxh264dec and omxh264enc it doesn’t happen

Hi,
Please share your release version( $ head -1 /etc/nv_tegra_release ).

And does it happen if you run this RTSP server:

$ ./test-launch "videotestsrc ! nvvidconv ! nvv4l2h264enc ! h264parse ! rtph264pay name=pay0 pt=96"

Hi, thanks for the reply.
My version is:

R32 (release), REVISION: 5.0, GCID: 25531747, BOARD: t210ref, EABI: aarch64, DATE: Fri Jan 15 22:55:35 UTC 2021

It doesn’t happen here.
I’ve tried to mix the plugins, and seems that the problem is with nvv4l2h264enc.

You may first rule out any USB bandwith issue checking with:

lsusb -t

If you have other USB cams or high speed devices, you may try without these.

You may also post the output of:

v4l2-ctl -d1 --all
v4l2-ctl -d1 --list-formats-ext

I have no such camera and no Nano, but I fail to reproduce your case from NX running R32.5.1 and simulating H264 camera with v4l2loopback. In my case I can RTSP stream without decoding/encoding, omxh264enc can also acheive high framerate, and in each case reading the stream (thru localhost), pausing, stopping restarting with VLC all work (only see increasing latency, might be VLC related), so I just wonder if you have the correct codec data for your cam.
You may contact cam vendor support for better support about this.

Hi,
Please apply this prebuilt lib:
Libv4l2_nvargus.so source code - #19 by JerryChang