Gstreamer-1.0 Issue

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

I have developed TC358748 Parallel to MIPI driver. And i can captures 1080p frames using yavta.
I can also preview and record h264 video using gstreamer-1.0.

I used follwing command for preview. Sensor output is in UYVY format.

gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=30/1' ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=30/1 ! autovideosink

If i use Gstreamer-1.0 v1.2.4 i get no latency in previewing 1080p 30/60 fps video.
But if i use Gstreamer-1.8.1 or 1.6.0 with the same command i got latency of around 500ms-1000ms with following debug

Additional debug info:
gstbasesink.c(2846): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstNvOverlaySink-nvoverlaysink:autovideosink0-actual-sink-nvoverlay:
There may be a timestamping problem, or this computer is too slow.

If i add sync=false at last i got no debug info and latency of around 300-500ms but not good as compared to v1.2.4.

I installed gstreamer-1.6.0 using “gst-install --prefix=/home/ubuntu/gst-1.6.0 --version=1.6.0” and then i export

export LD_LIBRARY_PATH=/home/ubuntu/gst-1.6.0/lib/arm-linux-gnueabihf
export PATH=/home/ubuntu/gst-1.6.0/bin:$PATH

So whats the reason of this latency?

And using gstreamer 1.6.0 and 1.8.1 i can encode h264 and record .mp4 video using

gst-launch-1.0 v4l2src ! 'video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=30/1' ! videoconvert ! video/x-raw,format=I420,width=1920,height=1080,framerate=30/1 ! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e

But i got same latency in recorded video.

I can’t test recording with gstreamer 1.2.4 because h264parse is missing in v1.2.4. So is there any other alternative for h264 encoding in v1.2.4?

And also how can i use nvvidconv for preview and record? Does it improve latency?

2

Hi

Generally I would advise to use nvvidconv instead of videoconvert for performance reasons. I have been testing something similar with a TC358840.
Preview with nvvidconv should work with a pipeline like this:

gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, format=UYVY, framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1' ! nvoverlaysink -vvv

Recording with nvvidconv should be possible too like this (maybe you can even leave the blocksize parameter to the v4l2src away):

gst-launch-1.0 v4l2src device=/dev/video0 do-timestamp=true blocksize=3110400 ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=I420' ! omxh264enc ! h264parse ! qtmux ! filesink location=test_input.mp4 sync=false -e -vvv

In terms of the latency I’m not sure where the difference between GSt1.2.4 and 1.6/1.8 comes from. But it sure makes a difference wether you use the sync=false parameter on the filesink. For live sources (like the HDMI input) it probably makes sense to set it to false, since this means that every frame that comes to the filesink will be handled immediately. If you set sync to True, it will compare the timestamp of the frame to what would be expected for the current frame. Meaning that if the delay through your pipeline is too large, the frame will be discarded.

To get the h264parse element in GSt1.2.4 you need to install the package gstreamer1.0-plugins-bad. If you dont find it with apt-cache search, you might need to enable the Universe repository in Ubuntu.

Regards

3

Thanks Kamm for the reply.

Your Both pipelines works with Gstreamer-1.2.4.
i got h264parse by installing gstreamer1.0-plugins-bad using apt-get.

But both these links not working with gstreamer-1.6.0. Got following error.

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2943): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason error (-5)
Execution ended after 0:00:00.070023267
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I think nvvidconv doing problem with 1.6.0, while videoconvert works good but giving latency.

4

with gstreamer 1.6.0 i can not run nvvidconv pipeline from Multimedia guide.

gst-launch-1.0 videotestsrc ! 'video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720' ! nvvidconv ! 'video/x-raw(memory:NVMM)’ ! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e

I got syntax error of ‘(’. And if i remove (string) from (string)byte-stream no error but no pipeline started just “>” symbol in terminal.

So how to use nvvidconv with higher gstreamer version than 1.2.4?

5

Unfortunately I can not use the nvvidconv element with GStreamer 1.6 either.

I think the problem is that nvvidconv is proprietary code and not included in the libgstomx.so sources for gstreamer1.0 available from Nvidia. Therefore when we rebuild the vendor specific GStreamer elements, we are still missing nvvidconv.

So unless Nvidia releases the sources for nvvidconv or updates the version of GStreamer they support (maybe in the next L4T release?) I can’t help you out with that.

6

Also for your pipeline with the H264 encoder, the encoder only supports the I420 and NV12 format, and you need to specify this explicitely.

gst-launch-1.0 videotestsrc ! 'video/x-raw, format=UYVY, width=1280, height=720, framerate=30/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420' ! omxh264enc ! 'video/x-h264, stream-format=byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e

By the way it looks like nvvidconv works with videotestsrc but only fails when used with v4l2src.

7

gstomx1_src is provided by nvidia.
So can we build with it gstreamer v1.6 or 1.8 to get all nv plugins?

I tried and built it with gstreamer 1.8.1. But after build it only generating libgstnveglglessink and libgstomx .so and .la.

no nvoverlaysink or other element in gstreamer.

8

If you check the “TEGRA X1/TEGRA LINUX DRIVER PACKAGE MULTIMEDIA USER GUIDE” you will see that Nvidia proprietary libraries have to be copied over to the new compile you have done:

9

I know this steps if i copied Nvidia proprietary libraries, i get nvoverlaysink and other plugins in gstreamer 1.8.1.

But currently my problem is i unable to use nvvidconv with new gstreamer version other than 1.2.4 that i have mentioned in previous post.

And if i use videoconvert plugin for UYVY to I420 conversion in gstreamer version other than 1.2.4, It adds latency in preview also.
But with 1.2.4 videoconvert and nvvidconv works fine.

So i thought may be compiling gstomx1_src with gstreamer-1.8.1 use hardware acceleration or remove pipeline ‘internal data flow error’.