Improving latency on YUV camera live streaming

[yuyeo0605]

Hi,

Recently, I have done a live streaming by using a YUV camera (Output : YUV422 8bit).

However, I used the command below,

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvoverlaysink

and I can see that there is a latency during live streaming.

It takes about 200ms.

Does that command make the live streaming slow?

or do I have to reset the parameters on the device tree file?

Thanks,

[ShaneCCC]

Try below command before gst-launch-1.0

v4l2-ctl -c low_latency_mode=1

[yuyeo0605]

Hi,

Still get the same situation.

What does that command mean?

Thanks

[ShaneCCC]

Sorry for typo should be low_latency_mode=1

[yuyeo0605]

Still same latency, about 200ms

Thanks

[ShaneCCC]

How do you check the latency?

[ShaneCCC]

Have a try nvv4l2camerasrc

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! nv3dsink -e

[yuyeo0605]

When I execute this command,

k@k-desktop:~$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)NV12’ ! nv3dsink -e

I get this error.

bash: syntax error near unexpected token `('

So I erase the ‘(’ like this,

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! ‘video/x-raw, width=1920, height=1080, framerate=60/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw, format=NV12’ ! nv3dsink -e

Then I get this error,

(gst-launch-1.0:9931): GStreamer-CRITICAL **: 14:57:39.662: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed

(gst-launch-1.0:9931): GStreamer-CRITICAL **: 14:57:39.664: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
WARNING: erroneous pipeline: syntax error

[yuyeo0605]

I connected my device with my laptop.

And I did the live streaming as a duplicate mode.

Then I started the clock on my laptop like the pic below.

so I record it with my smart phone then compared the latency between the screens

I know this is not the right method for checking it but I don’t know the exact method for how to check the latency.

[ShaneCCC]

Try copy this command.

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)UYVY" ! nvvidconv ! "video/x-raw(memory:NVMM), format=(string)NV12" ! nv3dsink -e

[yuyeo0605]

Thank you.

It works and the latency has been reduced.

I think there are two main changes: nvv4l2camerasrc and nv3dsink -e.

Can you explain this command?

How can be the latency reduced by only changing the command?

Thanks

[ShaneCCC]

Suppose it could be the v4l2src may have memory copy cause the latency but nvv4l2camerasrc doesn’t.

[yuyeo0605]

Oh… that’s the reason, I see.

One more question!

I am using the Xavier NX Dev Kit and I connected my device on the CSI connector like the below picture,

Do you know how many seconds it takes from the input CSI connector to the live streaming.?

What I mean is that I want to know the latency for the processing the input MIPI data.

Thank you.

[ShaneCCC]

Maybe about 80 -120 ms

[yuyeo0605]

Is it the time right before the live streaming or the time including it?

[ShaneCCC]

It’s glass to glass timing. From sensor lens to display.

[yuyeo0605]

Okay I got it.

Thank you so much.