GST UDP-RTP Delay

[rlatae123]

We are receiving UYVY (30FPS) via UDP-RTP pipeline from NX to PC and transmitting it as JPEG data.

When I took a time stamp on the data coming in through APPSINK, 33ms was not coming in.

SEND

"gst-launch-1.0 nvv4l2camerasrc device=/dev/video2 ! nvvidconv ! nvjpegenc ! rtpjpegpay mtu=64000 ! queue max-size-time=5000000000 max-size-buffers=10000000 max-size-bytes=50000000 ! udpsink host=192.168.8.8 port=50000

RECEIVE

It operates normally for 1CH, but if it exceeds that, symptoms appear.

1ch

4ch

Is there anything I can fix about this?

[DaneLLL]

Hi,
We have some suggestion to set up Xavier NX for the use-case:

1. Please execute sudo jetson_clocks to enable hardware engines at maximum throughput
2. Please assign different port number to each channel, to separate the RTP stream in different port
3. If your camera sourcs are connected to CSI ports, please enable CSI and VI engines at maximum clock by executing the commands:
   Jetson/l4t/Camera BringUp - eLinux.org

[rlatae123]

1. I ran sudo jetson_clocks
2. Different ports are used for each camera. ex) CAM1 - 50000, CAM2 - 50001…
3. I ran below command
   sudo su
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
   cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
   cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
   cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

I tried all three above, but the results were the same.

[rlatae123]

@DaneLLL
There seems to be a delay of about 0.3 to 4 ms for each channel.

[DaneLLL]

Hi,
Please run sudo tegrastats and see if there is more information. The capability of CPU cores of Xavier NX is not as good as AGX Xavier. This may be bottleneck in the use-case.

[rlatae123]

Hello, @DaneLLL

[rlatae123]

Looking at tegrastats and top, I don’t think there will be a CPU bottleneck. Could there be some other reason?

[DaneLLL]

Hi,
It seems like you have customization to the power mode. By default we have 6 cores@1.4GHz and 2 cores@1.9GHz. Your tegrastats shows 6 cores@1.9GHz. Maybe you can try default power modes.

And may try
taskset(1) - Linux manual page
to schedule gst-launch-1.0 commands to individual cores.

Adn can compare with videotestsrc to check if the latency is from the source:
Gstreamer TCPserversink 2-3 seconds latency - #5 by DaneLLL

[rlatae123]

Hello

I set this to maximize CPU performance. I changed it to basic mode and applied taskset to gst-launch-1.0, but it was the same as before.

When videotestsrc is-live=true was used instead of nvv4l2camerasrc, 4ch worked normally without delay.

How is it different from nvv4l2camerasrc?

[DaneLLL]

Hi,
Please try v4l2src.

[rlatae123]

Hi,

When using v4l2src based on 1 channel, data seems to come in at 37ms intervals as shown below.

When using 4 channels, there is a delay of 0.3 to 4 ms as before.

[DaneLLL]

Hi,
Ok. From your previous description, it sounds like the latency occurs only in nvv4l2camerasrc. If both plugins have the same phenomenon, the latency looks to be from the camera sources. Do you use YUV camera sensors connecting to CSI ports?

[rlatae123]

Our board is structured below.

imx390(raw) - isp(uyvy) - NX

[DaneLLL]

Hi,
If you have executed the steps:

$ sudo jetson_clocks

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

The hardware engines are in maximum throughout and no room for further improvement. Probably the CSI and VI engines of Xavier NX cannot achieve 30fps for 4 sources.

One more thing to try is if you do not use Jetpack 4.6.4 or 5.1.2, may upgrade to latest version for a try.

[rlatae123]

Hi

We are testing on JETPACK 4.6.4.

I don’t know what’s causing it

I’ll check a little more…