From time to time,NvEglRenderer cannot be Displayed in time

agx version
nvidia@localhost:~$ head -n 1 /etc/nv_tegra_release
# R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020
60 fps fpdlink camera
sudo nvpmodel -m 0 and sudo jetson_clocks is fine!

There are about three threads,Modified from demo【jetson_multimedia_api\samples\00_video_decode】:
Thread one:As soon as the rtp video stream is parsed out of nalu, it is put into queue1;
Thread two: NvVideoDecoder decode queue1, decode yuv data put queue2;
Thread three: NvEglRender renderer decodes yuv queue data queue2;

As shown in the above post image
The log shows that 7 frames of video data are decoded at 15:06:09, but the display queue is not updated to display new data, and 1466 frames of old video are still displayed
After some time, 10 frames of video data were decoded at 15:08:40, but the display queue was not updated to display new data, and 10,495 frames of old video were still displayed
How to solve this problem of not being able to refresh the latest video in time?
This is a continuation of the problem in the previous post【Agx Dynamic object video 60fps rtp send and recv some question? - #18 by DaneLLL

There is waiting pthread_cond_timedwait() in renderInternal(). Please remove it and give it a try. To call eglSwapBuffers() without the waiting and see if it helps.

yes,it helps
However, long time test, still appear from time to time.
It seems that the render thread is susceptible to other effects

Could you add gettimeofday() before and after eglswapBuffers() to confirm the function is getting slower time after time? We think this function should work without any issue. Would like to confirm this first.

Each frame is time-consuming, usually 14ms, but also time-consuming 38ms;
Spent more than 30 ms print, statistics for a period of time, there are 52 ms, ms, 57 74 ms…
60fps,NvEglRender also set 60fps//renderer->setFPS(60),and xrandr --output HDMI-0 --mode 1920x1080 --rate 60.02

Could you try Xavier developer kit + Jetpack 4.6.3? If the latest Jetpack 4 release does not have the issue, you may consider upgrade to later release.

agx Xavier
nvidia@tegra-ubuntu:~$ head -n 1 /etc/nv_tegra_release

R35 (release), REVISION: 1.0, GCID: 31346300, BOARD: t186ref, EABI: aarch64, DATE: Thu Aug 25 18:41:45 UTC 2022

The same test, also has this problem

2023-02-27 08:46:45.412 video_decode_main.cpp:392 render loop thread start,g_que size:0,numCapBuffers:5
2023-02-27 08:46:45.413 video_decode_main.cpp:330 read_decoder_chunk nalu type:1
2023-02-27 08:46:45.414 video_decode_main.cpp:330 read_decoder_chunk nalu type:32
Starting decoder capture loop thread
2023-02-27 08:46:45.414 video_decode_main.cpp:1015 decoder block input thread start
Video Resolution: 3840x2160
Decoder colorspace ITU-R BT.601 with standard range luma (16-235)
2023-02-27 08:46:45.515 video_decode_main.cpp:607 —ctx numCapBuffers:5
Query and set capture successful
2023-02-27 08:46:45.564 video_decode_main.cpp:406 render loop que long error… 5
2023-02-27 08:46:45.564 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:46:48.903 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:47:22.622 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:23
2023-02-27 08:47:34.579 video_decode_main.cpp:136 rtp seq error cur:4415,need:14414
2023-02-27 08:47:34.614 video_decode_main.cpp:406 render loop que long error… 5
2023-02-27 08:47:34.614 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:47:34.631 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:47:55.269 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:48:16.336 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:48:50.019 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:02.653 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:06.870 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:49:13.186 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:33.219 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:38.490 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:24
2023-02-27 08:49:47.147 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:49:53.239 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:24
2023-02-27 08:50:02.720 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:50:06.937 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:50:13.252 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:21.685 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:26.952 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:55.405 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:23
2023-02-27 08:51:05.935 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:51:18.585 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:51:23.854 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20

@DaneLLL Is there any good solution ?


Does this mean 20 milliseconds? The rendering is related to fps of display. If your display mode is in 30fps, 20 millisecond look fine since it renders the frame out per vsync.

The previous post stated that 60fps was set

20ms may be fine, but every now and then it takes 52,57,74ms

We try to run long-run video playback by applying the patch:

diff --git a/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp b/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
index a6c26df..98c5f84 100644
--- a/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
+++ b/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
@@ -199,6 +199,8 @@ read_decoder_input_chunk(ifstream * stream, NvBuffer * buffer)
+        stream->read((char *) buffer->planes[0].data, bytes_to_read);
+        buffer->planes[0].bytesused = stream->gcount();
     return 0;

And run

00_video_decode$ ./video_decode H264 …/…/data/Video/sample_outdoor_car_1080p_10fps.h264

Do not hit the issue in 4+ hour run. Please try the same and make sure the TV is always on(not go to blank screen).

We also didn’t have this problem with 30fps cameras, but we had this problem with 60fps cameras

Demo is to read the file, after decoding sequence execute render;

My test is 60 FPS camera, RTP data and read the file there is a difference between, RTP, transmission of data dynamically

When I found the problem, I specially did multiple threads processing, rtp packet parsing nalu thread, decoding thread, display render thread;

Normal reading files did not test this problem, but 60fps cameras with large data volumes and dynamic changes in rtp packets did have this problem

We try the command and do not observe the issue after 14+ hour run:

00_video_decode$ ./video_decode H264 -fps 60 …/…/data/Video/sample_outdoor_car_1080p_10fps.h264

It still renders in 60fps after overnight. Is it possible your device is in high temperature and triggers throttling, or the display output cannot render frames in time?

NvEglRender sample test
sudo xrandr --output HDMI-0 --mode 1920x1080 --rate 60.00

Although this simple test doesn’t take as long as the actual 60fps camera data test in 50, 70 milliseconds .
But 20 milliseconds is pretty frequent.

test.tar.gz (6.8 KB)

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.