From time to time，NvEglRenderer cannot be Displayed in time

happy886 · February 17, 2023, 7:28am

agx version
nvidia@localhost:~$ head -n 1 /etc/nv_tegra_release
# R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020
60 fps fpdlink camera
sudo nvpmodel -m 0 and sudo jetson_clocks is fine!

There are about three threads,Modified from demo【jetson_multimedia_api\samples\00_video_decode】：
Thread one:As soon as the rtp video stream is parsed out of nalu, it is put into queue1;
Thread two: NvVideoDecoder decode queue1, decode yuv data put queue2;
Thread three: NvEglRender renderer decodes yuv queue data queue2;

happy886 · February 17, 2023, 7:39am

As shown in the above post image
The log shows that 7 frames of video data are decoded at 15:06:09, but the display queue is not updated to display new data, and 1466 frames of old video are still displayed
After some time, 10 frames of video data were decoded at 15:08:40, but the display queue was not updated to display new data, and 10,495 frames of old video were still displayed
How to solve this problem of not being able to refresh the latest video in time?
This is a continuation of the problem in the previous post【Agx Dynamic object video 60fps rtp send and recv some question? - #18 by DaneLLL

DaneLLL · February 17, 2023, 8:20am

Hi,
There is waiting pthread_cond_timedwait() in renderInternal(). Please remove it and give it a try. To call eglSwapBuffers() without the waiting and see if it helps.

happy886 · February 23, 2023, 5:18am

yes,it helps
However, long time test, still appear from time to time.
It seems that the render thread is susceptible to other effects

DaneLLL · February 23, 2023, 6:42am

Hi,
Could you add gettimeofday() before and after eglswapBuffers() to confirm the function is getting slower time after time? We think this function should work without any issue. Would like to confirm this first.

happy886 · February 23, 2023, 7:16am

Each frame is time-consuming, usually 14ms, but also time-consuming 38ms;
Spent more than 30 ms print, statistics for a period of time, there are 52 ms, ms, 57 74 ms…
60fps,NvEglRender also set 60fps//renderer->setFPS(60),and xrandr --output HDMI-0 --mode 1920x1080 --rate 60.02

happy886 · February 23, 2023, 7:17am

DaneLLL · February 23, 2023, 7:39am

Hi,
Could you try Xavier developer kit + Jetpack 4.6.3? If the latest Jetpack 4 release does not have the issue, you may consider upgrade to later release.

happy886 · February 27, 2023, 12:53am

agx Xavier
nvidia@tegra-ubuntu:~$ head -n 1 /etc/nv_tegra_release

R35 (release), REVISION: 1.0, GCID: 31346300, BOARD: t186ref, EABI: aarch64, DATE: Thu Aug 25 18:41:45 UTC 2022

The same test, also has this problem

2023-02-27 08:46:45.412 video_decode_main.cpp:392 render loop thread start,g_que size:0,numCapBuffers:5
2023-02-27 08:46:45.413 video_decode_main.cpp:330 read_decoder_chunk nalu type:1
2023-02-27 08:46:45.414 video_decode_main.cpp:330 read_decoder_chunk nalu type:32
Starting decoder capture loop thread
2023-02-27 08:46:45.414 video_decode_main.cpp:1015 decoder block input thread start
Video Resolution: 3840x2160
Decoder colorspace ITU-R BT.601 with standard range luma (16-235)
2023-02-27 08:46:45.515 video_decode_main.cpp:607 —ctx numCapBuffers:5
Query and set capture successful
2023-02-27 08:46:45.564 video_decode_main.cpp:406 render loop que long error… 5
2023-02-27 08:46:45.564 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:46:48.903 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:47:22.622 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:23
2023-02-27 08:47:34.579 video_decode_main.cpp:136 rtp seq error cur:4415,need:14414
2023-02-27 08:47:34.614 video_decode_main.cpp:406 render loop que long error… 5
2023-02-27 08:47:34.614 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:47:34.631 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:47:55.269 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:48:16.336 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:48:50.019 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:02.653 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:06.870 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:49:13.186 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:33.219 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:49:38.490 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:24
2023-02-27 08:49:47.147 video_decode_main.cpp:406 render loop que long error… 4
2023-02-27 08:49:53.239 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:24
2023-02-27 08:50:02.720 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:50:06.937 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:21
2023-02-27 08:50:13.252 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:21.685 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:26.952 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:50:55.405 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:23
2023-02-27 08:51:05.935 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:51:18.585 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20
2023-02-27 08:51:23.854 NvEglRenderer.cpp:454 ###eglSwapBuffers cost:20

happy886 · February 27, 2023, 1:08am

happy886 · February 28, 2023, 1:21am

@DaneLLL Is there any good solution ？

DaneLLL · March 1, 2023, 7:00am

Hi,

Does this mean 20 milliseconds? The rendering is related to fps of display. If your display mode is in 30fps, 20 millisecond look fine since it renders the frame out per vsync.

happy886 · March 1, 2023, 7:04am

The previous post stated that 60fps was set

happy886 · March 1, 2023, 7:07am

20ms may be fine, but every now and then it takes 52,57,74ms

DaneLLL · March 8, 2023, 8:57am

Hi,
We try to run long-run video playback by applying the patch:

diff --git a/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp b/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
index a6c26df..98c5f84 100644
--- a/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
+++ b/multimedia_api/ll_samples/samples/00_video_decode/video_decode_main.cpp
@@ -199,6 +199,8 @@ read_decoder_input_chunk(ifstream * stream, NvBuffer * buffer)
     {
         stream->clear();
         stream->seekg(0,stream->beg);
+        stream->read((char *) buffer->planes[0].data, bytes_to_read);
+        buffer->planes[0].bytesused = stream->gcount();
     }
     return 0;
 }

And run

00_video_decode$ ./video_decode H264 …/…/data/Video/sample_outdoor_car_1080p_10fps.h264

Do not hit the issue in 4+ hour run. Please try the same and make sure the TV is always on(not go to blank screen).

happy886 · March 8, 2023, 9:00am

We also didn’t have this problem with 30fps cameras, but we had this problem with 60fps cameras

Demo is to read the file, after decoding sequence execute render;

My test is 60 FPS camera, RTP data and read the file there is a difference between, RTP, transmission of data dynamically

When I found the problem, I specially did multiple threads processing, rtp packet parsing nalu thread, decoding thread, display render thread;

Normal reading files did not test this problem, but 60fps cameras with large data volumes and dynamic changes in rtp packets did have this problem

DaneLLL · March 9, 2023, 1:44am

Hi,
We try the command and do not observe the issue after 14+ hour run:

00_video_decode$ ./video_decode H264 -fps 60 …/…/data/Video/sample_outdoor_car_1080p_10fps.h264

It still renders in 60fps after overnight. Is it possible your device is in high temperature and triggers throttling, or the display output cannot render frames in time?

happy886 · March 10, 2023, 9:11am

NvEglRender sample test
and
sudo xrandr --output HDMI-0 --mode 1920x1080 --rate 60.00

Although this simple test doesn’t take as long as the actual 60fps camera data test in 50, 70 milliseconds .
But 20 milliseconds is pretty frequent.

test.tar.gz (6.8 KB)

system · April 25, 2024, 8:48am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Why two NvEglRenderer on two threads cannot run at maximum fps in Jetson 35.2.1 (it worked in 35.1.0)? Jetson AGX Orin opengl	4	553	March 22, 2023
Multiple instances of NvEglRenderer Jetson TX2 mmapi , nvbugs	10	1388	October 18, 2021
Encoding performance issue on Xavier AGX, but no problem on Nano (simultaneous encoding) Jetson AGX Xavier gstreamer , nvbugs	7	1615	September 19, 2022
Use 00_ video_ Decode and render a frame of image (640x480) Jetson Xavier NX mmapi	11	541	March 9, 2022
Xavier NX hardware JPEG encoder extremely slow Jetson Xavier NX encoder	11	1070	February 21, 2023
xavier encode and decode do not match official description Jetson AGX Xavier	3	1123	October 18, 2021
Image display delay issue Jetson AGX Xavier camera	10	557	May 26, 2022
How much cameras AXG Xavier can processed? Jetson AGX Xavier camera , gstreamer	33	3355	October 18, 2021
Anomalous performance behavior of nvp-models when encoding h264-video Jetson TX2	14	1486	October 18, 2021
No EGL device available Jetson TX1	28	2735	October 18, 2021

From time to time，NvEglRenderer cannot be Displayed in time

R35 (release), REVISION: 1.0, GCID: 31346300, BOARD: t186ref, EABI: aarch64, DATE: Thu Aug 25 18:41:45 UTC 2022

Related topics