NvDRMRenderer fps setting

Hey,
I am new to the whole low latency thing, but I am trying to display a video stream with the lowest possible latency.
For capturing I use argus api and for display I use the NvDRMRenderer.

So I was wondering what can be the advantage of setting the fps of the NvDrmRenderer to the correct value of my camera stream instead of any very high number.

Because as I understand the code the fps is only used to wait with frames, that come too early and to mark frames that come too late for the profiler.

But is there any advantage in waiting with frames? Since I want all frames to be displayed as soon as possible, I do not get why I would want that.

Best regards,
jb

Hi,
We have seen in certain cases, CPU usage is high without the waiting, so we put the waiting. You may remove it and run sudo tegrastats. If the loading is acceptable, you may remove it.

I will try that. But today I found out that NvDrmRenderer seems to be limited to 30fps. Even when I set drmrenderer and camerastream to 90 fps, if I measure performance my program has only a performance of 30fps, beause it gets stuck waiting for buffers from drm.

This is strange, since in the TRM it is written that the display manager can output 60Hz.

Do you have any idea why that is or could be?
I enqueue FULL-HD ,NvBufferColorFormat_YUV420,NvBufferLayout_Pitch buffers. Is this the right format I get from Argus Camera or is there some kind of transformation involved?

Also are there any recommendations how to lower the isp latency? I read a lot in the forum but those latency improvement patches were based on older versions? Are there any improvement patches for current versions?

Do you think I could get better latency performance when using more powerful jetsons like the XAVIER AGX?

Do you think bypassing the ISP and debayering myself on GPU could improve latency performance?

Best regards,
Jb

Hi,
Please check if display device is in 60fps mode:

root@tx2:~# cat /sys/class/graphics/fb0/mode

For ISP latency, is is around 100ms(3-4 frame delay). If you require lass latency, we suggest use other Jetson platforms(such as TX2 or Xavier) and implement de-bayering through CUDA to utilize GPU engine.

Thanks for your fast response.
I measure 50-70 glass to glass latency with ISP, can this be true then? If not I have to double check my tests. I was using a 240fps IPhone and count frames between flashlight turning on in real and on display.

I will check fhe fb and respond! :)

This is my output:
U:1920x1080p-60

Do you have any more recommendation what to try? Can you reproduce the issue maybe?

I could try DRM Renderer example and see if it’s happening there too!

Hi,
We can observe the issue. It is under investigation. Will update once there is further findings.

Do you already have information if the 30 fps limitation is also happening on Jetson XAVIER NX or Jetson AGX?
Is this behaviour on older Jetpack versions, too?

I can’t tell for your case, but in my case with NX R32.5.1, I can use up to 50 fps.

Hi,
Please apply this patch and give it a try:

diff --git a/multimedia_api/ll_samples/include/NvDrmRenderer.h b/multimedia_api/ll_samples/include/NvDrmRenderer.h
index 9084904..243c895 100644
--- a/multimedia_api/ll_samples/include/NvDrmRenderer.h
+++ b/multimedia_api/ll_samples/include/NvDrmRenderer.h
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-2017, NVIDIA CORPORATION.  All rights reserved.
+ * Copyright (c) 2016-2021, NVIDIA CORPORATION.  All rights reserved.
  * NVIDIA CORPORATION and its licensors retain all intellectual property
  * and proprietary rights in and to this software, related documentation
  * and any modifications thereto.  Any use, reproduction, disclosure or
@@ -265,6 +265,7 @@ private:
     int flippedFd;
     bool flipPending;
     bool renderingStarted;
+    bool planeIsSet;
 
     uint32_t hdrBlobId;
     bool hdrBlobCreated;
diff --git a/multimedia_api/ll_samples/samples/common/classes/NvDrmRenderer.cpp b/multimedia_api/ll_samples/samples/common/classes/NvDrmRenderer.cpp
index b9c68ac..52885ca 100644
--- a/multimedia_api/ll_samples/samples/common/classes/NvDrmRenderer.cpp
+++ b/multimedia_api/ll_samples/samples/common/classes/NvDrmRenderer.cpp
@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016-2020, NVIDIA CORPORATION.  All rights reserved.
+ * Copyright (c) 2016-2021, NVIDIA CORPORATION.  All rights reserved.
  * NVIDIA CORPORATION and its licensors retain all intellectual property
  * and proprietary rights in and to this software, related documentation
  * and any modifications thereto.  Any use, reproduction, disclosure or
@@ -189,6 +189,7 @@ NvDrmRenderer::NvDrmRenderer(const char *name, uint32_t w, uint32_t h,
   stop_thread = false;
   flipPending = false;
   renderingStarted = false;
+  planeIsSet = false;
   activeFd = flippedFd = -1;
   last_fb = 0;
   int ret =0;
@@ -720,10 +721,14 @@ NvDrmRenderer::renderInternal(int fd)
       goto error;
     }
 
-    ret = setPlane(0, fb, 0, 0, width, height, 0, 0, width, height);
-    if(ret) {
-      COMP_ERROR_MSG("FAILED TO SET PLANE ");
-      goto error;
+    if (planeIsSet == false)
+    {
+      ret = setPlane(0, fb, 0, 0, width, height, 0, 0, width, height);
+      if(ret) {
+        COMP_ERROR_MSG("FAILED TO SET PLANE ");
+        goto error;
+      }
+      planeIsSet = true;
     }
 
     /* TODO:
-- 
2.7.4

Hey thanks for your patch! I will try it next week. I have to finish another project till the end of this week!

Thanks so much!

Can the patch fix your issue? Any update?

Hey I just tried and it seems to work. 60fps now.

I will test now if latency is lower with 60 fps compared to 30fps.

Another question:
Should it be possible to use another plane than plane 0? For example hardcode it or use plane id as constructor parameter?

Thank you so much for the patch!

Hi,
There are code about UI plane in ui_render_loop_fcn(). Please refer to it and see if it can be applied to your use-case.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.