Currently, I’m working on display the camera frame using DRM renderer. I tried to tweak the sample 12_v4l2_camera_cuda, but I have several issues:
Orthogonal stairway artifacts cross the image. I found probably the cause could be the non-matching FPS between incoming frame and the rendering process. Initially, I simply replaced the *NvEglRenderer renderer; to *NvDrmRenderer drm_renderer;, and manage the buffers in the same way as in 08_video_dec_drm, for example, enqueue the 4 buffers to drm_renderer in the first 4 frames, then dequeue → enqueue for the rest frames. I found the render_fd returned by drm_renderer->dequeBuffer(); is always the same… So I tried to manage the “dequeued” fd manually, then problem solved. Could you pls suggest if this is a right way?
Memory leak during rendering. While rendering, the memory leaking is huge, the RAM consumption keeps on increasing. I tried to comment the dequeue-enqueue lines within the capture loop, the memory-leak has gone, so I guess the leaking is coming from the NvDrmRender module. Is there any way to solve this?
My setting ups:
USB Camera;
V4L2 API interface;
Camera streaming: UYVU 1920x1080 @60fps;
V4L2_MEMORY_DMABUF.
Hi,
Please try latest Jetpack 5.1.1 and check if the issue is still present. If it still exists, please share a patch to 12_camera_vl42_cuda sample so that we can set up and give it a try. By default NvEglRenderer is used in 12_camera_vl42_cuda sample.
Thanks for reply! I forgot to mention that I’m using the emulated NX Orin 8GB from AGX Orin 64GB. The Jetpack version from the emulated platform is shown as “unknown”, but the L4T is v35.3.1, probably it’s already JetPack 5.1.1. Please find my script in the attachment. camera_v4l2_cuda.h (3.4 KB) camera_v4l2_cuda.cpp (26.9 KB)
Thanks for suggestions! Yes, in the beginning, I followed what you suggested, but I found the render_fd dequeued from drm_renderer is always the same. The memory leaking issue remains. Apart from that, I found there is a staircase-like artefact. I uploaded it in the attachment. It’s probably caused by the “render_fd” i guess. Because the render fd returned from drm renderer dequeue is always the same, when it’s still being rendered, the new frame came in advance, which breaks the frame into two sides separated by the staircase (old one vs new one). It became severe when I was fast moving the camera.
If it is something wrong in our low-level software stack, the issue should be seen in above methods. Would suggest try these for comparison and clarification.
I have tried 08_video_dec_drm, and the video playing without any memory leaking issue, but I have debugged the following issue: the ctx->drm_renderer->dequeBuffer() still returns the same fd, so the internal DRM queue seems not really dequeued?
For the gst-launch-1.0, I have tested, but some errors were prompted. Sorry I’m quit new to Jetson, probably some setting args were not correct?
nx8onagx64@nx8onagx64:~$ gst-launch-1.0 v4l2src ! nvvidconv ! ‘video/x-raw(memory:NVMM),format=I420’ ! nvdrmvideosink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
cannot create framebuffer
Cannot create frame buffer
cannot create framebuffer
Cannot create frame buffer
cannot create framebuffer
Cannot create frame buffer
cannot create framebuffer
Cannot create frame buffer
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3072): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:01.416710817
Setting pipeline to NULL …
cannot create framebuffer
Cannot create frame buffer
Freeing pipeline …
And additionally, the purpose that I’m dong the current tasks is that I want to revise the 12_v4l2_camera_cuda with DRM renderer. Would you be able to suggest a correct workflow to stream camera frame and display with DRM renderer pls? Appreciate it!
Hi,
Since the issue is in application layer, and it is open source, it would be great if you can do further debugging. It’s supposed to work the same as 08 sample.
if (ctx->capture_dmabuf) {
if (-1 == NvBufSurfaceMap (pSurf, 0, 0, NVBUF_MAP_READ_WRITE))
ERROR_RETURN("Failed to map buffer");
ctx->g_buff[index].start = (unsigned char *)pSurf->surfaceList[0].mappedAddr.addr[0];
ctx->g_buff[index].size = pSurf->surfaceList[0].dataSize;
}
to
if (ctx->capture_dmabuf) {
/* Cache sync for VIC operation since the data is from CPU */
if (-1 == NvBufSurfaceSyncForDevice (pSurf, 0, 0))
ERROR_RETURN("Cannot sync output buffer");
}
It is a leak(unnecessary map and not unmapped) in 12 sample and fixed in Jetpack 5.1.2. Not an issue in NvDrmRenderer.
Thanks for the follow up! I tried to change those lines to remove the unnecessary maps, but I got an error from that line.
NvMapMemCacheMaint Bad parameter
nvbusurface: NvBufSurfaceSyncForDevice: Error(4) in sync
Right now, I’m unable to flash the platform to 5.1.2 immediately, could you pls help attach the latest 12-sample source code here?
An additional observation is that when I commented the “enqueue-dequeue” pair, the mem leak has gone. Does it still mean there’s no issue in NvDrmRenderer?
Hi Dane, forgive me, may I cut in to ask another quick question?
My colleague is working on drawing the GUI using DRM. The platform is AGX Orin 32GB, he found a post saying that it’s unable to enable the multiplanar GUI in Orin platform. We are gonna use the primary plane to display the camera frames, and overlay plan to display some UI widgets. Is it feasible in JetPack 5.1.2?
I have flashed the device to Jetson Linux 35.4.1 with Jetpack 5.1.2, but the problem still exists. I checked the sample 12 source code, i found it’s the same with the previous one. May I check with you whether the fix you mentioned had been updated to Jetpack 5.1.2 or it’s one an internal patch? Thanks!
