How can i camera display in lowest lantency?

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

my project use nvidia tx2 and Fpdlink camera ,display on drm method,but it cannot achieve my low lantency goal(80ms~150ms) ,besides this topic method:
First test: Fast copy of DMA buffers via NvBufferTransform

微信截图_3
微信截图_31356×124 10.2 KB

camera thread: uyvy 1920x1081
drm thread: direct display uyvy 1920x1081–>drm
cost about :120ms~220ms

Second test:
camera thread: uyvy 1920x1081 —>NvBufferTransform(will cost 2ms~3ms)->yuv420 1920x1080(for osd purpose)
drm thread: direct display yuv420 1920x1080–>drm(support uyvy,yuv420)
cost about :150ms~250ms

Third test:
camera thread: uyvy 1920x1081 —>NvBufferTransform(will cost 2ms~3ms)->abgr32 1920x1080(for osd purpose)
drm thread: abgr32 1920x1080 -->NvBufferTransform(will cost 1ms~2ms)—>yuv420 1920x1080–>drm(not support abgr32)
cost about :138ms~281ms

3

Please apply below patches to verify.

4

thank you for your reply,but my project use v4l2 and dmabuf
///////
if (-1 == ioctl(m_nDevFd, VIDIOC_QUERYCAP, &cap)) {
if (EINVAL == errno) {
plog(m_nDevIndex, “Given device is no V4L2 device”);
}
plog(m_nDevIndex, “Error occurred when querying capabilities”);
return ERR;
}

    if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
            plog(m_nDevIndex, "Given device is no video capture device");
            return ERR;
    }

//////
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = width;
fmt.fmt.pix.height = height;
fmt.fmt.pix.pixelformat = format;
fmt.fmt.pix.field = V4L2_FIELD_ANY;

    if (-1 == ioctl(m_nDevFd, VIDIOC_S_FMT, &fmt))
    {
            plog(m_nDevIndex, "Error occurred when trying to set format");
            return ERR;
    }

//////
int VideoCapture::init_dmabuf(void)
{
struct v4l2_requestbuffers req;
int ret = 0;

    prepare_dmabufs();

    CLEAR(req);

    memset(&req, 0, sizeof(req));
    req.count = NUM_BUFFS;
    req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    req.memory = V4L2_MEMORY_DMABUF;

    if (-1 == ioctl(m_nDevFd, VIDIOC_REQBUFS, &req)) {
            if (EINVAL == errno) {
                    plog(m_nDevIndex, "The device does not support dmabuf");
            }
            return ERR;
    }

    if (req.count < 1 || req.count != NUM_BUFFS) {
            plog(m_nDevIndex, "Insufficient memory to allocate buffers");
            return ERR;
    }

////
int VideoCapture::prepare_dmabufs()
{
int index;
NvBufferCreateParams input_params = {0};

input_params.payloadType = NvBufferPayload_SurfArray;
input_params.width = m_nWidth;//CAMERA_SRC_WIDTH;
input_params.height = m_nHeight;//CAMERA_SRC_HEIGHT;
input_params.layout = NvBufferLayout_Pitch;

// Create buffer and provide it with camera
for (index = 0; index < NUM_BUFFS; index++)
{
    int fd;
    NvBufferParams params = {0};

    input_params.colorFormat = NvBufferColorFormat_UYVY; 
    input_params.nvbuf_tag = NvBufferTag_CAMERA;
    if (-1 == NvBufferCreateEx(&fd, &input_params))
        plog(m_nDevIndex, "Failed to create NvBuffer");

    dmabuf_fd[index] = fd;

    if (-1 == NvBufferGetParams(fd, &params))
{
        plog(m_nDevIndex, "Failed to get NvBuffer parameters");
}
    plog(m_nDevIndex, "###VideoCapture[%d] dmabuf_fd[%d-%d]",m_nDevIndex,dmabuf_fd[index],index);

    // TODO add multi-planar support
    // Currently it supports only YUV422 interlaced single-planar
    if (-1 == NvBufferMemMap(fd, 0, NvBufferMem_Read_Write,(void**)&(fd_va[index])))
        plog(m_nDevIndex, "Failed to map buffer");
}

plog(m_nDevIndex, "Succeed in preparing stream buffers[%d]",m_nDevIndex);

return true;

}

5

Yes, those patches for VI driver to return the capture buffer to user space quickly to decrease the latency.

6

hello ShaneCCC
I find my program block on the drm dequeBuffer,the dequeBuffer function task 40ms~60ms,(test on tx2,jetson version{# R32 (release), REVISION: 5.0, GCID: 25531747, BOARD: t186ref, EABI: aarch64, DATE: Fri Jan 15 23:21:05 UTC 2021 ),but i test on my old jetson verson[# R32 (release), REVISION: 2.3, GCID: 17644089, BOARD: t186ref, EABI: aarch64, DATE: Tue Nov 5 21:48:17 UTC 2019],the same code,it does not appear,the dequeBuffer does not block,how can i find the reason for my drmrender lantency?
// NvBufferCreateEx create 3 NUB_BUFFS for drmRender
index=(renderCount++)%NUM_BUFFS;
pThis->m_pRenderer->enqueBuffer(pThis->buf_fd[index]);
if(renderCount>=NUM_BUFFS)
{
pThis->m_pRenderer->dequeBuffer();
}

7

微信截图1
微信截图1975×906 75 KB

微信截图2
微信截图21085×914 79.9 KB

8

video_dec_drm.cpp (35.6 KB)
this is the original code of the sample of 08_video_dec_drm,i only add time cost ,the new jetson version ,the dequeBuffer funcation cost 60ms ~70ms
dequeBuffer cost:73
Transform cost:5
dequeBuffer cost:61
Transform cost:5
dequeBuffer cost:61
Transform cost:5
dequeBuffer cost:62

微信截图_3
微信截图_31242×704 22.4 KB

10

Hi,
For running 08_video_dec_drm, please apply this patch and try again:
NvDRMRenderer fps setting - #14 by DaneLLL

And there is a waiting per fps setting. You may remove it:
NvDRMRenderer fps setting - #3 by DaneLLL

11

ok,thanks,i will apply this patch
"remove it "?remove what? i did not get

12

@DaneLLL ，i apply this patch,this is no dequeBuffer delay,what is the version of the path in the multimedia_api ? Can i download and use the new patch multimedia_api version?

13

@DaneLLL ,but i find a new problem,the shake problem,4 camera NvBufferComposite asynchronous
if i move,the shake problem appear,
the static camera pic,no shake problem

14

Hi,
The patch is included in next release. For using r32.5.1, please apply it.

And for clearness, please crate new topic for the new issue, and provide steps for reproducing it.

16

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.