Can I use the `NvSciBufObj imgBuf` retrieved from dwImageNvMedia as the input for IJPE?

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

My pipeline is like this:
camera → image rectification → JPEG Compression

For image rectification, I need dwImage_CUDA. Then I use dwImageStreamer to convert it to dwImage_NvMedia. It has NvSciBufObj imgBuf, which holds the pointer to the NvSciBufObj image. As the input of NvMediaIJPEFeedFrame is also NvSciBufObj bufObj, I think it should be possible to get the buffer pointer from dwImage_NvMedia and directly feed it to IJPE. But IJPE fails with NVMEDIA_STATUS_ERROR and a terminal output: Failed to get primitive surface types.

My sample code is:

/*********Init**************/
// Initialize NvMedia structures
    err = NvSciBufModuleOpen(&bufModule);
    err = NvSciSyncModuleOpen(&syncModule);
    err = NvSciSyncCpuWaitContextAlloc(syncModule, &cpuWaitContext);

    NvSciBufAttrListCreate(bufModule, &bufAttributeList);
    NvMediaIJPEFillNvSciBufAttrList(NVMEDIA_JPEG_INSTANCE_0, bufAttributeList);

    status = PopulateNvSciBufAttrList(
            chroma,
            m_width,
            m_height,
            true,                           /* needCpuAccess */
            NvSciBufImage_PitchLinearType,
            NUM_SURFACES,                             /* plane count */
            NvSciBufAccessPerm_ReadWrite, /* access permission */
            256U,                        /* doesn't matter, it specifies the alignment of the starting address of each plane in memory */
            NvSciColorStd_REC709_ER,
            NvSciBufScan_ProgressiveType,  /* default scanType */
            bufAttributeList);

    if (NVMEDIA_STATUS_OK != status) {
        LOG_ERR("NVDrive: Failed to populate attributes: %d\n", status);
    }

    NvSciBufAttrList bufConflictList;
    err = NvSciBufAttrListReconcile(&bufAttributeList, 1U,
            &bufReconciledList, &bufConflictList);
    if (err != NvSciError_Success) {
        LOG_ERR("NVDrive: Reconciliation for input frame failed: %d\n", err);
    }

    appBuffer = (NvMediaAppBuffer *)(malloc(sizeof(NvMediaAppBuffer)));
    memset(appBuffer, 0x0, sizeof(NvMediaAppBuffer));

    dwImage_getNvMedia(&dw_nvmedia_image, m_nvmedia_image);

    // err = NvSciBufObjAlloc(bufReconciledList, &appBuffer->bufObj);
    err = NvSciBufObjAlloc(bufReconciledList, &dw_nvmedia_image->imgBuf);
    if (err != NvSciError_Success) {
        LOG_ERR("NVDrive: NvSciBufObjAlloc failed: %d\n", err);
    }

    ijpeCtx = NvMediaIJPECreate(bufReconciledList,               // inputFormat
                                n_inputs,                        // maxOutputBuffering
                                MAX_BITSTREAM_SIZE,              // maxBitstreamBytes
                                NVMEDIA_JPEG_INSTANCE_0);        // HW instance id

    if(!ijpeCtx) {
        LOG_ERR("NVDrive: NvMediaIJPECreate failed\n");
    }

    /* The reconciled list is needed for later */
    NvSciBufAttrListFree(bufAttributeList);
    NvSciBufAttrListFree(bufConflictList);

    sAllocEOFNvSciSyncObj(ijpeCtx, syncModule, &eofSyncObj);
    sAllocPreNvSciSyncObj(ijpeCtx, syncModule, &preSyncObj);
    NvMediaIJPERegisterNvSciSyncObj(ijpeCtx, NVMEDIA_EOFSYNCOBJ, eofSyncObj);
    NvMediaIJPERegisterNvSciSyncObj(ijpeCtx, NVMEDIA_PRESYNCOBJ, preSyncObj);
    NvMediaIJPESetNvSciSyncObjforEOF(ijpeCtx, eofSyncObj); // Runtime function, but only needed to be called once
    // NvMediaIJPERegisterNvSciBufObj(ijpeCtx, appBuffer->bufObj);
    NvMediaIJPERegisterNvSciBufObj(ijpeCtx, dw_nvmedia_image->imgBuf);

/*********Runtime**************/
// stream CUDA image to NvMedia image ...

// feed nvmedia frame to ijpe
dwImage_getNvMedia(&dw_nvmedia_image, m_nvmedia_image);
    status = NvMediaIJPEInsertPreNvSciSyncFence(ijpeCtx, &appBuffer->preFence);
    err = NvSciSyncObjSignal(preSyncObj);

    GetTimeMicroSec(&startTime);
    LOG_DBG("nvmedia: Encoding frame #%d\n", frameCounter);

    status = NvMediaIJPEFeedFrame(ijpeCtx,
                                //   appBuffer->bufObj,
                                imgbuf,   
                                  quality,
                                  NVMEDIA_JPEG_INSTANCE_0);

    if (status != NVMEDIA_STATUS_OK)
    {
        std::cerr << "NvMedia: NvMediaIJPEFeedFrame failed: " << status << std::endl;
    }

Now my workaround is, I use NvSciBufObjGetPixels() to read the buffer of dwImage_NvMedia from GPU to CPU, and use NvSciBufObjPutPixels() to put it back to another empty buffer in GPU. Then I could run IJPE without error, but I feel it is not efficient.

Could I avoid putting image buffer back and forth between GPU and CPU?

Thanks.

Dear @yuchen.liu1,

This seems to indicate some of the scibuf attributes are missing

Can you share the working code with NvSciBufObjPut(Get)Pixels to reproduce the issue?

This is the sample code from my side.
compression_sample.txt (8.2 KB)

After I get a nvmedia image, I need to write a function buffer_prepare() to get the buffer from GPU → CPU → GPU to make ijpe work.

My question is, can I replace buffer_prepare() with some builtin function that could directly feed the nvmedia image buffer into IJPE, without having to go round the CPU?

Thanks.

The reason for failure is when nvscibuf is shared across different engine, attribute list for each engine needs to be created and reconciled. It guarantees the buffer is allocated satisfying all the requirements of engines.

got it. So how to create and reconcile two attribute lists for ijpe and dwImageNvMedia?

To put it more specifically, if I run these lines to get nvmedia images as the input of ijpe:

dwImageStreamer_producerSend(dw_cuda_image, m_streamerCUDA2NvMedia);
dwImageStreamer_consumerReceive(&m_nvmedia_image, 1000, m_streamerCUDA2NvMedia);
dwImage_getNvMedia(&dw_nvmedia_image, m_nvmedia_image);

NvMediaIJPEInsertPreNvSciSyncFence(ijpeCtx, &appBuffer->preFence);
NvSciSyncObjSignal(preSyncObj);
NvMediaIJPEFeedFrame(ijpeCtx,
                                  dw_nvmedia_image->imgBuf,
                                  quality,
                                  NVMEDIA_JPEG_INSTANCE_0);

How should I allocate attributes to the buffer of dw_nvmedia_image and feed it to IJPE?