Decoder_to_fd performance

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

Can NvTransform be eliminated from JPEG decode pipeline?

Environment

  • Platform: Jetson AGX orin
  • Camera: MJPEG, 3840x1080, target ≥60fps
  • Decoder: NvJPEGDecoder::decodeToFd()

Current Pipeline

MJPEG frame → decodeToFd() → NvTransform() → EGL Map → NPP YUV420ToRGB → D2H

Performance Breakdown (per frame)

┌───────────────────┬───────────┬────────────────────────────────────────────────────────────────────────────────┐
│ Step │ Time (ms) │ Notes │
├───────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────┤
│ decodeToFd │ 6.2 │ NVDEC hardware decode, outputs to NVBUF_MEM_DEFAULT │
├───────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────┤
│ NvTransform │ 6.4~9.0 │ Same resolution, same format copy: NVBUF_MEM_DEFAULT → NVBUF_MEM_SURFACE_ARRAY │
├───────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────┤
│ YUV420ToRGB (NPP) │ 1.6 │ │
├───────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────┤
│ D2H memcpy │ 1.7 │ │
├───────────────────┼───────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Total │ ~16 │ │
└───────────────────┴───────────┴────────────────────────────────────────────────────────────────────────────────┘

NvTransform accounts for 40%~50% of the total grab time. It performs no format conversion or scaling — it is purely a memory copy from NVBUF_MEM_DEFAULT to NVBUF_MEM_SURFACE_ARRAY so that CUDA can
access the decoded data via EGL mapping. This copy is the single largest bottleneck preventing us from reaching ≥60fps.

Failed Attempts to Remove NvTransform

Attempt 1: Direct EGL mapping on surf_decoder_ (NVBUF_MEM_DEFAULT)

NvBufSurfaceMapEglImage(surf_decoder_, 0);
cuGraphicsEGLRegisterImage(&cudaRes, eglImage, …);
cuGraphicsResourceGetMappedEglFrame(&eglFrame, cudaRes, 0, 0);

// After decodeToFd()
pSrc[0] = (Npp8u *)eglFrame.frame.pPitch[0]; // Y plane — all zeros

Result: CUDA sees all zeros in the mapped memory. NVDEC output is not visible to CUDA via EGL on NVBUF_MEM_DEFAULT buffers, even with NvBufSurfaceSyncForDevice() and
cuGraphicsMapResources/UnmapResources.

Attempt 2: Change surf_decoder_ to NVBUF_MEM_SURFACE_ARRAY

create_params_.memType = NVBUF_MEM_SURFACE_ARRAY; // was NVBUF_MEM_DEFAULT
NvBufSurfaceCreate(&surf_decoder_, 1, &create_params_);

Result: decodeToFd returns success but subsequent access produces:
nvbufsurface: Wrong buffer index (0)
decodeToFd does not support output to NVBUF_MEM_SURFACE_ARRAY buffers.

Questions

  1. Can decodeToFd output directly to NVBUF_MEM_SURFACE_ARRAY? If so, what configuration is needed?
  2. Is there any API or mechanism for CUDA to directly access NVDEC output in NVBUF_MEM_DEFAULT without the NvTransform copy?
  3. Does NvJPEGDecoder provide an alternative decode interface (e.g., decodeToBuffer) that outputs to CUDA-accessible memory?
  4. Is zero-copy access from CUDA to NVDEC JPEG decode output possible on Jetson? If so, what is the recommended API and workflow?

Goal

Eliminate the NvTransform copy to reduce grab time from ~16ms to ~10ms, enabling ≥60fps for MJPEG cameras.

bool V4l2Cam::decoder_to_fd()
{
int res = ctx_decoder_.jpegdec->decodeToFd(fd_decoder_, ctx_decoder_.in_buffer, ctx_decoder_.in_file_size,
pixfmt_decoder_, dev_width, dev_height);
if (res != 0 || fd_decoder_ < 0) {
LOG(ERROR) << “decodeToFd failed, res=” << res << “, fd_decoder_=” << fd_decoder_;
return false;
}

{
    res = NvBufSurf::NvTransform(&transform_params_, fd_decoder_, dst_dma_fds_[0]);
    if (res != 0 || dst_dma_fds_[0] < 0) {
        LOG(ERROR) << "NvTransform, res=" << res << std::endl;
        return false;
    }
}
const Npp8u *pSrc[3];
int aSrcStep[3];
pSrc[0] = (Npp8u *)eglFrame_[0].frame.pPitch[0];
aSrcStep[0] = (int)eglFrame_[0].pitch;
pSrc[1] = (Npp8u *)eglFrame_[0].frame.pPitch[1];
pSrc[2] = (Npp8u *)eglFrame_[0].frame.pPitch[2];
aSrcStep[1] = u_pitch_;
aSrcStep[2] = v_pitch_;
NppiSize oSize = {(int)dev_width, (int)dev_height};
Npp8u *pDst = d_rgb_image_;
nppiYUV420ToRGB_8u_P3C3R_Ctx(pSrc, aSrcStep, pDst, dev_width * 3, oSize, npp_rgb_context_);
cudaStreamSynchronize(rgb_stream_);

return true;

}

3

Thanks for the detailed breakdown and the two failed attempts—that’s very helpful context. To help narrow down the zero-copy options, a few follow-up questions:

  1. NvJPEGDecoder API modes: Have you reviewed the NvJPEGDecoder API documentation for alternative output buffer types or decode modes that might support direct CUDA-accessible output? Specifically, does the decoder support output modes other than NVBUF_MEM_DEFAULT?

  2. NVBUF_MEM_SURFACE_ARRAY error details: When you tried changing to NVBUF_MEM_SURFACE_ARRAY, can you share your exact NvBufSurfaceCreate parameters and the full error context around the “Wrong buffer index” message? This will help determine if it’s a configuration issue or a fundamental API limitation.

  3. CUDA unified memory: Have you explored using CUDA unified memory (cudaMallocManaged) or other memory allocation strategies for the decode output buffer? This might bypass the NvBufSurface memory type constraints.

  4. Jetson multimedia alternatives: Does the Jetson multimedia stack provide any zero-copy or DMA-based alternatives to NvTransform for buffer format conversion (e.g., VIC hardware, alternative decode interfaces)?

Once we understand which of these paths are viable on your platform, we can identify the best approach to eliminate the NvTransform copy.

6

Hi,
Please check the sample in Jetpack 6.2.2 r36.5:

/usr/src/samples/06_jpeg_decode/

It supports dumping decoded NvBufSurface directly. So if your JPEG is decoded to YUYV( or UYVY), you can implement YUYV to RGB conversion to eliminate the NvBufSurfaceTransform()

Or may try to keep current implementation and enable VIC at maximum throughput by running the script:
VPI - Vision Programming Interface: Performance Benchmark

See if it achieves the performance requirement.