Issue with NvJPEGDecoder::decodeToFd in Jetson Multimedia API, used in ROS2 Foxy on Jetson Xavier AGX Running JP 5.1.2


I am encountering an issue while using ROS2 Foxy on my Jetson Xavier AGX, which is running JetPack 5.1.2. The problem arises with an image subscriber node where the callback receives a JPEG byte array that needs to be decoded using the NvJPEGDecoder::decodeToFd function from the Jetson Multimedia API.


The decodeToFd function only works correctly for the first callback invocation. For the rest of the execution time, it shows the same output image despite the input data changing.

Debugging Steps Taken:

1.Data Validation: I confirmed that the data coming into the callback is changing by saving the incoming JPEG byte arrays as files. These files show updated images. (At the start of the callback)
2. Issue Persistence: When saving the decoded image after the decodeToFd and format conversion function call, the output image does not update and remains the same as the first decoded image.
3. Resetting the Decoder: I tried resetting the decoder in each callback invocation. This approach leads to the output images changing as expected.

Here is the callback function:
callback.txt (3.8 KB)

void topic_callback(const std_msgs::msg::UInt8MultiArray::SharedPtr msg){

      static int dst_dma_fd = -1;
      int fd = 0;
      int mid_fd = 0;
      unsigned long in_buf_size = msg->data.size();
      unsigned char *in_buf = new unsigned char[in_buf_size];
      memcpy(in_buf, msg->, in_buf_size);

      /* Data changes on the saved image here*/
      std::ofstream file;"/mnt/nvme_drive/swift_sense_ws/input_image.jpg", std::ios::out | std::ios::trunc);
      file.write((char*)in_buf, in_buf_size);

      /* Reset Decoder here, output is as expeceted*/

      int ret = ctx.jpegdec->decodeToFd(fd, in_buf, in_buf_size, pixfmt, width, height);
      if(ret != 0) {
        std::cerr << "Error decoding image" << std::endl;
        // goto cleanup;

      NvBufSurf::NvCommonAllocateParams params;
      params.memType = NVBUF_MEM_SURFACE_ARRAY;
      params.width = 1920;
      params.height = 1536;
      params.layout = NVBUF_LAYOUT_PITCH;
      // params.colorFormat = NVBUF_COLOR_FORMAT_YUV420;
      // params.colorFormat = NVBUF_COLOR_FORMAT_YUYV;
      params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
      params.memtag = NvBufSurfaceTag_VIDEO_CONVERT;

      ret = NvBufSurf::NvAllocate(&params, 1, &dst_dma_fd);
      if(ret != 0) {
        std::cerr << "Error allocating buffer" << std::endl;
        // goto cleanup;

      NvBufSurf::NvCommonTransformParams transform_params;
      transform_params.src_top = 0;
      transform_params.src_left = 0;
      transform_params.src_width = width;
      transform_params.src_height = height;
      transform_params.dst_top = 0;
      transform_params.dst_left = 0;
      transform_params.dst_width = width;
      transform_params.dst_height = height;
      transform_params.flag = NVBUFSURF_TRANSFORM_FILTER;
      transform_params.flip = NvBufSurfTransform_None;
      transform_params.filter = NvBufSurfTransformInter_Nearest;

      ret = NvBufSurf::NvTransform(&transform_params, fd, dst_dma_fd);
      if(ret != 0) {
        std::cerr << "Error transforming image" << std::endl;
        // goto cleanup;

      NvBufSurface *nvbuf_surf = nullptr;
      ret = NvBufSurfaceFromFd(dst_dma_fd, (void**)(&nvbuf_surf));
      if (ret != 0) {
          std::cerr << "NvBufSurfaceFromFd failed with error: " << ret << std::endl;
          // goto cleanup;

      ret = NvBufSurfaceMap(nvbuf_surf, 0, 0, NVBUF_MAP_READ_WRITE);
      if (ret < 0) {
          std::cerr << "NvBufSurfaceMap failed with error: " << ret << std::endl;
          // goto cleanup;

      NvBufSurfaceSyncForCpu(nvbuf_surf, 0, 0);

      unsigned char *data = (unsigned char *)nvbuf_surf->surfaceList->mappedAddr.addr[0];

      cv::Mat rgb_image(height, width, CV_8UC4, data);

      cv::cuda::GpuMat rgb_image_D;
      rgb_image_D.upload(rgb_image, stream);
      cv::cuda::cvtColor(rgb_image_D, resized, cv::COLOR_RGBA2BGR, 0, stream);
      cv::Mat host_frame;, stream);

      /* Output does not get updated */
      cv::imwrite("/mnt/nvme_drive/swift_sense_ws/output_image.png", rgb_image);

      ret = NvBufSurf::NvDestroy(dst_dma_fd);
      if (ret != 0) {
          std::cerr << "NvBufSurfaceDestroy failed with error: " << ret << std::endl;
          // goto cleanup;

      NvBufSurfaceUnMap(nvbuf_surf, 0, 0);
      delete[] in_buf;
    rclcpp::Subscription<std_msgs::msg::UInt8MultiArray>::SharedPtr subscription_;

The following is the resetDecoder function i added,

void NvJPEGDecoder::reset() {
    jpeg_destroy_decompress(&cinfo); // Destroy the decompression object
    jpeg_create_decompress(&cinfo); // Reinitialize the decompression object

Could you please help me understand why decodeToFd is not updating the output image after the first callback and how to resolve this issue? Any insights or suggestions would be greatly appreciated.

Thank you!

This issue is known. Please use Jetpack 5.1.3 + the lib:

Jetson/L4T/r35.5.x patches -
[MMAPI/gstramer] NvJpegDecoder/nvjpegdec does not work properly

Hey thank you for pointing this out. Unfortunately, I am restricted to JP5.1.2. Are there any other fixes I can try?

If you have to use Jetpack 5.1.2, would suggest use software JPEG decoder.

I tried using the decodeToBuffer function, same behavior is observed. Did you mean that I should not use Nv JPEG for decoding?


Yes. This is expected since both decodeToBuffer() and decodeToFd() uses hardware JPEG decoder. Please use software decoder like OpenCV or libjpeg.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.