Frame Saved With Deepstream Corrupted On Jetson Orin But Not On dGPU

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, , , , ,
1

I saw the sample app deepstream_image_meta_test, but I didn’t succeed to extract image with it, I don’t want to save processed image or cropped image, only raw frame

Frame Saved Corrupted On Jetson Orin But Not On dGPU

• Jetson Orin Nano
• Deepstream 7.0
• JetPack 6.0
• 8.6.2.3-1+cuda12.2
• bugs
• Adding this code snippet in a callback probe on a usb camera sample
• c++ 17; openCv

I’m building a Deepstream app on c++ where I want to save a frame (without any cropping) in JPG format every 10 second , My app is available on x86 dGPU and jetson Orin.

When I use my application on dGPU, the image is correct:

snap-20240816124612476
snap-202408161246124761920×1080 314 KB

But when I use it on my Orin I get discontigous image :

snap-20240816124714010
snap-202408161247140101920×1080 224 KB

snap-20240816124703990
snap-202408161247039901920×1080 217 KB

Here the code I use inside the loop of my Callback Probe Function :

#include <string>
#include <sstream>
#include <unordered_map>
#include <utility>       // For std::pair
#include <cstdlib>       // For exit
#include <ctime>         // For localtime
#include <iomanip>       // For std::setw and std::setfill
#include <chrono>        // For time calculations
#include <cstring>
#include <iostream>
#include <filesystem>
#include <system_error>

// System headers
#include <sys/file.h>    // For flock
#include <sys/stat.h>    // For stat
#include <fcntl.h>       // For open
#include <unistd.h>      // For close

// CUDA headers
#include <cuda_runtime_api.h>

// GStreamer and DeepStream headers
#include <gst/gst.h>
#include <gst/video/video.h>
#include "gstnvdsmeta.h"
#include "nvdsmeta_schema.h"
#include "nvds_roi_meta.h"
#include <nvds_obj_encode.h>

// NVIDIA headers
#include <nvbufsurface.h>
#include <nvbufsurftransform.h>

// OpenCV headers
#include <opencv2/opencv.hpp>


/**
 * @brief Converts NvBufSurface to cv::Mat using CUDA.
 *
 * @param surface Pointer to the NvBufSurface.
 * @param batch_id Batch ID.
 * @return A cv::Mat object containing the image data.
 */


cv::Mat NvBufSurfaceToCvMat(NvBufSurface *surface, int batch_id) {
    //surface->memType = NVBUF_MEM_DEFAULT;  // Ensure the memory type is set to surface array no
    //surface memtype is array
    // Map the surface for reading
    if (NvBufSurfaceMap(surface, -1, -1,  NVBUF_MAP_READ_WRITE) != 0) {
        std::cerr << "Failed to map NvBufSurface" << std::endl;
        return cv::Mat();  // Return an empty cv::Mat on failure
    }

    // Sync the surface memory for CPU access
    NvBufSurfaceSyncForDevice(surface, batch_id, 0);
    NvBufSurfaceParams params = surface->surfaceList[0];  // Get the surface parameters for the specified batch ID
    cv::Mat img(params.height, params.width, CV_8UC4, params.dataPtr, params.pitch);

    

    // Unmap the surface after reading
    NvBufSurfaceUnMap(surface, 0, 0);

    return img;  // Return the created cv::Mat
}

/**
 * @brief Saves a frame image to a file.
 *
 * @param buf Pointer to the GstBuffer.
 * @param logDir Directory where the image file will be saved.
 * @param filename The filename to save the image.
 */
void saveFrameImage(GstBuffer* buf, const std::string& logDir, const std::string& filename) {
    NvBufSurface* surface = nullptr;
    GstMapInfo map;

    if (gst_buffer_map(buf, &map, GST_MAP_READ)) {
        surface = (NvBufSurface*)map.data;
       // gst_buffer_unmap(buf, &map);
    } else {
        g_printerr("Unable to map GstBuffer to NvBufSurface\n");
        return;
    }

    int batch_id = 0;
    cv::Mat frame = NvBufSurfaceToCvMat(surface, batch_id);
    //display the image
    struct stat sb;
    if(stat(logDir.c_str(), &sb) != 0) {
        system(("mkdir -p " + logDir).c_str());
    }

    if (!cv::imwrite(logDir + "/" + filename, frame)) {
        g_printerr("Failed to save frame to file\n");
    } else {
        g_print("Frame saved to %s\n", filename.c_str());
    }
}

Here the beginig of pipeline:

pipeline:
  - v4l2src:
      device: /dev/video0
  - capsfilter:
      caps: "image/jpeg, width=1920, height=1080, framerate=30/1"
  - jpegdec: {}
  - videoconvert: {}
  - nvvideoconvert: {}
  - capsfilter:
      caps: "video/x-raw(memory:NVMM), format=RGBA, width=1920, height=1080"
  - mux.sink_0:
       nvstreammux:
          name: mux
          batch-size: 1
          width: 1920
          height: 1080
          batched-push-timeout: 4000000
          live-source: 1
          num-surfaces-per-frame: 1
          sync-inputs: 0
          max-latency: 0

Thanks for any help !

3

Could you just display the camera source directly before the nvstreammux plugin and see if there’s still the same problem?
Like

pipeline:
  - v4l2src:
      device: /dev/video0
  - capsfilter:
      caps: "image/jpeg, width=1920, height=1080, framerate=30/1"
  - jpegdec: {}
  - videoconvert: {}
  - nvvideoconvert: {}
  - nvds3dsink
4

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks