I have been struggling to find an accelerated gstreamer pipeline that works to write frames from a cv::Mat allocated with cudaMallocHost() to a file.
Currently we are using the following configuration, with some items omitted for brevity:
INPUT_FRAME_HEIGHT = 3040; INPUT_FRAME_WIDTH = 4032; INPUT_FPS = 30; cv::Mat largeFrame(INPUT_FRAME_HEIGHT,INPUT_FRAME_WIDTH,CV_8UC3,uni_frameLargeAddress); cv::cuda::GpuMat d_largeFrame(INPUT_FRAME_HEIGHT,INPUT_FRAME_WIDTH,CV_8UC3, uni_frameLargeAddress); captureString = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM),width="+std::to_string(INPUT_FRAME_WIDTH)+",height="+std::to_string(INPUT_FRAME_HEIGHT)+",framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink"; cap.open(captureString,cv::CAP_GSTREAMER); cap >> largeFrame; std::string gstreamer_pipeline = "appsrc ! videoconvert ! video/x-raw, width=" + std::to_string(largeFrame.cols) + ", height=" + std::to_string(largeFrame.rows) +", format=UYVY, framerate="+std::to_string(INPUT_FPS)+"/1 ! videoconvert ! omxh264enc insert-aud=true ! splitmuxsink muxer=mpegtsmux location=/mnt/sdcard/videos/temp/%09d.ts max-size-time=2000000000"; writer.open(gstreamer_pipeline, cv::CAP_GSTREAMER, -1, 30, cv::Size(largeFrame.cols, largeFrame.rows));
We want to do two things:
- replace the egress pipeline with one that can use hardware to do the encoding
- CPU is taking ~110ms to do the encoding. We would like to be in the sub 30ms range, with 2-4ms being ideal
- while the path for the image write says “sdcard,” it is actually on an nvme SSD,
- see if there is a faster way to load images from the camera into memory
- Right now the image load takes ~7ms, while our resize operation only takes ~900us.
The camera we are using is based on the imx477, and is connected via CSI
Any help pointing us in the right direction would be greatly appreciated