• Hardware Platform (Jetson / GPU) : Jetson
• DeepStream Version : DS6.2
• JetPack Version (valid for Jetson only) : JP5.1.2
• Issue Type( questions, new requirements, bugs) : Question
Hi,
I want to apply some cuda kenel based image processing that is transforming the input buffer and pushing to the downstream.
I’m using gst-nvdsvideotemplate
as a base source.
Q1. does this make sense in the performance perspective?
I want to use the color format NV12 so that there won’t be any colour conversion in the whole pipeline.
ex) nvarguscamerasrc (input, NV12) → nvdsvideotemplate(custom implementation, NV12) → nvinfer(if neeeded, NV12) → nvv4l2h264enc (NV12)
Q2. the main problem is that NV12 format seems not properly working.
how can I use NV12 nvbufsurface to opencv format?
I used CV_U8C1 with size of width and height *3/2.
// gst-nvdsvideotemplate/customlib_impl
// in the SampleAlgorithm::ProcessBuffer(GstBuffer *inbuf)
if (m_inVideoFmt == GST_VIDEO_FORMAT_RGBA)
{
// This is working fine, I can also use any custom cuda kernel
cv::cuda::GpuMat d_input_rgba = cv::cuda::GpuMat(in_surf->surfaceList[i].height, in_surf->surfaceList[i].width, CV_8UC4, (unsigned char *)in_surf->surfaceList[i].dataPtr);
d_input_rgba.setTo(cv::Scalar::all(0));
}
else if (m_inVideoFmt == GST_VIDEO_FORMAT_NV12)
{
// this does not wrap the data properly. only dark image.
cv::cuda::GpuMat d_input_nv12 = cv::cuda::GpuMat(in_surf->surfaceList[i].height *3 / 2, in_surf->surfaceList[i].width, CV_8UC1, (unsigned char *)in_surf->surfaceList[i].dataPtr);
// This gives error, I cannot use any other custom cuda kernel.
d_input_nv12.setTo(cv::Scalar::all(0));
}
the error message with NV12 is as below
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.5.4) /opt/nvidia/deepstream/deepstream-6.2/opencv-4.5.4/modules/core/src/cuda/gpu_mat.cu:389: error: (-217:Gpu API call) invalid argument in function 'setTo'
or, in case of custom kernel,
Caught SIGSEGV
Here is the gstreamer pipeline for your reference(only format=RGBA / NV12 difference)
RGBA → working fine
gst-launch-1.0 filesrc location= h264.mp4 ! qtdemux ! h264parse ! queue ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=3840 height=2160 batch-size=1 ! nvvideoconvert nvbuf-memory-type=1 compute-hw=1 ! 'video/x-raw(memory:NVMM), width=3840, height=2160' ! queue ! nvdsvideotemplate customlib-name="./customlib_impl/libcustom_videoimpl.so" customlib-props="scale-factor:2.0" ! nvvideoconvert ! 'video/x-raw(memory:NVMM), format=RGBA, width=3840, height=2160' ! nvv4l2h264enc ! ...
NV12 → error or seems there is no data in the ptr
gst-launch-1.0 filesrc location= h264.mp4 ! qtdemux ! h264parse ! queue ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=3840 height=2160 batch-size=1 ! nvvideoconvert nvbuf-memory-type=1 compute-hw=1 ! 'video/x-raw(memory:NVMM), width=3840, height=2160' ! queue ! nvdsvideotemplate customlib-name="./customlib_impl/libcustom_videoimpl.so" customlib-props="scale-factor:2.0" ! nvvideoconvert ! 'video/x-raw(memory:NVMM), format=NV12, width=3840, height=2160' ! nvv4l2h264enc ! ...