Color correction using dsexample

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

**• Hardware Platform (Jetson / GPU)**dGPU
• DeepStream Version 5.1
• JetPack Version (valid for Jetson only)
• TensorRT Version The latest version that came with Deepstream 5.1
• NVIDIA GPU Driver Version (valid for GPU only) 460.39
• Issue Type( questions, new requirements, bugs) Question
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Hello, My use case is to perform auto color correction through a custom element(the function SimplestCB performs this), colorbalance in an image using c++ and opencv - Stack Overflow.

Looking at Not able to draw line on incoming frame, *dsexample->cvmat, is the cv::Mat object of the frame, which I would need to modify.

I call the SimplestCB function, right after the transcoding of RGBA to BGR conversion takes place inside, get_converted_mat

#if (CV_MAJOR_VERSION >= 4)
  cv::cvtColor(in_mat, *dsexample->cvmat, cv::COLOR_RGBA2BGR);
#else
  cv::cvtColor(in_mat, *dsexample->cvmat, CV_RGBA2BGR);
  SimplestCB(*dsexample->cvmat, tmp, 1);
  NvBufSurfaceSyncForDevice(dsexample->inter_buf, 0, 0);
  cv::cvtColor(*dsexample->cvmat, in_mat, cv::COLOR_BGR2RGBA);
  NvBufSurfaceMap(dsexample->inter_buf, 0, 0, NVBUF_MAP_READ_WRITE);
  *dsexample->cvmat = tmp.clone();

Here’s the gst-launch command.
gst-launch-1.0 uridecodebin uri='file:///opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_1080p_h264.mp4' ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-5.1/sources/deepstream-python-pipeline-structuring/dstest_imagedata_config.ini ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=RGBA" ! dsexample full-frame=1 ! nvvideoconvert ! "video/x-raw(memory:NVMM), format=I420" ! nvv4l2h264enc bitrate=2000000 ! h264parse ! flvmux latency=200 ! rtmpsink location='rtmp://13.xxx.xxx.xx/streamkey'

The issue is that I’m not able to get the color-corrected frame.

Here, is the normal image before color correction named(Normal-image.png)

Normal-image
Normal-image1920×1080 337 KB

Here’s the color-corrected image, i.e what the frame should look like if processed correctly.

colorcorrection-1percent
colorcorrection-1percent1920×1080 380 KB

I have tried adding NvBuf map and unmap, but that gave internal data stream errors, hence I decided to exclude it from the code I’ve posted here.

Thank you in advance!

3

gst-dsexample works in “in-place” mode( GstBaseTransform (gstreamer.freedesktop.org)).

And if you have read the code carefully, you will find the “dsexample->inter_buf” is just an internal buffer to get the surface data be transferred to “dsexample->cvmat” and finally it will be save as an image. There is no path to pass the “dsexample->inter_buf” to the src pad.

Please make sure you are familiar with Gstreamer plugin and application development and coding skills before you customize new feature with deepstream APIs. https://gstreamer.freedesktop.org/

If you want to do such change, you may refer to Deepstream sample code snippet - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

5

Thank you for your reply, and guidance @Fiona.Chen.

Following, the link in my original post and Performing OpenCV operations on pipeline buffer, I was able to carry out the color balance task by modifying the get_converted_mat function itself and send the processed buffers downstream