How to perform ColorBalance?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) dGPU
• DeepStream Version 5.1
• TensorRT Version The latest version that came with Deepstream 5.1
• NVIDIA GPU Driver Version (valid for GPU only) 460.39
• Issue Type( questions, new requirements, bugs) Question

I want to perform color balancing(hue shifting for my use case) on a video, Looking at GStreamer docs there’s supposed to be a “colorbalance” element
(GstColorBalance)

Example elements are ‘xvimagesink’ and ‘colorbalance’

However, upon executing gst-inspect-1.0 colorbalance, it says "
No such element or plugin 'colorbalance' ".

Tried to install GStreamer base plugins again, but that didn’t help (I’m using deepstream:5.1-21.02-devel container).

I don’t have direct access to the camera source, so I won’t be able to tune settings on that to account for color balancing. I’d have to perform all operations on the pipeline itself, before being processed by nvinfer.

I’d be using uridecodebin primarily, any suggestions on how I can go about it?
Thank you!

3

Seems you should install the related libs per Video Library , I don’t think it’s a Deepstream issue/question.

1 Like
4

Well, when I try to play the file, I get an Nvidia decoder error. I tried testing the file with sample apps as well, and I got the same error. I’m able to play and run the mp4 file though.
[D 210514 04:25:52 gst_utils:254] Creating source_bin 0 for: file:////opt/nvidia/deepstream/deepstream-5.1/sources/deepstream-python-pipeline-structuring/cam4.mp4
[I 210514 04:25:52 GstPipeline:357] Adding elements to Pipeline
[I 210514 04:25:52 GstPipeline:391] Linking elements in the Pipeline
[I 210514 04:25:52 GstPipeline:460] Finished building pipeline
[I 210514 04:25:52 GstPipeline:478] Starting pipeline
[D 210514 04:25:52 gst_utils:476] Decodebin child added:source
[D 210514 04:25:52 gst_utils:476] Decodebin child added:decodebin0
[D 210514 04:25:52 gst_utils:476] Decodebin child added:qtdemux0
[D 210514 04:25:52 gst_utils:476] Decodebin child added:multiqueue0
[D 210514 04:25:52 gst_utils:476] Decodebin child added:h265parse0
[D 210514 04:25:52 gst_utils:476] Decodebin child added:capsfilter4
[D 210514 04:25:52 gst_utils:476] Decodebin child added:avdec_h265-0
[E 210514 04:25:52 gst_utils:463] Error: Decodebin did not pick nvidia decoder plugin.

Anyhow, I need to perform color balancing operations and was wondering how to go about creating a deepstream element for the same(I’m aware of modifying the dsexample plugin).
But I guess I have to import the colorbalance interface to dsexample plugin and go from there, could you point me to any references?

5

Hey @bcao, after some probing I found that the videobalance element videobalance does color-balancing operations.

The following pipeline works
gst-launch-1.0 uridecodebin uri=‘file:///opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_1080p_h264.mp4’ ! videobalance saturation=1.0 ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=I420” ! nvv4l2h264enc bitrate=2000000 ! h264parse ! flvmux latency=200 ! rtmpsink location=‘rtmp:// 13.xxx.xxx.xx’

However setting the saturation property to 1.5, gives errors -
gst-launch-1.0 uridecodebin uri=‘file:///opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_1080p_h264.mp4’ ! videobalance saturation=1.5 ! nvvideoconvert ! “video/x-raw(memory:NVMM), format=I420” ! nvv4l2h264enc bitrate=2000000 ! h264parse ! flvmux latency=200 ! rtmpsink location=‘rtmp:// 13.xxx.xxx.xx’

WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1:
not negotiated
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
8

From the log, the uridecodebin can not output proper format for “videobalance saturation=1.5”, you may try to add nvvideoconvert before videobalance, it is better to use capsfilter to designate the format needed than to let the pipeline to negotiate the format.

It is not deepstream related. It is just a gstreamer skill. Please refer to gstreamer resources.

There are hundreds of elements in gstreamer open source, it is impossible for us to try them one by one. It has nothing to do with deepstream. Please focus on deepsteam topics in deepstream forum.

1 Like
9

I’ve tried this command can work:

gst-launch-1.0 uridecodebin uri='file:///opt/nvidia/deepstream/deepstream-5.1/samples/streams/sample_1080p_h264.mp4' ! nvvideoconvert ! videobalance saturation=1.5 ! nvvideoconvert ! 'video/x-raw(memory:NVMM), format=I420' ! nvv4l2h264enc bitrate=2000000 ! h264parse ! flvmux latency=200 ! rtmpsink location='rtmp:// 13.xxx.xxx.xx'
1 Like
10

Thank you, for the assistance. The reason, I posted here was, when the pipeline is executed through a python script, there’s an Nvidia decoder error. However, that doesn’t occur if the pipeline is executed through the command line.

11

There is no difference between gst-launch-1.0 command and gstreamer python script.