Make RGBA buffer transparent but not the nvdsosd bounding boxes

I am making a RGBA buffer transparent by zeroing out the alpha channel before feeding it to nvdsosd, while the class labels are not transparent the bounding boxes are transparent, is there a way to make only the image transparent?

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)

• Issue Type( questions, new requirements, bugs)

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

**• Hardware Platform: RTXA4000
**• DeepStream Version: 6.0
**• TensorRT Version: 8.0.1
**• NVIDIA GPU Driver Version: 470.94
**• Issue Type: Question regarding nvdsosd rectangle drawing using alpha in GPU MODE

I am using this pipeline:

    std::ostringstream launch_stream;
    launch_stream
    << "filesrc location=/Videos/sample_720p.mp4 ! "
    << "qtdemux ! "
    << "queue ! "
    << "h264parse ! "
    << "nvv4l2decoder ! "
    << "nvvideoconvert ! "
    << "m.sink_0 nvstreammux name=m gpu-id=0 batch-size=1 width=1280 height=720 enable-padding=0 live-source=0 batched-push-timeout=40000 ! "
    << "nvinfer model-engine-file=/opt/nvidia/deepstream/deepstream-6.0/samples/triton_model_repo/yolov4_1_3_416_416_static/1/model.plan "
    << "gpu-id=0 config-file-path=/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app-triton/config_infer_primary_yoloV4.txt "
    << "batch-size=1 unique-id=1 ! "
    << "nvvideoconvert ! "
    << "nvdsvideotemplate customlib-name=/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libcustom_impl.so ! "
    << "nvdsosd process-mode=0 ! "
    << "nvvideoconvert flip-method=6 ! "
    << "video/x-raw(memory:NVMM), width="<< WIDTH <<", height="<< HEIGHT <<",format=(string)RGBA ! "
    << "appsink name=mysink ";

The nvdsvideotemplate plugin zeros out the alpha channel, the nvdsosd plugin uses the CPU to draw the rectangles as process-mode is set to zero. appsink blends the outputted bounding-boxes and label drawings onto another image, but not the original input to the nvinfer plugin because that has become a transparent image. When using process-mode=1 (GPU MODE) with the nvdsosd plugin, the bounding-boxes become transparent as well. Can we get GPU-optimized drawings that take alpha into consideration? What happens with using the CPU to draw, would it download the buffer to hostmemory and upload it? This seems to be inefficient.

why do you zero out the alpha channel? will it let the origin frame be transparent ? please share your current screenshot, and what kind of result are you expecting?
here is the my test without code modification.
image

Yes, we do not intend to display the origin frame, we intend to display another frame with the origin bounding box results. This works with process-mode=0 as shown:


However when using process-mode=1 we get this:

It seems like GPU mode is ignoring the alpha channel when drawing the rectangles, so the rectangles are transparent as well.

how do you zero out the alpha channel? could you provide code to reproduce this issue?

Hi,

Here is basic code to reproduce the issue.

nvdsosd-transparency-demo-main.zip (29.9 KB)

The pipeline is as follows: filesrc > … > nvinfer > nvvideoconvert > nvdsvideotemplate > nvdsosd > appsink

Nvdsvideotemplate has a custom kernel which makes the incoming buffer transparent by zeroing out the alpha channel, appsink uses opencv, it takes the result of the nvdsosd rendering and blends it with an opaque gray image, then outputs the results.

Follow the readme:

# 1- Build container
./docker_build.sh

# 2 - Run container
./docker_run.sh

#3 - Run application
cd appsink_nvmm/build
./appsink_nvmm

Your result should be:

# 4 - change process-mode from 0 to 1 in appsink_nvmm.cpp
# 5 - repeat steps 1 through 3

Now your results should be:

Seems like the gpu mode doesn’t take alpha into consideration when rendering the bounding boxes, though having the boxes opaque is a desired feature.

will file a bug, will update here if fix.
that is, in nvdsosd GPU mode, modifying rect_params->border_color.alpha dose not take effect while CPU mode is OK.

  1. after internal discussion, drawing transparent border is supported in CPU mode not in GPU mode.
  2. about “What happens with using the CPU to draw, would it download the buffer to hostmemory and upload it?”, there will not a memcopy if upcoming buffer is allocated using cudamalllochost.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.