Modified NVDewarper plugin produces black frames

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Orin NX 16 GB
• DeepStream Version deepstream:6.3-triton-multiarch docker container
• JetPack Version (valid for Jetson only) 5.1.1 - L4T r35.3.1
• TensorRT Version irrelevant
• NVIDIA GPU Driver Version (valid for GPU only) irrelevant

Hello everyone, I modified the gst-nvdewarper plugin to make it cleaner and simple so that I can build an application on top of it however it is not working as I expected. It produces only black frames. When I compile the unchanged source, it works properly. I also tried the dewarp parts of the code in another application that is not a gstreamer plugin, it is also working. I suspect the CUDA context or driver is not initialized properly because of the multithreaded structure of the gstreamer.

I uploaded the code please help me. I added debug lines there to output the first frame and abort.

compile command:

CUDA_VER=11.4 make install

run command:

gst-launch-1.0 uridecodebin uri=file:///workspace/input_video.mp4 \
                ! nvvidconv ! 'video/x-raw(memory:NVMM), width=4000, height=1500, framerate=30/1, format=RGBA' \
                ! nvdewarper output-resolution=1920x1080 \
                ! nvvidconv ! 'video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1, format=I420' \
                ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=output_video.mp4

gst-nvdewarper.zip (13.8 KB)

3

Since the original plugin and the original library can work. You need to debug what you have modified. If you find any issue with the DeepStream interfaces, you can ask questions or raise issues here.

4

Hi Fiona,
Thanks for your reply. Since the NVWarp library is inside deepstream SDK, I created this topic in this forum. I tried to debug it however unfortunately I couldn’t find the reason for the issue. So I’m asking for help.

5

What is your findings?

6

I implemented another application that is not a gstreamer plugin but uses gstreamer to read frames from videos. Dewarp functions are the same as my gstreamer plugin however it is working perfectly fine while the gstreamer plugin is not. I cannot find the difference between the gstreamer plugin and this application regarding the NVWarp library.

compile command:

mkdir build && cd build
cmake ../
make

run command:

./dewarper

dewarper.zip (4.2 KB)

7

You may try to dump the RGBA data in “in_surface” before “dewarp(in_surface, nvdewarper);” and the RGBA data in “nvdewarper->surface” after "dewarp(in_surface, nvdewarper); "

8

I did it already. The dump of the input surfaces in the plugin and other application are the same. The output(nvdewarper->surface) of the plugin is completely black while the output of the other application is proper dewarped frame. I can provide you with inputs and outputs if you want.

9

I was trying random things and found the solution. I put the warper initialization into dewarp function so that the warp function and the warper reside in the same thread and it worked.