NvDewarper not working properly

[rajupadhyay59]

Hi everyone,

I was trying to apply nvdewarper to my double fisheye camera and was trying to implement Equirect2Perspective option using this command.

GST_DEBUG=3 gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_qHD.mp4 ! nvvidconv ! tee name=t t.src_0 ! nvdewarper config-file= ./dewarper_config_1.txt ! m.sink_0 t.src_1 ! nvdewarper config-file= ./dewarper_config_2.txt ! m.sink_1 t.src_2 !  nvdewarper config-file= ./dewarper_config_3.txt ! m.sink_2 t.src_3 !  nvdewarper config-file= ./dewarper_config_4.txt ! m.sink_3 nvstreammux name=m width=2560 height=480 batch-size=4 num-surfaces-per-frame=1 ! nvmultistreamtiler columns=4 rows=1 width=2560 height=480 ! nvvideoconvert ! nvinfer config-file-path= config_infer_primary.txt ! queue ! nvdsosd display-mask=1 ! nvvidconv ! nv3dsink sync=0

I know my gst command shows uridecodebin3 instead of my camera but the output is similar which I have attached. (Funny thing is, when I ran the same command with same configuration files on my x86 machine on a triton docker, it works absolutely fine)

I also tried an opencv method and it also displays this weird black lines. (Again same code and it works fine on my x86 machine).
I created a virtualenv to use the opencv version similar to that of x86 but it still displays the black lines.

If anyone has any idea what the cause is or how can I resolve this, please let me know.

I am also attaching one of my nvdewarp config files.
dewarper_config_4.txt (170 Bytes)

Thanks!

System Environment:
Jetson AGX ORIN
Jetpack: 5.1.1
Deepstream: 6.2
TensorRT: 8.5.2.2
Opencv: 4.6 + CUDA

[Fiona.Chen]

@rajupadhyay59

GST_DEBUG=3 gst-launch-1.0 uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_qHD.mp4 ! nvvidconv ! tee name=t t.src_0 ! nvdewarper config-file= ./dewarper_config_1.txt ! m.sink_0 t.src_1 ! nvdewarper config-file= ./dewarper_config_2.txt ! m.sink_1 t.src_2 ! nvdewarper config-file= ./dewarper_config_3.txt ! m.sink_2 t.src_3 ! nvdewarper config-file= ./dewarper_config_4.txt ! m.sink_3 nvstreammux name=m width=2560 height=480 batch-size=4 num-surfaces-per-frame=1 ! nvmultistreamtiler columns=4 rows=1 width=2560 height=480 ! nvvideoconvert ! nvinfer config-file-path= config_infer_primary.txt ! queue ! nvdsosd display-mask=1 ! nvvidconv ! nv3dsink sync=0

I don’t think this pipeline can run on x86. Besides you can not put nvinfer after nvmultistreamtiler. The pipeline is wrong.

[rajupadhyay59]

I cannot or I should not? (Because this command does run on the jetson)

Also regarding the pipeline, I need to create a new one anyway but that is not the point, the point is that even the simplest gst-pipeline works fine on the triton docker but when trying to run it on the jetson, it gives those weird line stuff.

Ok this command I remember running on triton and it worked fine.

gst-launch-1.0 thetauvcsrc ! decodebin ! nvvideoconvert ! nvdewarper config-file"dewarper_config.txt" ! m.sink_0 nvstreammux name=m width=640 width=480 batch-size=1 ! nvmultistreamtiler columns=1 rows=1 ! nveglglessink sync=0

To run on jetson, I changed it like this

gst-launch-1.0 thetauvcsrc ! h264parse ! nvv4l2decoder ! nvvidconv ! nvdewarper config-file= ../dewarper_config.txt ! m.sink_0 nvstreammux name=m batch-size=1 width=640 height=480 ! nvmultistreamtiler columns=1 rows=1  ! nvegltransform  ! nveglglessink sync=0

So does OpenCV, works alright on the triton but not on the jetson.

So, do you have any idea what could be causing this?

[Fiona.Chen]

@rajupadhyay59

I cannot or I should not? (Because this command does run on the jetson)

You shouldn’t.

nvvideoconvert works both on x86 and Jetson.

@rajupadhyay59

So does OpenCV, works alright on the triton but not on the jetson.

It has nothing to do with DeepStream.

[rajupadhyay59]

Ok, thanks for letting me know, I’ll remember that nvmultistreamtiler thing.

OpenCV has got nothing to do with deepstream yes but NvDewarper is acting the same way.
If you or anyone has any experience with this, let me know.

[Fiona.Chen]

@rajupadhyay59

NvDewarper is acting the same way

Do you mean nvdewarper output wrong image on Jetson? Please provide the correct pipeline you are using.

[rajupadhyay59]

Hi,
Sorry I was a bit busy.

This is the pipeline I am currently using to verify perspective transformation

gst-launch-1.0 uridecodebin3 uri=file:///home/user/video.mp4 ! nvvidconv ! nvdewarper config-file= ./dewarper_config.txt ! m.sink_0 nvstreammux name=m batch-size=1 width=640 height=480 ! nvmultistreamtiler ! nv3dsink sync=0

I tried many algorithms and NvDewarper too so that I can use the perspective form of my equirectangular image.

I found out that not just NvDewarper but even other algorithms have this weird stuff going on but it is not affecting the performance as such. (Basically the transformation is not smooth)

It is no longer an issue but thank you for the nvmultistreamtiler thing.