Resizing frames after dewarping not working as expected

[B.Buschmaas]

General Information:

• Deepstream 5.1
• Jetpack 4.5.1 [L4T 32.5.1]
• Jetson AGX Xavier

Using the python bindings I created a pipeline of which one part is to dewarp a fish eye image, resize it and then crop it for further processing. Everything works so far, but I seem to be restricted in the resolutions I can choose.

This is the config for the dewarper.

[property]
num-batch-buffers=1
output-with=480
output-height=800

[surface0]
surface-index=0
projection-type=1
width=1316
height=2172
focal-length=483
top-angle=55
bottom-angle=-55
pitch=0
yaw=0
roll=0

With output-width/height as in the config everything works as expected. See image 1.

But If I choose something like output-width=842 and out put-height=1072 the resulting image has the right dimensions but the content of the image will be “zoomed in” so that I am missing parts of the image. See image 2.

I would expect a resizing function to output the exact same image (meaning content) with just a different resolution.
For me so far the problem seems to be related to the ratio between output-width and output-height it has to roughly match the ratio of width and height (of surface0) to not get this “zoomed in” effect.

I also tried to not use the output-width/height function of nvdewarper but instead to use a nvvideoconverter to resize the image after it has been dewarped, but this results in the same problem.

Also for some resolutions I get some black borders. In image one such a border can be seen at the bottom.

Am I misunderstanding something about the resize procedure?

[bcao]

Have you checked Nvdewarper dewarp 360 into one image - #16 by bcao

[B.Buschmaas]

@bcao
Yes, I have seen this thread. But I didn’t find anything that applies to my situation. Did you have a specific post of this thread in mind?

Hopefully the following example can make my problem more clear. You should also be able to reproduce it given the uploaded video or any other fish-eye video.
test_video_camL

gst-launch-1.0 filesrc location=test_video_camL.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! \
nvvideoconvert ! m.sink_0 nvstreammux name=m width=1448 height=1448 batch-size=1 num-surfaces-per-frame=1 ! \
nvdewarper config-file=test_dewarp.txt ! \
nvvideoconvert ! nvv4l2h264enc ! h264parse ! qtmux ! filesink location=out.mp4

The used config file

[property]
num-batch-buffers=1

# 1 Works as expected
#output-width=480
#output-height=800

# 2 Unexpected "cropping / zooming in" of the image
#output-width=842
#output-height=1072

# 3 Works as expected
#output-width=1316
#output-height=2172

# 4 Unexpected "cropping / zooming in" of the image
#output-width=800
#output-height=800

[surface0]
surface-index=0
projection-type=1
width=1316
height=2172
focal-length=483
top-angle=55
bottom-angle=-55
pitch=0
yaw=0
roll=0.5

I took a frame from each produced video to be able to showcase the problem here. Images are shown in the same order as in the config file.

I would expect image 2 and 4 to show as much of the room as image 1 and 3, just in a different resolution. Could this be a bug or am I misunderstanding something.

[B.Buschmaas]

@bcao any updates?

[bcao]

Sorry for the late, we are checking.

[B.Buschmaas]

@bcao Any updates?

[B.Buschmaas]

@bcao Any updates?

[B.Buschmaas]

@bcao Any updates?

[B.Buschmaas]

@bcao Are you still looking into this issue? If yes is their any knew information you could pass on to me?

[bcao]

@B.Buschmaas sorry, we are setting up the repro environment with the fish eye camera, will update you ASAP.

[B.Buschmaas]

@bcao Thanks for the update. While this environment will probably help with al kind of different issues, my issue, as stated above, is also reproducible with a fish eye video.

[Amycao]

Hi,
Can you try below dewarper size, it will give better results for all the 4 cases you listed(scaled to different output width and height)
width=3886
height=666