Performace issues with dewarper

Unfortunately my last topic was closed. How to remove lens distortion from input images in Deepstream? - #17 by yuweiw I tried the patch, but I didn’t see any improvement in performance. With dewarper disabled I have 54 fps with almost constant 100% GPU utilization. When I activate the dewarper I only have 18 fps and my GPU load jumps back and forth from 0% to 65%. Can someone help me please? All necessary information is listed in the old topic.

• Hardware Platform (Jetson / GPU) Jetson Orin NX 16GB
• DeepStream Version Deepstream-6.2
• JetPack Version (valid for Jetson only) Jetpack 5.1
• TensorRT Version 5.1
• OpenCV without CUDA
The last topic was almost a year old. Could you please provide the following information again on this topic?

  1. How do you use that patch on the last topic?
  2. How do you get the fps and GPU loading?
  3. What’s your whole pipeline and could you attach your source video to us?

After installing deepstream-6.2 I ran the following patch: DeepStream SDK FAQ - #31 by yuweiw

It is then possible to activate the dewarper. I do this in my deepstream_config:

[dewarper]
enable=1
config-file=config_dewarper_1280x720.txt
gpu-id=0
source-id=0
nvbuf-memory-type=0

This ist my config_dewarper:
config_dewarper_1280x720.txt (2.1 KB)

  1. I changed the code and recompiled it with make install. However, this hasn’t changed anything. How to remove lens distortion from input images in Deepstream? - #17 by yuweiw
  2. I can see the GPU utilization with jtop. I get my FPS with:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

  1. This is my pipeline:

    I use as input source: /opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_1080p_h265.mp4

We have made the dewarper in deepstream-app available in the latest version. Could you update to the latest version and try?

samples/configs/deepstream-app/source2_dewarper_test.txt

I have tried that on my Jetson Orin, the GPU load is above 80%.

Is there a way I can upgrade to the latest Deepstream version without re-flashing the Jetson? We have a special kernel overlay that was developed exactly for this jetpack version. In addition, we already have over 300 Jetsons running in our series product that we cannot simply re-flash.

I tested your config and get the error:

** WARN: <parse_dewarper:1120>: Unknown key ‘num-output-buffers’ for group [dewarper]

Could this be related to the fact that I have such poor performance? What is your FPS with dewarper enabled and disabled?

No.

No. It depends on the DeepStream version, we have some optimizations for dewarper in the new version.

Jetson Orin with DeepStream 6.4:
enable: Around 112
disable: Around 270

Some algorithms in the dwarper plugin will result in a decrease in fps. This is normal. What is your expected fps with dewarper?

If you already have over 300 Jetsons, would you mind telling us your company name and project? If you don’t want to disclose the information, you can just message to me in private.

I didn’t expect that the dewarper would result in such a high FPS loss. In your case with Deepstream 6.4 the FPS also drops by over half. However, the FPS and GPU are not fully utilized. I get similar values with my code.

Where can the high FPS drop come from? With OpenCV and cv::remap I don’t have such performance problems.

We’ll check if the fps reduction is normal with DeepStream 6.4 on Jetson Orin first.