Questions about get_converted_buffer in gstnvinfer.cpp

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU
• DeepStream Version
5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
7.1
• NVIDIA GPU Driver Version (valid for GPU only)
440
• Issue Type( questions, new requirements, bugs)
question

For the debug reason, I have following questions:

  1. what if the input width or height are smaller than network width or height, will it cause any risk?
  2. I wish I could convert the converted_frame_ptr to cv_Image. no idea how to copy this cuda memory to cpu and save it using opencv.
  3. how to make the centered crop, instead of left-top?
  4. where did this code the resizing work, what kind of resize method is used?
  1. For nvinfer plugin, the input video/image resolution can be different to network width and height, nvinfer plugin will do the pre-processing including scaling to meet the network requirment.
  2. Do you want to dump network input picture data? If so, please refer to DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums, item 2.
  3. Do you mean when you by enabling “maintain-aspect-ratio”, you need the input image is centered? If so, current interfaces do not support it.
  4. The code for resizing and other preprocessing is in InferPreprocessor::transform() function in /opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_context_impl.cpp. The algorithm is in /opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_conversion.cu

Hello, Miss Chen.

The info above provided by you help me a lot.

Yes, I mean the input image is centered when I enabled maintain-aspect-ratio.

I have noticed that the algorithm in /opt/nvidia/deepstream/deepstream-5.0/sources/libs/nvdsinfer/nvdsinfer_conversion.cu is supposed to do some jobs like color format convertion or normalization, instead of resizing.

The resize job is taken over by NvBufSurfTransform in gstnvinfer.cpp, But the source code of nvbufsurftransform is not opened, is it correct?

Is it possbile to make some modifcation in the function get_converted_buffer(in gstnvinfer.cpp) to do the center-crop when I enable the maintain-aspect-ratio? for instance, change the padding offsets?

No. It is not open source.

Currently we can not. Our development team will investigate this case. As a workaround, you can retrain your network with the images which are left-top images.

I did some tries to modify some source code in gstnvinfer.cpp, it just could work for now.

however the infer results is not ok, needs to modify the post-process for the new offsets.

1 Like