Clarifications about the nvdspreprocess component

[juan.isern95]

Hello everyone,

Please provide complete information as applicable to your setup:

- Hardware Platform (Jetson / GPU): RTX2080TI
- DeepStream Version: 6.2 (python bindings)
- TensorRT Version: 8.5.2-1+cuda11.8
- NVIDIA GPU Driver Version (valid for GPU only): 530.30.02
- Issue Type( questions, new requirements, bugs): Question

I am experimenting with nvdspreprocess and I have several questions regarding the component that are not clear from the documentation:

1. I see that there are properties that are repeated with respect to nvinfer (pixel-normalization-factor vs net-scale-format, offsets, etc.). When these properties are repeated, is the preprocessing done twice? In case I select them in the nvpreprocess config, should I remove them from nvinfer? If they are repeated, which one has priority? What is the best practice to set these properties?

2. When defining ROI(s), if more than one is defined on a certain stream from an input source, is an inference made for each ROI (i.e. an inference batch with each ROI)? Or is a “mask” generated internally in the component where the rest of the image regions outside the ROI(s) are omitted and an inference is performed for a frame independently of the number of ROIs?

3. If I have a high resolution input (for example, 4k), and I want to zoom in a specific region of the image, could I make a ROI in that region keeping the original resolution? I want to know if the output of the nvdspreprocess would be the crop of the original frame, scaling the original resolution to the target resolution (processing-width, processing-height)? For example: if I generate an ROI in 1/4 part of a 4k image, the output of the preprocess would be the result of scaling 1920x1080 to the target width and height? This would be particularly attractive to perform various nvinfer on different regions of the same high quality panoramic image which without this preprocessing would be very small.

4. Finally, is it possible to update on-the-fly the properties of different input sources? I am implementing the runtime_source_add_delete example and I want to know how to add ROI(s) dynamically without having to restart the pipeline. I don’t know if it is possible to update the file at runtime.

Best regards,

Juan Isern

[fanzh]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

1. if using nvdspreprocess + nvinfer in pipeline, preprocessing will be done in nvdspreprocess plugin, nvinfer will do inference on the preprocessed tensor data directly, it will not do preprocessing again.
2. each stream can have multiple ROIs, only the regions inside the ROIs will be done inferences, the regions outside the ROIs will not be done inferences.
3. you can set symmetric-padding and maintain-aspect-ratio to keep the aspect ratio of ROI. please refer to explanation doc.
4. currently nvdspreprocess dose not support updating cfg on-the-fly because old date will not be removed. but nvdspreprocess is opensource, you can customize it.