Apply opencv frame to Sgie

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
RTX2080
• DeepStream Version
6.1
• JetPack Version (valid for Jetson only)
• TensorRT Version
8.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Im working on RetinaFace model, which has 3 outputs, bbox, conf and landmarks. I’m doing alignment using the extracted landmarks from the model . So, I have converted the cropped image of the face to mat and I did the alignment using Opencv. Next step is feed this mat to a recognition model.

How to feed the opencv mat to SGIE?? I’m doing all of these in osd_sink_pad_buffer_probefunction from deepstream_infer_tensor_meta_test.cpp

There is no way to feed such things since the buffer should be GPU memory.

Can you elaborate the detailed algorithm? Do you mean you cropped the image of the face with bbox and then do some alignment according to the landmarks?

There is already bbox based image cropping inside nvinfer plugin, so we need to know what kind of preprocessing is missing.

Yes

I did the preprocessing for the alignment using opencv and now I want to pass the aligned face to SGIE for feature extraction

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

You should add your processing into the preprocessing part in nvinfer source code. nvinfer needs GPU buffer for processing(including inferencing by TensorRT), if you need to use opencv, you need to copy the CUDA memory to CPU memory, processing by opencv and then copy the result back to CUDA memory. It will make the whole processing be very slow. The better way is to implement the algorithm with CUDA. Gst-nvinfer — DeepStream 6.1.1 Release documentation
Another way is to customize your own nvdspreprocess libs to generate tensor data with your algorithms. Gst-nvdspreprocess (Alpha) — DeepStream 6.1.1 Release documentation
All source code can be found in /opt/nvidia/deepstream/deepstream/sources/

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.