SGIE,transfer image to Device from OpenCV custom preprocessing

Hi Deepstream Team!

I am doing a custom preprocessing algorithm on the cropped images before SGIE which is face alignment. Therefore, I decided to follow the dsexample to get the Image from GPU(using get_converted_mat, NvBufSurfaceSyncForCpu …) and have done that. After the face alignment, I have 112*112 Opencv Mat and want to transfer it to GPU for replacing the origin cropped Image in order to pass it through a TensorRT ArcFace model.

How can I do that? Please give me a hint!

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU):GPU
• DeepStream Version:5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version:7.0
• NVIDIA GPU Driver Version (valid for GPU only):440

any help please!

Hey, what’s your pipeline.
Seems it’s similar as deepstream_reference_apps/back-to-back-detectors at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub , could you check it?

Hi there, I don’t see any custom preprocessing on your example. Please through more light into it.

By the way, I have done that by using the idea of the dsexample by coping the full image to CPU for doing custom preprocessing with Opencv and then synchronize it back to the GPU.

I can share the results if anybody is interested in it!

Hi @ledaiduongvnth I am indeed interested in the application that you mention.
I have replied to a post on the same topic some time ago. I have not figured it out how to solve my problem yet.

Thanks in advance!

Hi @borelli.g92!
We did a trick to walk around the problem of doing preprocessing on GPU by using dsexample plugin. More details, you can look at our repos:

The idea is doing face alignment on CPU than change the boxes in object_metas to the boxes on the top of the frame. After getting the output from Secondary nvinfer, we return back the actual boxes to the object_metas

For the example on comment 2, you can do your custom preprocess by installing a probe.

And if you want to do it in dsexample, you can refer the NVIDIA DeepStream SDK Developer Guide — DeepStream 6.1.1 Release documentation , it,s an inverse process of get_converted_mat

Hello, I read the code you shared and wrote it very well. But I’m not sure about a few points. Can you give me some guidance?

  1. On face_ds library code does not use dsexample, how to use it?

  2. On face_ds the code of DS library, landmarks is added to the data structure of nvdsinfraobjectdetectioninfo. How to add landmarks?

  3. What network do your detectors and recognizers use?