Please provide complete information as applicable to your setup.
• Hardware Platform: GTX 1080 • DeepStream Version: 5.0 • NVIDIA GPU Driver Version (valid for GPU only): 440.33.01
I am working on a pipeline which face detection as primary inference and generate face feature vector as secondary inference. Before the secondary inference, I have a face landmark detector which is added to user-meta-data. I want to align the faces using these landmarks points to feed the sgie.
Presently I am achieving this by changing the(custom plugin dsexample) blur-objects function to align-objects where I change the ROI of faces(in_mat) to aligned faces. But in this case, the original frames are distorted.
Is there a way to define some function which preprocesses the in_mat for the sgie before feeding it.
Any help will be appreciated.
Customizing preprocess will make deepstream very flexible and will be useful for many more test cases if it is supported later.
The reference provided seems to be different than my use-case. It suggests how to provide one models output as input to other as ensemble model for trition(In the post they have face detection and landmark given to align-model).
For my case I do not use model for alignment rather a algorathim approach to create new MAT for aligned objects. If there is some way I could attach those new aligned-Mat(containing the aligned faces) to the buffers and direct the secondary inference directly on these buffers without changing the orig frame.
The solution @bcao suggest is not what I was looking for. The only way I found was to do some kind of preprocessing in the original image itself using a custom plugin and pass it downstream. That is not a good way for most use cases as I preprocess in the Orig frame itself.
the buffer is on the GPU so applying opencv filters may be issue. Either u have to use the buffer transform api of deepstream. Also look at the blur object option on dsexample there the objects are cropped and blurred.
I have just realized that I get 5168 Segmentation fault (core dumped)
Even if I try to copy the buffer instead of doing a transform: cv::Mat image_copy = in_mat.clone();
However, no segmentation fault at all if I use another opencv function such as: cv::filter2D(in_mat, in_mat,-1, kernel);
The problem is that of course the function is not doing what I need to do…
But the PGIE is correctly receiving the filtered images:
I would not know the answer to your query for sure, but I have also tried a few affine transformations (wrapAffine) on the crop of the orig frame. It worked for me. I won’t be able to tell if there is an issue with some specific transformation. I won’t be able to share any code as it is proprietary.