Hi, I want to get something done that was discussed quite a few times in this forum but was never really answered or the provided answer was for an outdated deepstream version.
My pipeline looks something like this: pgie (detector, outputs bboxes) → sgie1 (network-type=100, output-tensor-meta=1; outputs landmarks which i parse and attach to the respective bbox meta) → sgie2
sgie2 expects a bbox that is aligned in respect to the landmarks from sgie1’s output. So the question is how to realize that?
Here it is suggested to do an affine transformation on every bbox. I have no idea how this should be realized within deepstream API as a bbox is defined by it’s upper left corner as well as width and height, so it gives a rectangle but applying an affine transformation on a rectangle not necessarily yields a rectangle.
Next thing is that the piece of code which is suggested to be changed in /opt/nvidia/deepstream/deepstream-4.0/sources/libs/nvdsinfer/nvdsinfer_context_impl.cpp is not available anymore in my deepstream version.
This suggests to follow the dsexample plugin and to use NPP API. However I didn’t manage to find any example how to use the NPP API. Additionally it would be nice to see an example how to connect DS and NPP API. Maybe someone can point to a resource.
You can update/change the meta data (left corner as well as width and height) via installing the probe on sgie1 src pad in your case, then sgie2 will do infer on your updated bbox, you need to configure sgie2 do inference based on sgie1 through operate-on-gie-id. Referring Gst-nvinfer — DeepStream 6.3 Release documentation
Unfortunately the above picture cannot be mapped to DS API via left corner, width and height. However, I want to crop this box like this (Excuse my crappy free hand paint cutting)
I don’t see how this could be accomplished using left corner, width and height. You might be able to do it in OpenCV but I guess NPP API is also capable of that. Any idea how? I would like to stay in the NVIDIA stack and ideally handle all this or as much as possible on the GPU. Additionally I would probably need to not only change the rect_params fields in the probe callbacks but also some buffers I guess?
NvBufSurfTransform and convert_batch_and_push_to_input_thread is two function in gstnvinfer.cpp, I mean you need to replace current NvBufSurfTransform with “affine transformation” you need to implement it yourself.