This is a feature request to the engineering team based on discussion on this thread on Nivida tao toolkits recipes repo: https://github.com/NVIDIA-AI-IOT/tao_toolkit_recipes/issues/2
I want to integrate TAO Action recognition classifier to deepstream 6 as a SGIE after PGIE object detector, so that:
PGIE (Faster RCNN) → pass bounding box information of each cows in the frame → Call SGIE activity recognition model only on particular class (e.g class cow) of PGIE, only using bounding box regions as inputs.
Unfortunately, I was told that current action recognition sample in DS6 does not support or work with a detection model. It only supports manually setting region of interests. This is very limiting, because bounding boxes detected from SGIE is never the same in terms its size and location, and it is not possible to hard define region of interests. And there are obviously many objects detected per frame, and it sounds natural for PGIE to do the object localization and let SGIE action model to make subsequent classification decisions.
Classifier must be usuable as SGIE to be really useful in the production, but the current set up seems more like a video frame classifier, which is quite far from being usable in production. Is there any plan to improve this model?