I need to track items as they are inserted into a bin. To do that, I have a tiny YOLOv3 detector that identifies the end effector. From that, I trace the contours around the item it is holding and fit a bounding box around the item. I want to run inference on the item with DeepStream, not the end effector. Since I have to provide a custom function to return the bounding boxes from the tiny YOLOv3 model (parse-bbox-func-name), it seems a natural fit to add my custom logic there. To do that, I need the RGB(A) frames for inference. Is there a way to get them from within the function? If not, how may I build such a model with DeepStream?
The function is given a std::vector of NvDsInferLayerInfo objects. I’m assuming each of them represents one image frame, and the documentation says it contains a (void *) field named buffer that is a "Pointer to the buffer for the layer data". Does that mean they have references to their associated image frames? If so, how may I get them?