Integrating external detector with NVDCF tracker

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Is there any example of integrating external Object detectors like yolox with deepstream based trackers ? If not could you tell me how could i approach this problem?

Thanks

3

Can you check the DeepStream samples? You can configure the PGIE to any detection model. nvtracker should works fine:
C/C++ Sample Apps Source Details — DeepStream documentation 6.4 documentation
Python Sample Apps and Bindings Source Details — DeepStream documentation 6.4 documentation

4

Thanks i took a look at them already , so what i exactly need is to only pass the output of external detector to NVDCF tracker , would that be possible without integrating the whole detector?

5

Yes, but you need to feed the same gstbuffer(meta data) as the output of nvinfer.

6

Ok , so NvDsBatchMeta is what we need in this case?

7

Yes, you are right. The NvDsBatchMeta need include the bboxes which generated by detector.

8 
def stream_buffer_probe(pad, info, u_data):
    buffer = info.get_buffer()
    batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(buffer))
    if not batch_meta:
        return Gst.PadProbeReturn.OK

        # Iterate over frames in the batch
    frame_meta_list = batch_meta.frame_meta_list
    while frame_meta_list is not None:
        frame_meta = pyds.NvDsFrameMeta.cast(frame_meta_list.data)

        # Run object detection on the frame
        #detections = detector.detect_objects(frame)

        # Convert the detection results to a format compatible with nvtracker
        bbox_list = []
        detections = []
        detection = {}
        for i in range(5):
            detection['x'] = 1920/2 - i * 10
            detection['y'] = 1080/2 - i * 10
            detection['width'] = i * 2
            detection['height'] = i * 2

        for detection in detections:
            # Convert bounding box coordinates to DeepStream format
            bbox = pyds.NvDsRectangle()
            bbox.left = detection['x']
            bbox.top = detection['y']
            bbox.width = detection['width']
            bbox.height = detection['height']

            # Create object metadata
            obj_meta = pyds.NvDsObjectMeta()
            obj_meta.rect_params = bbox
            obj_meta.confidence = 1.0
            obj_meta.class_id = 0
            # Add more attributes as needed

            # Append the object metadata to the list
            bbox_list.append(obj_meta)

        # Pass the detection results to nvtracker
        # Your code to send detection results to nvtracker here
            tracker_meta = batch_meta.get_tracker_meta(1)
            if tracker_meta is None:
                tracker_meta = pyds.nvds_add_tracker_meta_to_frame(frame_meta, 1)

                # Append object metadata to object history in NvTrackerMeta
            tracker_meta.append_object_info(obj_meta)
    return Gst.PadProbeReturn.OK

@kesong I added probe on streammux so that dummy metadata can be sent to tracker , but it gets stuck inside nvdstracker plugin. Do you know what could be causing it?

9

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

You may need compare the metadata with the one which works (nvinfer->nvtracker).