Accessing PGIE Tensor Inference Values from Nvds Tracker

Hi,

I have a question regarding DeepStream and accessing inference values from PGIE in nvds_tracker. The key here is that the PGIE not only outputs bounding box values, but also additional learned representations that need to be accessed by the tracker.

Suppose that the pipeline structure is the same deepstream-test 4.

Assuming that we implement methods defined in nvdstracker.h to customize the tracker, is there a way to obtain the output tensors from the PGIE inside of the following function or any functions defined in nvdstracker.h?

NvMOTStatus NvMOT_Process(NvMOTContextHandle contextHandle, NvMOTProcessParams *pParams, NvMOTTrackedObjBatch *pTrackedObjectsBatch) {
    // Want to access PGIE tensor outputs that are available 
    // inside of nvdsinfer_custombboxparser.cpp implementing "nvdsinfer_custom_impl.h"
}

If this is not possible, I am assuming that the output tensors can also be accessed via NvDsBatchMeta (please correct me if I am wrong).
Is there a way to access the NvDsBatchMeta via function implementations in nvdstracker.h?

DeepStream Environment Specifications

• Hardware Platform (Jetson / GPU): dGPU
• DeepStream Version: 5.1
• TensorRT Version: 7.2.2.1
• NVIDIA GPU Driver Version (valid for GPU only): 460.32.03

Thank you for your time.

Hi,
You can get access tensor output by enabling nvinfer property output-tensor-meta, please refer to function pgie_pad_buffer_probe in sample deepstream-infer-tensor-meta-test for how to parse it.
regarding low level tracker implementation, here is one topic you can refer to, DCF Tracker REFUSING to work while using low level library - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums

You can get access tensor output by enabling nvinfer property output-tensor-meta, please refer to function pgie_pad_buffer_probe in sample deepstream-infer-tensor-meta-test for how to parse it.

Thank you for the response.
I examined the deepstream-infer-tensor-meta-test sample and saw the example of using a src_probe function to access the tensor metadata.

My original queestion was: Is there a way to access the same tensor metadata or the PGIE inference values inside of NvMOTStatus NvMOT_Process defined in nvdstracker.h?

Thank you for your time.

PGIE tensor data is stored in NvDsFrameMeta’s user meta. NvDsFrameMeta is not directly passed via NvMOT APIs, but can be accessed indirecly by the following steps.
(1) In nvdstracker.h, from NvMOTProcessParams access each NvMOTObjToTrack. NvMOTObjToTrack.pPreservedData is a pointer to NvDsObjectMeta of that input object.
(2) From NvDsObjectMeta.base_meta.batch_meta acquire NvDsBatchMeta for that batch.
(3) From NvDsBatchMeta acquire each NvDsFrameMeta’s frame_user_meta_list. That contains the PGIE tensor data.