Primary and secondary updating metadata workflow

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) RTX3090
• DeepStream Version 7.0
• JetPack Version (valid for Jetson only)
• TensorRT Version DEEPSTERAM 7.0 ISNTALLAITON
• NVIDIA GPU Driver Version (valid for GPU only) 535
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
**• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

i wanna know more about async and som other thing in deepstream:
1- does secondary and primary always which can be detection or classification model have a same frame_num? even if we use async mode or nvinfer interval property

2- what will hapend to kafkabroker if i use async mode? will it wait until get result and then send the whole frame or not**

3- in this code in deepstream gst-nvinfer source code:

if (nvinfer->classifier_async_mode) {

/\* Asynchronous mode. Push the buffer immediately instead of waiting for \* the results. \*/

nvtxDomainRangeEnd(nvinfer->nvtx_domain, buf_process_range);

nvds_set_output_system_timestamp(inbuf, GST_ELEMENT_NAME(nvinfer));

GstFlowReturn flow_ret = gst_pad_push (GST_BASE_TRANSFORM_SRC_PAD (nvinfer), inbuf);

if (nvinfer->last_flow_ret != flow_ret) {

switch (flow_ret) {

/\\\* Signal the application for pad push errors by posting a error message       \\\* on the pipeline bus. \\\*/

case GST_FLOW_ERROR:

case GST_FLOW_NOT_LINKED:

case GST_FLOW_NOT_NEGOTIATED:

GST_ELEMENT_ERROR (nvinfer, STREAM, FAILED,

      ("Internal data stream error."),            ("streaming stopped, reason %s (%d)", gst_flow_get_name (flow_ret),                flow_ret));

break;

default:

break;

} }

nvinfer->last_flow_ret = flow_ret;

return flow_ret;

} else {

/\* Queue a push buffer batch. This batch is not inferred. This batch is to \* signal the input-queue and output thread that there are no more batches \* belonging to this input buffer and this GstBuffer can be pushed to \* downstream element once all the previous processing is done. \*/ buf_push_batch = new GstNvInferBatch;

buf_push_batch->inbuf = inbuf;

buf_push_batch->push_buffer = TRUE;

buf_push_batch->nvtx_complete_buf_range = buf_process_range;

g_mutex_lock (&nvinfer->process_lock);

if (nvinfer->input_queue_thread)

g_queue_push_tail (nvinfer->input_queue, buf_push_batch);

else

g_queue_push_tail (nvinfer->process_queue, buf_push_batch);

g_cond_broadcast (&nvinfer->process_cond);

g_mutex_unlock (&nvinfer->process_lock);

}

what will happend if inference didn’t compelete before nvdsosd and nvmsgbroker?

3

Yes, frame_number is always the same.

Questions 2 and 3 should be answered together.

In the documentation:

Enables inference on detected objects and asynchronous metadata attachments. Works only when tracker-ids are attached. Pushes buffer downstream without waiting for inference results. Attaches metadata after the inference results are available to next Gst Buffer in its internal queue.

This means that when classifier-async-mode=1, you need to work with nvtracker to get the corresponding object meta from the cache even if the inference is not completed.
So in nvdsosd and nvmsgbroker, you can get all the object meta when using nvtracker.

4

my question was somrhing else, in my senario :
we have pgie(detection) + sgie(detection) + sgie(async classifier) and i wanna get everything in nvdsosd before tee which forwawrd path to nvmsgconv, but when i want to access to them i wanna know what happend to async inference result it stay in next element until it get result?can you tell me about this in more detail

5

You have the following pipeline, correct?

If the classifier is running asynchronously, it won’t wait for inference to complete, but nvmsgconv only needs the correct data in object_meta.

An asynchronous classifier, on the other hand, uses the tracker to fetch object_meta from the cache and append it to each object. Once the asynchronous classifier completes inference, it updates object_meta.

pgie --> sgie --> nvtracker --> sgie --> nvdsosd --> tee --> | --> display
                                                             | --> nvmsgconv --> nvmsgbroker

You can check attach_metadata_classifier function in /opt/nvidia/deepstream/deepstream/sources/gst-plugins/gst-nvinfer/gstnvinfer.cpp