Using Secondary Inference on all objects of a particular class in a frame

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

Please provide complete information as applicable to your setup.

• Hardware Platform ( GPU)
• DeepStream Version5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 440.64.00

I am trying to parse they meta data coming out of the https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/deepstream-test2 example

I have removed the sgie2 and sgie3 elements and updated the sgie1 config file to the following
[property]
gpu-id=0
net-scale-factor=1
model-file=…/…/…/…/samples/models/Secondary_CarColor/resnet18.caffemodel
model-engine-file=…/…/…/…/samples/models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
proto-file=…/…/…/…/samples/models/Secondary_CarColor/resnet18.prototxt
mean-file=…/…/…/…/samples/models/Secondary_CarColor/mean.ppm
labelfile-path=…/…/…/…/samples/models/Secondary_CarColor/labels.txt
int8-calib-file=…/…/…/…/samples/models/Secondary_CarColor/cal_trt.bin
batch-size=14
# 0=FP32 and 1=INT8 mode
network-mode=1
input-object-min-width=0 #changed this
input-object-min-height=0 #changed this
process-mode=2
model-color-format=1
gpu-id=0
gie-unique-id=2
operate-on-gie-id=1
operate-on-class-ids=0
is-classifier=1
output-blob-names=predictions/Softmax
classifier-async-mode=1
classifier-threshold=0 #changed this

I want to be able to access secondory classifier inference on all objects in a frame
but for some reason
l_class=obj_meta.classifier_meta_list
returns None value for most objects of a particular class ( for e.g if 6 objects of class 0 are in a frame it will return Not None value for only 1/2 max)

How do you suggest changing the config to be able to access this for all objects in a frame?

#3

I didn’t get, can you explain more?

#4

sorry for the incoherence
basically what I’m saying is in deepstream-test-app2(the one with sgie1,sgie2 and sgie3) (https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-test2/deepstream_test_2.py)

If I augment the pipeline by removing sgie2 and sgie3 (All I’m left with is sgie1) (for context sgie1 is a car color classifier operating on only the vehicle class)

What I’ve noticed is that sgie1 one does not do inference on all vehicles in a frame (say there are 10 vehicles in a frame, I can access and see classifier meta data of atmost 2)

I would like to be able to use sgie on all objects of a particular class ( class is vehicles in this instance) , but I’m unable to achieve this behaviour

I have tried modifying the min-input height, width and classifer threshold in the sgie config file and I’m getting the same issue

NOTE : To observe this yourself you can remove sgie2 and 3 from deepstream-test-app2 and observe the osd. What will happen is that not all vehicles will get the output of classifier attached to it.

If I would like sgie to operate on all objects (of a particular class ) in frame , what changes do you suggest I make?

Should I provide you with instructions to reproduce this?

#5

Could you disable this config item and give a try?

#6

Thanks that did the trick.

In the documentation the description for this config item is as follows

“Enables inference on detected objects and asynchronous metadata attachments. Works only when tracker-ids are attached. Pushes buffer downstream without waiting for inference results. Attaches metadata after the inference results are available to next Gst Buffer in its internal queue”

could I know what does this have to to do with sgie inferring on every object? and why was it not doing so before when it was disabled?

#7

When you enable classifier-async-mode, the meta data will be attached asynchronously

1 Like