Deepstream python action recognition as sgie - how to get results?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) : GPU
• DeepStream Version : 6.2
• NVIDIA GPU Driver Version (valid for GPU only): 525.147.05
• Issue Type( questions, new requirements, bugs): questions, bug
• How to reproduce the issue?

I have person detection as pgie
i have actionrecognitionnet model provided by nvidia as sgie
i have a preprocess before sgie

self.pgie.link(self.queue2)
self.queue2.link(self.tracker)
self.tracker.link(self.queue3)
self.queue3.link(self.preprocess)
self.preprocess.link(self.queue4)
self.queue4.link(self.sgie1)
self.sgie1.link(self.queue5)

this is config file for pgie

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
labelfile-path=labels_yolov8.txt
model-engine-file=…/primary_yolov8.engine
input-dims=3;640;640;0
uff-input-blob-name=images
batch-size=16
process-mode=1
maintain-aspect-ratio=1
symmetric-padding=0
model-color-format=0
network-mode=0
num-detected-classes=80
cluster-mode=4
interval=5
gie-unique-id=1
output-blob-names=num_dets;bboxes;scores;labels
parse-bbox-func-name=NvDsInferParseCustomYoloV8
custom-lib-path=libnvdsinfer_custom_bbox_yoloV8_infer.so
network-type=0
detected-min-w=50
detected-min-h=50
detected-max-w=500
detected-max-h=500

this is config file for preprocess

[property]
enable=1
gpu-id=0
unique_id=3
process-on-frame=0
target-unique-ids=2
operate-on-gie-id=1
network-input-order=2
network-input-shape=1;3;32;224;224
network-color-format=0
tensor-data-type=0
tensor-name=input_rgb
processing-width=224
processing-height=224
scaling-pool-memory-type=0
scaling-pool-compute-hw=0
scaling-filter=0
tensor-buf-pool-size=8

custom-lib-path=/opt/nvidia/deepstream/deepstream/lib/libnvds_custom_sequence_preprocess.so
custom-tensor-preparation-function=CustomSequenceTensorPreparation

[user-configs]
channel-scale-factors=0.007843137;0.007843137;0.007843137
channel-mean-offsets=127.5;127.5;127.5
stride=1
subsample=0

this is config file for sgie

[property]
gpu-id=0
tlt-encoded-model=…/resnet18_3d_rgb_hmdb5_32.etlt
tlt-model-key=nvidia_tao
model-engine-file=…/resnet18_3d_rgb_hmdb5_32.etlt_b4_gpu0_fp16.engine
process-mode=2
labelfile-path=labels.txt
gie-unique-id=2
batch-size=1
input-tensor-from-meta=1
network-mode=2
network-type=1
output-tensor-meta=1
parse-classifier-func-name=NvDsInferParseCustomNVPlate
custom-lib-path=libnvdsinfer_custom_impl_action.so
num-detected-classes=5

  • The pipeline runs without an error
  • The person detector is working properly and I can get the results both in NvDsObjectMeta inside probe and bounding boxes are also properly displayed on the output rtsp

The problem is that my sgie is not producing any results. I have placed a cout inside NvDsInferParseCustom c++ code and that is not printing anything

3

Just from your snippet code, there is no preprocess plugin.

You can also add a probe function to the src_pad of the sgie and check the results from the probe function.

4

sorry my bad @yuweiw . i made an error in posting that snippet. it is actually

        self.pgie.link(self.queue2)
        self.queue2.link(self.tracker)
        self.tracker.link(self.queue3)
        self.queue3.link(self.preprocess)
        self.preprocess.link(self.queue4)
        self.queue4.link(self.sgie1)
        self.sgie1.link(self.queue5)

so the preprocess is present.

result is still the same as mentioned in post above

5

Could you refer to our C/C++ open source code demo sources\apps\sample_apps\deepstream-3d-action-recognition first? You can run that demo first and compare it to your python code.

6

@yuweiw i have already tried it. Its doing recognition on predefined rois. what I’m trying to do here is recognition on ROIs generated by PGIE.

7

We also have a similar C/C++ demo that you can refer to, deepstream-pose-classification.

8

I have already checked this as well @yuweiw and the configs ive shared are written following the two examples you have shared

also, the bodypose classification example has

[group-0]
src-ids=0;1
custom-input-transformation-function=CustomTransformation
process-on-roi=1
process-on-all-objects=1
roi-params-src-0=0;0;100;100
input-object-min-width=100
input-object-min-height=100

in the preprocess config

as far as i understand here too ROI are fixed and are not coming from pgie detector, right?

9

It depends on whether you have done the math based on this ROI. Anout your issue, could you try to add the [group-0] in your preprocess config file? At least you should configure src-ids, otherwise it won’t process any of the sources.

10

on adding just

[group-0]
src-ids=0

i get the error

Error: gst-library-error-quark: Configuration file parsing failed (5): gstnvdspreprocess.cpp(448): gst_nvdspreprocess_start (): /GstPipeline:pipeline0/GstNvDsPreProcess:preprocess-plugin:
Config file path: configs/config_preprocess_3d_custom.txt
Exiting app

on adding

[group-0]
src-ids=0
process-on-roi=1
process-on-all-objects=1
roi-params-src-0=0;0;100;100

there is no error in parsing the config, but the pipeline just gets stuck after a few frames .

11

Maybe there’s something wrong with your preprocess code, you can try to add some log messages in it and debug why there are crash and hang problems.

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks