Deepstream-app(SECONDARY_DETECTOR)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

For the deepstream-app example, I want to change the secondary classifier to the detector

I have referred to back to back example

/* Change this to 0 to make the 2nd detector act as a primary(full-frame) detector.

  • When set to 1, it will act as secondary(operates on primary detected objects). */
    #define SECOND_DETECTOR_IS_SECONDARY 1

in Deepstream-app how do change secondary classifier to the detector, that operates under the primary detector

create_common_elements (NvDsConfig * config, NvDsPipeline * pipeline,
    GstElement ** sink_elem, GstElement ** src_elem,
    bbox_generated_callback bbox_generated_post_analytics_cb)
{
  gboolean ret = FALSE;
  *sink_elem = *src_elem = NULL;

  if (config->primary_gie_config.enable) {
    if (config->num_secondary_gie_sub_bins > 0) {
      if (!create_secondary_gie_bin (config->num_secondary_gie_sub_bins,
              config->primary_gie_config.unique_id,
              config->secondary_gie_sub_bin_config,
              &pipeline->common_elements.secondary_gie_bin)) {
        goto done;
      }
      
       gst_bin_add (GST_BIN (pipeline->pipeline),
          pipeline->common_elements.secondary_gie_bin.bin);

      if (!*src_elem) {
        *src_elem = pipeline->common_elements.secondary_gie_bin.bin;
      }

      if (*sink_elem) {
        NVGSTDS_LINK_ELEMENT (pipeline->common_elements.secondary_gie_bin.bin,
            *sink_elem);
      }
      *sink_elem = pipeline->common_elements.secondary_gie_bin.bin;
    }
  }
2

Please refer to secondary_detector_config.txt in back-to-back-detector example to set secondary classifier to secondary detector.
U need to update the secondary classifier config group section as per above mentioned secondary detector config file.

3 
[secondary-gie0]
enable=1
gpu-id=0
#model-engine-file=../../models/lp_eu_full/model_iter_45000.caffemodel_b30_fp16.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=30
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
config-file=lp_eu_full.txt

[tests]
file-loop=0

in lp_eu.txt

[property]
gpu-id=0
process-mode=2

it throws out a segmentation fault

mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),

custom-lib-path, network-mode(Default=0 i.e FP32)

4

Would you mind to share the lp_eu_full.txt which should refer secondary detector config file in comment #2?

5 
[property]
gpu-id=0

model-file=odel_iter_45000.caffemodel
proto-file=net_tensorRT.proto
model-engine-file=model_iter_45000.caffemodel_b30_fp16.engine
labelfile-path=corresp.txt

batch-size=30
process-mode=2
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=2
interval=0
gie-unique-id=2
output-blob-names=[removed, this is working for primary, plugin is tested and working]
parse-bbox-func-name=NvDsInferParseCustomSSD
custom-lib-path=libnvdsinfer_custom_impl_ssd_lp.so
enable-dbscan=1
input-object-min-width=64
input-object-min-height=64

[class-attrs-all]
threshold=0.3
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.3
#minBoxes=3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

Do I need to add

operate-on-gie-id=1
gie-unique-id=2
operate-on-class-ids=1;2;3;4;5;6;7
6

If your case is to make the 2nd detector act as a primary(full-frame) detector, you do not need this.

7

Primary detector [car, bus] → Secondary detector[ROI] → Secondary classifier[0]–> Secondary classifier[1]

This is the pipeline

8

There is no reference of process-mode 2 in the reference documentation

https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/blob/master/back-to-back-detectors/secondary_detector_config.txt

https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/#page/DeepStream_Development_Guide%2Fdeepstream_app_config.3.1.html
9

Gst Properties

process-mode

Infer Processing Mode

1=Primary Mode
2=Secondary Mode

Integer, 1 or 2

10

HI
about change the secondary classifier to the detector, as I know, classifier can not be used as detector

11

I went through deepstream_primary_gie_bin.c and deepstream_secondary_gie_bin.c

I think we can set it up as detector, need to create bbox function for secondary gie in deepstream_app.c[371]. I will keep you posted

12

/* Change this to 0 to make the 2nd detector act as a primary(full-frame) detector.

  • When set to 1, it will act as secondary(operates on primary detected objects). */
#define SECOND_DETECTOR_IS_SECONDARY 1

g_object_set (G_OBJECT (secondary_detector), "config-file-path", "secondary_detector_config.txt",
          "unique-id", SECONDARY_DETECTOR_UID, "process-mode", <b>SECOND_DETECTOR_IS_SECONDARY ? 2 : 1</b>, NULL);

what does this mean ? unique-id 2, process-mode 1 why do you need to double check ?

13

process-mode as explained before
unique-id, see this, https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_details.02.01.html%23wwpID0E0IZ0HA
Gst Properties part

14

Can you release an example using NVinfer as primary and secondary in the same program

15

you can refer to back to back detector

16

Thanks, we resolved it, we are releasing it at Berlin event.

17

Hi Ravik, I want to integrate a secondary detector in deepstream-app. How did you modify the source code to support this, could you share some experience? Many thanks