Deepstream-app(SECONDARY_DETECTOR)

For the deepstream-app example, I want to change the secondary classifier to the detector

I have referred to back to back example

/* Change this to 0 to make the 2nd detector act as a primary(full-frame) detector.

  • When set to 1, it will act as secondary(operates on primary detected objects). */
    #define SECOND_DETECTOR_IS_SECONDARY 1

in Deepstream-app how do change secondary classifier to the detector, that operates under the primary detector

create_common_elements (NvDsConfig * config, NvDsPipeline * pipeline,
    GstElement ** sink_elem, GstElement ** src_elem,
    bbox_generated_callback bbox_generated_post_analytics_cb)
{
  gboolean ret = FALSE;
  *sink_elem = *src_elem = NULL;

  if (config->primary_gie_config.enable) {
    if (config->num_secondary_gie_sub_bins > 0) {
      if (!create_secondary_gie_bin (config->num_secondary_gie_sub_bins,
              config->primary_gie_config.unique_id,
              config->secondary_gie_sub_bin_config,
              &pipeline->common_elements.secondary_gie_bin)) {
        goto done;
      }
      
       gst_bin_add (GST_BIN (pipeline->pipeline),
          pipeline->common_elements.secondary_gie_bin.bin);

      if (!*src_elem) {
        *src_elem = pipeline->common_elements.secondary_gie_bin.bin;
      }

      if (*sink_elem) {
        NVGSTDS_LINK_ELEMENT (pipeline->common_elements.secondary_gie_bin.bin,
            *sink_elem);
      }
      *sink_elem = pipeline->common_elements.secondary_gie_bin.bin;
    }
  }

Please refer to secondary_detector_config.txt in back-to-back-detector example to set secondary classifier to secondary detector.
U need to update the secondary classifier config group section as per above mentioned secondary detector config file.

[secondary-gie0]
enable=1
gpu-id=0
#model-engine-file=../../models/lp_eu_full/model_iter_45000.caffemodel_b30_fp16.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=30
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
config-file=lp_eu_full.txt

[tests]
file-loop=0

in lp_eu.txt

[property]
gpu-id=0
process-mode=2

it throws out a segmentation fault

mean-file, gie-unique-id(Default=0), offsets, process-mode (Default=1 i.e. primary),

custom-lib-path, network-mode(Default=0 i.e FP32)

Would you mind to share the lp_eu_full.txt which should refer secondary detector config file in comment #2?

[property]
gpu-id=0

model-file=odel_iter_45000.caffemodel
proto-file=net_tensorRT.proto
model-engine-file=model_iter_45000.caffemodel_b30_fp16.engine
labelfile-path=corresp.txt

batch-size=30
process-mode=2
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=2
interval=0
gie-unique-id=2
output-blob-names=[removed, this is working for primary, plugin is tested and working]
parse-bbox-func-name=NvDsInferParseCustomSSD
custom-lib-path=libnvdsinfer_custom_impl_ssd_lp.so
enable-dbscan=1
input-object-min-width=64
input-object-min-height=64

[class-attrs-all]
threshold=0.3
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.3
#minBoxes=3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

Do I need to add

operate-on-gie-id=1
gie-unique-id=2
operate-on-class-ids=1;2;3;4;5;6;7

If your case is to make the 2nd detector act as a primary(full-frame) detector, you do not need this.

Primary detector [car, bus] --> Secondary detector[ROI] --> Secondary classifier[0]–> Secondary classifier[1]

This is the pipeline

There is no reference of process-mode 2 in the reference documentation

https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/blob/master/back-to-back-detectors/secondary_detector_config.txt
https://docs.nvidia.com/metropolis/deepstream/4.0/dev-guide/#page/DeepStream_Development_Guide%2Fdeepstream_app_config.3.1.html

https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_details.02.01.html%23wwpID0E0IZ0HA
Gst Properties

process-mode

Infer Processing Mode

1=Primary Mode
2=Secondary Mode

Integer, 1 or 2

HI
about change the secondary classifier to the detector, as I know, classifier can not be used as detector

I went through deepstream_primary_gie_bin.c and deepstream_secondary_gie_bin.c

I think we can set it up as detector, need to create bbox function for secondary gie in deepstream_app.c[371]. I will keep you posted

/* Change this to 0 to make the 2nd detector act as a primary(full-frame) detector.

  • When set to 1, it will act as secondary(operates on primary detected objects). */
#define SECOND_DETECTOR_IS_SECONDARY 1
g_object_set (G_OBJECT (secondary_detector), "config-file-path", "secondary_detector_config.txt",
          "unique-id", SECONDARY_DETECTOR_UID, "process-mode", <b>SECOND_DETECTOR_IS_SECONDARY ? 2 : 1</b>, NULL);

what does this mean ? unique-id 2, process-mode 1 why do you need to double check ?

process-mode as explained before
unique-id, see this, https://docs.nvidia.com/metropolis/deepstream/plugin-manual/index.html#page/DeepStream_Plugin_Manual%2Fdeepstream_plugin_details.02.01.html%23wwpID0E0IZ0HA
Gst Properties part

Can you release an example using NVinfer as primary and secondary in the same program

you can refer to back to back detector
https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/blob/master/back-to-back-detectors/

Thanks, we resolved it, we are releasing it at Berlin event.