Wrong detect in sgie with yolo by using nvinerserver

• Hardware Platform GeForce RTX 2080 Ti
• DeepStream Version 6.2
• Issue Type questions
I tried to use two yolo modes to detect people and their head by using nvinferserver,one yolo model detect person as pgie, another one detect head as sgie, sgie based on pgie, i wana sgie detect cropped roi from pgie, pgie can detect correctly, but sgie cant detect correctly. When i chage the process_mode to 1 in sgie, sgie can detect correctly, so i think my preprocess in sgie may wrong.
Blew are my config files:
pgie:

infer_config {
  unique_id: 1
  gpu_ids: [0]
  max_batch_size: 1
  backend {
    inputs: [ {
      #name: "input"
      name: "images"
    }]
    outputs: [
      #{name: "boxes"},
      #{name: "scores"},
      #{name: "classes"}
      {name: "output"}
      #{name: "onnx::Sigmoid_401" },
      #{name: "onnx::Sigmoid_467" },
      #{name: "onnx::Sigmoid_533" }
    ]
    triton {
      model_name: "Meeting_Person"
      version: -1
      model_repo {
        root: "../../../../samples/triton_model_repo"
        strict_model_config: false
      }
    }
  }

  preprocess {
    network_format: IMAGE_FORMAT_RGB
    tensor_order: TENSOR_ORDER_LINEAR
    tensor_name: "images"
    maintain_aspect_ratio: 1
    symmetric_padding: 1
    normalize {
      scale_factor: 0.0039215697906911373
      #scale_factor: 1.0
      channel_offsets: [0,0,0]
    }
  }

  postprocess {
    labelfile_path: "./labels.txt"
    detection {
      num_detected_classes: 1
      #custom_parse_bbox_func: "NvDsInferParse"
      custom_parse_bbox_func: "NvDsInferParseCustomYoloV5"
      nms {
        confidence_threshold: 0.5
        iou_threshold: 0.3
        topk : 200
      }
    }
  }
  custom_lib {
    path: "./old_lib/libnvdsinfer_custom_impl_Yolo.so"
    #path: "./nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so"
  }

  extra {
    copy_input_to_host_buffers: false
    output_buffer_pool_size: 2
  }
}
input_control {
  process_mode: PROCESS_MODE_FULL_FRAME
  interval: 0
}

sgie:

infer_config {
  unique_id: 2
  gpu_ids: [0]
  max_batch_size: 1
  backend {
    inputs: [
      {name: "images"}
    ]
    outputs: [
      {name: "output"}
    ]
    triton {
      model_name: "Meeting_Head"
      version: -1
      model_repo {
        root: "../../../../samples/triton_model_repo"
        strict_model_config: false
      }
    }
  }

  preprocess {
    network_format: IMAGE_FORMAT_RGB
    tensor_order: TENSOR_ORDER_LINEAR
    #frame_scaling_hw: FRAME_SCALING_HW_DEFAULT
    #frame_scaling_filter: 1
    #tensor_name: "images"
    maintain_aspect_ratio: 1
    symmetric_padding: 1
    normalize {
      scale_factor: 0.0039215697906911373
      #scale_factor: 1.0
      channel_offsets: [0,0,0]
    }
  }

  postprocess {
    labelfile_path: "./labels_call.txt"
    detection {
      num_detected_classes: 1
      custom_parse_bbox_func: "NvDsInferParseCustomYoloV5"
      nms {
        confidence_threshold: 0.5
        iou_threshold: 0.3
        topk : 200
      }
    }
  }
  custom_lib {
    path: "./old_lib/libnvdsinfer_custom_impl_Yolo.so"
}

  extra {
    copy_input_to_host_buffers: false
    output_buffer_pool_size: 2
  }
}
input_control {
  #process_mode: PROCESS_MODE_FULL_FRAME
  interval: 0
  process_mode: PROCESS_MODE_CLIP_OBJECTS
  operate_on_gie_id: 1
  operate_on_class_ids: [0]
  #async_mode: true
}

Anyone helps?

sorry for the late reply. I am checking.

  1. In sgie 's configuration, is operate_on_class_ids correct?
  2. what do you mean about " sgie cant detect correctly"? no bbox or bbox is wrong?
  3. please refer to this similar sample faciallandmark. pgie detects face, sgie detects face’s facelandmark. this sample supports nvinferserver.

Thank you for your reply and sorry for my late reply.
Today i found my sgie model cant infer by the crop roi from pgie, it needs bigger bbox, so i use nvds_add_obj_meta_to_frame function add bigger box into frame, i changed the new class_id, but it still display wrong bbox.
Here are my answer to your reply:
1.yes, operate_on_class_ids is setted correectly.
2.the bbox is wrong, and they display unsteadily, the bbox pos and num is always changing
3. i have tried but failed and dont think i can learn from it for our sgie has different processing
Is there any way to find my mistake or other sample like my pro?
Thank you for your reply again and im looking forward to your new reply!

you can modify the bbox rect’s width and height to let bbox be bigger. please refer to deepstream_test4_app.c for how to access bbox rect.

Thank u for your reply, i will check this way later, the question is, i truely make the bbox bigger, but it still cant detect correctly. when i set the process_mode to full frame, it worked. I feel confused about this.

Question: How can i save the input picture before sgie’s infering? I want to check it after plugin’s prepross(crop and resize).

please refer this FAQ Dump the Inference outputs.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.