Inferring detectnet_v2 .trt model in python

Same methods as you suggested…

The peopleNet is based on detectnet_v2 network from TLT.
But yolo_v3 network is different from detectnet_v2 network.
So, it cannot leverage the inference code directly.

More info including output and postprocessing can be found in GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream

Hi @Morganh,
Thanks… I could see information about output… And i am getting 4 lists as output which is expected from yolo… But i couldn’t see how to post process it. I am using python…

For previous error “[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin BatchTilePlugin_TRT version 1”, please add below code. It will fix the error.

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

For postprocessing, please refer to

Hi @Morganh
Thanks for sharing this commands. Now I get 4 lists as output from inference in this format
Here i am getting all the results are zero.
But when i run the same model with tlt-infer yolo inside tlt container ,i got the detections perfectly.

Please modify

binding_to_type = {
“input_1”: np.float32,
“output_bbox/BiasAdd”: np.float32,
“output_cov/Sigmoid”: np.float32,


binding_to_type = {
“Input”: np.float32,
“BatchedNMS”: np.int32,
“BatchedNMS_1”: np.float32,
“BatchedNMS_2”: np.float32,
“BatchedNMS_3”: np.float32

Also, please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in


Thanks a lot Morganh. Problem solved.Now i can do Inference with yolo trt engine.

1 Like

@jothi.ramasubramanian hii
can you please share your script in which you are loading Yolo engine model file and doing predictions as I am stuck with the same issue.
I will be grateful to you . thanks

@jazeel.jk hii
using this 2 script I can load my Yolo engine model and run predictions on input image right ?
because I am also looking for the same script.
one question Yolo has this output binding layer called BatchedNMS.
but in your you are using same bindings as what in can you help me in this confusion?
I will be great full to you.

yes @senbhaskar26 … Need to edit it. That one won’t work for yolo. Need to edit binding_to_type… And need to add some post processing steps… Please refer to this comment, Inferring detectnet_v2 .trt model in python - #46 by Morganh

@jazeel.jk binding I already did but not able to get an idea regarding post processing steps. what it requires ?

please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in

1 Like

@jazeel.jk thanks will look into it .

1 Like

Hi @Morganh
As you suggest I have done the preprocessing with the following steps
1)read RGB image with CV2
2)Resized the image with the the input size of model
4) inf_img.transpose((2, 0, 1)

The same trt engine giving good results in Deepstream and tlt-infer. But when i infer with script i am getting false positives. Can you help me pls?

Please double check your preprocessing and postprocessing in your yolo_v3 standalone inference script.

This is my preprocessing steps

def process_image(self,arr):
    image_resized=cv2.resize(arr,(self.model_w, self.model_h))
    img_np = image_resized.astype(np.float32)
    # HWC -> CHW
    img_np = img_np.transpose((2, 0, 1))
    img_np = img_np.ravel()
    return img_np

This is Postprocessing steps

 def _nms_boxes(self, boxes, box_confidences):

    x_coord = boxes[:, 0]
    y_coord = boxes[:, 1]
    width = boxes[:, 2]
    height = boxes[:, 3]

    areas = width * height
    ordered = box_confidences.argsort()[::-1]

    keep = list()
    while ordered.size > 0:
        # Index of the current element:
        i = ordered[0]
        xx1 = np.maximum(x_coord[i], x_coord[ordered[1:]])
        yy1 = np.maximum(y_coord[i], y_coord[ordered[1:]])
        xx2 = np.minimum(x_coord[i] + width[i], x_coord[ordered[1:]] + width[ordered[1:]])
        yy2 = np.minimum(y_coord[i] + height[i], y_coord[ordered[1:]] + height[ordered[1:]])

        width1 = np.maximum(0.0, xx2 - xx1 + 1)
        height1 = np.maximum(0.0, yy2 - yy1 + 1)
        intersection = width1 * height1
        union = (areas[i] + areas[ordered[1:]] - intersection)

        # Compute the Intersection over Union (IoU) score:
        iou = intersection / union

        # The goal of the NMS algorithm is to reduce the number of adjacent bounding-box
        # candidates to a minimum. In this step, we keep only those elements whose overlap
        # with the current bounding box is lower than the threshold:
        indexes = np.where(iou <= self.nms_threshold)[0]
        ordered = ordered[indexes + 1]

    keep = np.array(keep)
    return keep

def postprocess(self, outputs, wh_format=True):
    Postprocesses the inference output
        outputs (list of float): inference output
        min_confidence (float): min confidence to accept detection
        analysis_classes (list of int): indices of the classes to consider

    Returns: list of list tuple: each element is a two list tuple (x, y) representing the corners of a bb
    p_keep_count = outputs[0]
    p_bboxes = outputs[1]
    p_scores = outputs[2]
    p_classes = outputs[3]
    analysis_classes = list(range(self.NUM_CLASSES))
    threshold = self.min_confidence
    p_bboxes = np.array_split(p_bboxes,len(p_bboxes)/4)
    bbs = []
    class_ids = []
    scores = []
    x_scale = self.img_shape[1] / self.model_w
    y_scale = self.img_shape[0] / self.model_h
    for i in range(p_keep_count[0]):
        assert(p_classes[i] < len(analysis_classes))
        if p_scores[i]>threshold:
            x1 = int(np.round(p_bboxes[i][0]*x_scale))
            y1 = int(np.round(p_bboxes[i][1]*y_scale))
            x2 = int(np.round(p_bboxes[i][2]*x_scale))
            y2 = int(np.round(p_bboxes[i][3]*y_scale))
    bbs = np.asarray(bbs)
    class_ids = np.asarray(class_ids)
    scores = np.asarray(scores)
    nms_boxes, nms_categories, nscores = list(), list(), list()
    for category in set(class_ids):
        idxs = np.where(class_ids == category)
        box = bbs[idxs]
        category = class_ids[idxs]
        confidence = scores[idxs]

        keep = self._nms_boxes(box, confidence)
    if len(nms_boxes)==0:
        return [],[],[]
    return nms_boxes, nms_categories, nscores

can you ,let me know if any mistakes i need to correct here?

Hi jothi.ramasubramanian,

Please help to open a new topic if it’s still an issue. Thanks

1 Like

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

Where is the modification location?

It locates at the code written by above end user.