Inferring detectnet_v2 .trt model in python

Can you share your standalone inference script here?

trt_loader.py (9.0 KB) main.py (2.2 KB)

main.py is the one i run… it will load trt_loader… works fine for detectnet_v2

Same methods as you suggested…

The peopleNet is based on detectnet_v2 network from TLT.
But yolo_v3 network is different from detectnet_v2 network.
So, it cannot leverage the inference code directly.

More info including output and postprocessing can be found in https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/

Hi @Morganh,
Thanks… I could see information about output… And i am getting 4 lists as output which is expected from yolo… But i couldn’t see how to post process it. I am using python…

For previous error “[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin BatchTilePlugin_TRT version 1”, please add below code. It will fix the error.

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

For postprocessing, please refer to https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/nvdsinfer_customparser_yolov3_tlt/nvdsinfer_custombboxparser_yolov3_tlt.cpp

Hi @Morganh
Thanks for sharing this commands. Now I get 4 lists as output from inference in this format
num_detections
nmsed_boxes
nmsed_scores
nmsed_classes
Here i am getting all the results are zero.
But when i run the same model with tlt-infer yolo inside tlt container ,i got the detections perfectly.

Please modify

binding_to_type = {
“input_1”: np.float32,
“output_bbox/BiasAdd”: np.float32,
“output_cov/Sigmoid”: np.float32,
}

to

binding_to_type = {
“Input”: np.float32,
“BatchedNMS”: np.int32,
“BatchedNMS_1”: np.float32,
“BatchedNMS_2”: np.float32,
“BatchedNMS_3”: np.float32
}

Also, please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in deepstream_tlt_apps/nvdsinfer_custombboxparser_yolov3_tlt.cpp at master · NVIDIA-AI-IOT/deepstream_tlt_apps · GitHub

2 Likes

Thanks a lot Morganh. Problem solved.Now i can do Inference with yolo trt engine.

1 Like

@jothi.ramasubramanian hii
can you please share your script in which you are loading Yolo engine model file and doing predictions as I am stuck with the same issue.
I will be grateful to you . thanks

@jazeel.jk hii
using this 2 script I can load my Yolo engine model and run predictions on input image right ?
because I am also looking for the same script.
one question Yolo has this output binding layer called BatchedNMS.
but in your try_loader.py you are using same bindings as what in detectnet_v2.py. can you help me in this confusion?
I will be great full to you.
thanks

yes @senbhaskar26 … Need to edit it. That one won’t work for yolo. Need to edit binding_to_type… And need to add some post processing steps… Please refer to this comment, Inferring detectnet_v2 .trt model in python - #46 by Morganh

@jazeel.jk binding I already did but not able to get an idea regarding post processing steps. what it requires ?
thanks

please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in deepstream_tlt_apps/nvdsinfer_custombboxparser_tlt.cpp at master · NVIDIA-AI-IOT/deepstream_tlt_apps · GitHub

1 Like

@jazeel.jk thanks will look into it .

1 Like

Hi @Morganh
As you suggest I have done the preprocessing with the following steps
1)read RGB image with CV2
2)Resized the image with the the input size of model
3)np.array(inf_img).astype(np.float32)
4) inf_img.transpose((2, 0, 1)

The same trt engine giving good results in Deepstream and tlt-infer. But when i infer with script i am getting false positives. Can you help me pls?

@jothi.ramasubramanian
Please double check your preprocessing and postprocessing in your yolo_v3 standalone inference script.

@Morganh
This is my preprocessing steps

def process_image(self,arr):
    image_resized=cv2.resize(arr,(self.model_w, self.model_h))
    img_np = image_resized.astype(np.float32)
    # HWC -> CHW
    img_np = img_np.transpose((2, 0, 1))
    img_np = img_np.ravel()
    return img_np

This is Postprocessing steps

 def _nms_boxes(self, boxes, box_confidences):

    x_coord = boxes[:, 0]
    y_coord = boxes[:, 1]
    width = boxes[:, 2]
    height = boxes[:, 3]

    areas = width * height
    ordered = box_confidences.argsort()[::-1]

    keep = list()
    while ordered.size > 0:
        # Index of the current element:
        i = ordered[0]
        keep.append(i)
        xx1 = np.maximum(x_coord[i], x_coord[ordered[1:]])
        yy1 = np.maximum(y_coord[i], y_coord[ordered[1:]])
        xx2 = np.minimum(x_coord[i] + width[i], x_coord[ordered[1:]] + width[ordered[1:]])
        yy2 = np.minimum(y_coord[i] + height[i], y_coord[ordered[1:]] + height[ordered[1:]])

        width1 = np.maximum(0.0, xx2 - xx1 + 1)
        height1 = np.maximum(0.0, yy2 - yy1 + 1)
        intersection = width1 * height1
        union = (areas[i] + areas[ordered[1:]] - intersection)

        # Compute the Intersection over Union (IoU) score:
        iou = intersection / union

        # The goal of the NMS algorithm is to reduce the number of adjacent bounding-box
        # candidates to a minimum. In this step, we keep only those elements whose overlap
        # with the current bounding box is lower than the threshold:
        indexes = np.where(iou <= self.nms_threshold)[0]
        ordered = ordered[indexes + 1]

    keep = np.array(keep)
    return keep


def postprocess(self, outputs, wh_format=True):
    """
    Postprocesses the inference output
    Args:
        outputs (list of float): inference output
        min_confidence (float): min confidence to accept detection
        analysis_classes (list of int): indices of the classes to consider

    Returns: list of list tuple: each element is a two list tuple (x, y) representing the corners of a bb
    """
    
    
    p_keep_count = outputs[0]
    p_bboxes = outputs[1]
    p_scores = outputs[2]
    p_classes = outputs[3]
    analysis_classes = list(range(self.NUM_CLASSES))
    threshold = self.min_confidence
    p_bboxes = np.array_split(p_bboxes,len(p_bboxes)/4)
    bbs = []
    class_ids = []
    scores = []
    
    x_scale = self.img_shape[1] / self.model_w
    y_scale = self.img_shape[0] / self.model_h
    
    for i in range(p_keep_count[0]):
        assert(p_classes[i] < len(analysis_classes))
        if p_scores[i]>threshold:
            
            x1 = int(np.round(p_bboxes[i][0]*x_scale))
            y1 = int(np.round(p_bboxes[i][1]*y_scale))
            x2 = int(np.round(p_bboxes[i][2]*x_scale))
            y2 = int(np.round(p_bboxes[i][3]*y_scale))
            
            bbs.append([x1,y1,x2,y2])
            class_ids.append(p_classes[i])
            scores.append(p_scores[i])
    #print(class_ids)    
    bbs = np.asarray(bbs)
    class_ids = np.asarray(class_ids)
    scores = np.asarray(scores)
            
    nms_boxes, nms_categories, nscores = list(), list(), list()
    for category in set(class_ids):
        idxs = np.where(class_ids == category)
        box = bbs[idxs]
        category = class_ids[idxs]
        confidence = scores[idxs]

        keep = self._nms_boxes(box, confidence)
        nms_boxes.append(box[keep])
        nms_categories.append(category[keep])
        nscores.append(confidence[keep])
    if len(nms_boxes)==0:
        return [],[],[]
    
    return nms_boxes, nms_categories, nscores

can you ,let me know if any mistakes i need to correct here?

Hi jothi.ramasubramanian,

Please help to open a new topic if it’s still an issue. Thanks

1 Like

@Morganh
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

Where is the modification location?