Same methods as you suggested…

The peopleNet is based on detectnet_v2 network from TLT.

But yolo_v3 network is different from detectnet_v2 network.

So, it cannot leverage the inference code directly.

More info including output and postprocessing can be found in GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream

Hi @Morganh,

Thanks… I could see information about output… And i am getting 4 lists as output which is expected from yolo… But i couldn’t see how to post process it. I am using python…

For previous error “[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin BatchTilePlugin_TRT version 1”, please add below code. It will fix the error.

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

For postprocessing, please refer to https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/nvdsinfer_customparser_yolov3_tlt/nvdsinfer_custombboxparser_yolov3_tlt.cpp

Hi @Morganh

Thanks for sharing this commands. Now I get 4 lists as output from inference in this format

num_detections

nmsed_boxes

nmsed_scores

nmsed_classes

Here i am getting all the results are zero.

But when i run the same model with tlt-infer yolo inside tlt container ,i got the detections perfectly.

Please modify

binding_to_type = {

“input_1”: np.float32,

“output_bbox/BiasAdd”: np.float32,

“output_cov/Sigmoid”: np.float32,

}

to

binding_to_type = {

“Input”: np.float32,

“BatchedNMS”: np.int32,

“BatchedNMS_1”: np.float32,

“BatchedNMS_2”: np.float32,

“BatchedNMS_3”: np.float32

}

Also, please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/nvdsinfer_customparser_yolov3_tlt/nvdsinfer_custombboxparser_yolov3_tlt.cpp

Thanks a lot Morganh. Problem solved.Now i can do Inference with yolo trt engine.

@jothi.ramasubramanian hii

can you please share your script in which you are loading Yolo engine model file and doing predictions as I am stuck with the same issue.

I will be grateful to you . thanks

@jazeel.jk hii

using this 2 script I can load my Yolo engine model and run predictions on input image right ?

because I am also looking for the same script.

one question Yolo has this output binding layer called BatchedNMS.

but in your try_loader.py you are using same bindings as what in detectnet_v2.py. can you help me in this confusion?

I will be great full to you.

thanks

yes @senbhaskar26 … Need to edit it. That one won’t work for yolo. Need to edit binding_to_type… And need to add some post processing steps… Please refer to this comment, Inferring detectnet_v2 .trt model in python - #46 by Morganh

@jazeel.jk binding I already did but not able to get an idea regarding post processing steps. what it requires ?

thanks

please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh

Refer to postprocess way in https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/post_processor/nvdsinfer_custombboxparser_tlt.cpp

Hi @Morganh

As you suggest I have done the preprocessing with the following steps

1)read RGB image with CV2

2)Resized the image with the the input size of model

3)np.array(inf_img).astype(np.float32)

4) inf_img.transpose((2, 0, 1)

The same trt engine giving good results in Deepstream and tlt-infer. But when i infer with script i am getting false positives. Can you help me pls?

@jothi.ramasubramanian

Please double check your preprocessing and postprocessing in your yolo_v3 standalone inference script.

@Morganh

This is my preprocessing steps

```
def process_image(self,arr):
image_resized=cv2.resize(arr,(self.model_w, self.model_h))
img_np = image_resized.astype(np.float32)
# HWC -> CHW
img_np = img_np.transpose((2, 0, 1))
img_np = img_np.ravel()
return img_np
```

This is Postprocessing steps

```
def _nms_boxes(self, boxes, box_confidences):
x_coord = boxes[:, 0]
y_coord = boxes[:, 1]
width = boxes[:, 2]
height = boxes[:, 3]
areas = width * height
ordered = box_confidences.argsort()[::-1]
keep = list()
while ordered.size > 0:
# Index of the current element:
i = ordered[0]
keep.append(i)
xx1 = np.maximum(x_coord[i], x_coord[ordered[1:]])
yy1 = np.maximum(y_coord[i], y_coord[ordered[1:]])
xx2 = np.minimum(x_coord[i] + width[i], x_coord[ordered[1:]] + width[ordered[1:]])
yy2 = np.minimum(y_coord[i] + height[i], y_coord[ordered[1:]] + height[ordered[1:]])
width1 = np.maximum(0.0, xx2 - xx1 + 1)
height1 = np.maximum(0.0, yy2 - yy1 + 1)
intersection = width1 * height1
union = (areas[i] + areas[ordered[1:]] - intersection)
# Compute the Intersection over Union (IoU) score:
iou = intersection / union
# The goal of the NMS algorithm is to reduce the number of adjacent bounding-box
# candidates to a minimum. In this step, we keep only those elements whose overlap
# with the current bounding box is lower than the threshold:
indexes = np.where(iou <= self.nms_threshold)[0]
ordered = ordered[indexes + 1]
keep = np.array(keep)
return keep
def postprocess(self, outputs, wh_format=True):
"""
Postprocesses the inference output
Args:
outputs (list of float): inference output
min_confidence (float): min confidence to accept detection
analysis_classes (list of int): indices of the classes to consider
Returns: list of list tuple: each element is a two list tuple (x, y) representing the corners of a bb
"""
p_keep_count = outputs[0]
p_bboxes = outputs[1]
p_scores = outputs[2]
p_classes = outputs[3]
analysis_classes = list(range(self.NUM_CLASSES))
threshold = self.min_confidence
p_bboxes = np.array_split(p_bboxes,len(p_bboxes)/4)
bbs = []
class_ids = []
scores = []
x_scale = self.img_shape[1] / self.model_w
y_scale = self.img_shape[0] / self.model_h
for i in range(p_keep_count[0]):
assert(p_classes[i] < len(analysis_classes))
if p_scores[i]>threshold:
x1 = int(np.round(p_bboxes[i][0]*x_scale))
y1 = int(np.round(p_bboxes[i][1]*y_scale))
x2 = int(np.round(p_bboxes[i][2]*x_scale))
y2 = int(np.round(p_bboxes[i][3]*y_scale))
bbs.append([x1,y1,x2,y2])
class_ids.append(p_classes[i])
scores.append(p_scores[i])
#print(class_ids)
bbs = np.asarray(bbs)
class_ids = np.asarray(class_ids)
scores = np.asarray(scores)
nms_boxes, nms_categories, nscores = list(), list(), list()
for category in set(class_ids):
idxs = np.where(class_ids == category)
box = bbs[idxs]
category = class_ids[idxs]
confidence = scores[idxs]
keep = self._nms_boxes(box, confidence)
nms_boxes.append(box[keep])
nms_categories.append(category[keep])
nscores.append(confidence[keep])
if len(nms_boxes)==0:
return [],[],[]
return nms_boxes, nms_categories, nscores
```

can you ,let me know if any mistakes i need to correct here?

@Morganh

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)

trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')

Where is the modification location?

It locates at the code written by above end user.