Same methods as you suggested…
The peopleNet is based on detectnet_v2 network from TLT.
But yolo_v3 network is different from detectnet_v2 network.
So, it cannot leverage the inference code directly.
More info including output and postprocessing can be found in GitHub - NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream
Hi @Morganh,
Thanks… I could see information about output… And i am getting 4 lists as output which is expected from yolo… But i couldn’t see how to post process it. I am using python…
For previous error “[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin BatchTilePlugin_TRT version 1”, please add below code. It will fix the error.
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')
For postprocessing, please refer to https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/nvdsinfer_customparser_yolov3_tlt/nvdsinfer_custombboxparser_yolov3_tlt.cpp
Hi @Morganh
Thanks for sharing this commands. Now I get 4 lists as output from inference in this format
num_detections
nmsed_boxes
nmsed_scores
nmsed_classes
Here i am getting all the results are zero.
But when i run the same model with tlt-infer yolo inside tlt container ,i got the detections perfectly.
Please modify
binding_to_type = {
“input_1”: np.float32,
“output_bbox/BiasAdd”: np.float32,
“output_cov/Sigmoid”: np.float32,
}
to
binding_to_type = {
“Input”: np.float32,
“BatchedNMS”: np.int32,
“BatchedNMS_1”: np.float32,
“BatchedNMS_2”: np.float32,
“BatchedNMS_3”: np.float32
}
Also, please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh
Refer to postprocess way in https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/nvdsinfer_customparser_yolov3_tlt/nvdsinfer_custombboxparser_yolov3_tlt.cpp
Thanks a lot Morganh. Problem solved.Now i can do Inference with yolo trt engine.
@jothi.ramasubramanian hii
can you please share your script in which you are loading Yolo engine model file and doing predictions as I am stuck with the same issue.
I will be grateful to you . thanks
@jazeel.jk hii
using this 2 script I can load my Yolo engine model and run predictions on input image right ?
because I am also looking for the same script.
one question Yolo has this output binding layer called BatchedNMS.
but in your try_loader.py you are using same bindings as what in detectnet_v2.py. can you help me in this confusion?
I will be great full to you.
thanks
yes @senbhaskar26 … Need to edit it. That one won’t work for yolo. Need to edit binding_to_type… And need to add some post processing steps… Please refer to this comment, Inferring detectnet_v2 .trt model in python - #46 by Morganh
@jazeel.jk binding I already did but not able to get an idea regarding post processing steps. what it requires ?
thanks
please refer to preprocess way in Discrepancy between results from tlt-infer and trt engine - #8 by Morganh
Refer to postprocess way in https://github.com/NVIDIA-AI-IOT/deepstream_tlt_apps/blob/master/post_processor/nvdsinfer_custombboxparser_tlt.cpp
Hi @Morganh
As you suggest I have done the preprocessing with the following steps
1)read RGB image with CV2
2)Resized the image with the the input size of model
3)np.array(inf_img).astype(np.float32)
4) inf_img.transpose((2, 0, 1)
The same trt engine giving good results in Deepstream and tlt-infer. But when i infer with script i am getting false positives. Can you help me pls?
@jothi.ramasubramanian
Please double check your preprocessing and postprocessing in your yolo_v3 standalone inference script.
@Morganh
This is my preprocessing steps
def process_image(self,arr):
image_resized=cv2.resize(arr,(self.model_w, self.model_h))
img_np = image_resized.astype(np.float32)
# HWC -> CHW
img_np = img_np.transpose((2, 0, 1))
img_np = img_np.ravel()
return img_np
This is Postprocessing steps
def _nms_boxes(self, boxes, box_confidences):
x_coord = boxes[:, 0]
y_coord = boxes[:, 1]
width = boxes[:, 2]
height = boxes[:, 3]
areas = width * height
ordered = box_confidences.argsort()[::-1]
keep = list()
while ordered.size > 0:
# Index of the current element:
i = ordered[0]
keep.append(i)
xx1 = np.maximum(x_coord[i], x_coord[ordered[1:]])
yy1 = np.maximum(y_coord[i], y_coord[ordered[1:]])
xx2 = np.minimum(x_coord[i] + width[i], x_coord[ordered[1:]] + width[ordered[1:]])
yy2 = np.minimum(y_coord[i] + height[i], y_coord[ordered[1:]] + height[ordered[1:]])
width1 = np.maximum(0.0, xx2 - xx1 + 1)
height1 = np.maximum(0.0, yy2 - yy1 + 1)
intersection = width1 * height1
union = (areas[i] + areas[ordered[1:]] - intersection)
# Compute the Intersection over Union (IoU) score:
iou = intersection / union
# The goal of the NMS algorithm is to reduce the number of adjacent bounding-box
# candidates to a minimum. In this step, we keep only those elements whose overlap
# with the current bounding box is lower than the threshold:
indexes = np.where(iou <= self.nms_threshold)[0]
ordered = ordered[indexes + 1]
keep = np.array(keep)
return keep
def postprocess(self, outputs, wh_format=True):
"""
Postprocesses the inference output
Args:
outputs (list of float): inference output
min_confidence (float): min confidence to accept detection
analysis_classes (list of int): indices of the classes to consider
Returns: list of list tuple: each element is a two list tuple (x, y) representing the corners of a bb
"""
p_keep_count = outputs[0]
p_bboxes = outputs[1]
p_scores = outputs[2]
p_classes = outputs[3]
analysis_classes = list(range(self.NUM_CLASSES))
threshold = self.min_confidence
p_bboxes = np.array_split(p_bboxes,len(p_bboxes)/4)
bbs = []
class_ids = []
scores = []
x_scale = self.img_shape[1] / self.model_w
y_scale = self.img_shape[0] / self.model_h
for i in range(p_keep_count[0]):
assert(p_classes[i] < len(analysis_classes))
if p_scores[i]>threshold:
x1 = int(np.round(p_bboxes[i][0]*x_scale))
y1 = int(np.round(p_bboxes[i][1]*y_scale))
x2 = int(np.round(p_bboxes[i][2]*x_scale))
y2 = int(np.round(p_bboxes[i][3]*y_scale))
bbs.append([x1,y1,x2,y2])
class_ids.append(p_classes[i])
scores.append(p_scores[i])
#print(class_ids)
bbs = np.asarray(bbs)
class_ids = np.asarray(class_ids)
scores = np.asarray(scores)
nms_boxes, nms_categories, nscores = list(), list(), list()
for category in set(class_ids):
idxs = np.where(class_ids == category)
box = bbs[idxs]
category = class_ids[idxs]
confidence = scores[idxs]
keep = self._nms_boxes(box, confidence)
nms_boxes.append(box[keep])
nms_categories.append(category[keep])
nscores.append(confidence[keep])
if len(nms_boxes)==0:
return [],[],[]
return nms_boxes, nms_categories, nscores
can you ,let me know if any mistakes i need to correct here?
@Morganh
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(TRT_LOGGER, ’ ')
Where is the modification location?
It locates at the code written by above end user.