How to run YOLOv7 on Deepstream

yingliu · September 29, 2022, 1:50am

Check out this great tutorial by @vcmike on how to run YoloV7 on Deepstream:

bartdeboeck · October 26, 2022, 6:38pm

Levi_Pereira · November 7, 2022, 3:08pm

Deepstream/Triton Server working here.

github.com/WongKinYiu/yolov7

Yolov7 Deepstream/Triton Server

opened 11:26PM - 30 Sep 22 UTC

levipereira

Just sharing info. I was to be able to configure Deepstream/ Triton Server usin…g Custom Model Yolov7 How to configure [Triton Server](https://github.com/WongKinYiu/yolov7/tree/main/deploy/triton-inference-server) DeepStream Config: config_infer_primary_triton-server.txt ``` infer_config { unique_id: 1 gpu_ids: [0] max_batch_size: 20 backend { triton { model_name: "yolov7" version: -1 grpc { url: "0.0.0.0:8001" } } } preprocess { network_format: IMAGE_FORMAT_RGB tensor_order: TENSOR_ORDER_LINEAR maintain_aspect_ratio: 1 frame_scaling_hw: FRAME_SCALING_HW_DEFAULT frame_scaling_filter: 1 normalize { scale_factor: 0.0039215697906911373 } } postprocess { labelfile_path: "/apps/configs/labels.txt" detection { num_detected_classes: 8 custom_parse_bbox_func: "NvDsInferParseCustomEfficientNMS" nms { confidence_threshold: 0.5 topk: 300 iou_threshold: 0.45 } } } extra { copy_input_to_host_buffers: false } custom_lib { path : "/opt/nvidia/deepstream/deepstream-6.1/sources/libs/nvdsinfer_customparser/libnvds_infercustomparser.so" } } input_control { process_mode : PROCESS_MODE_FULL_FRAME interval : 0 } ``` To be able to map the model outputs from efficientNMS to the NVIDIA NvDsObjectMeta data structure you need this code which needs to be added/compiled in /opt/nvidia/deepstream/deepstream-6.1/sources/libs/nvdsinfer_customparser/nvdsinfer_custombboxparser.cpp ``` extern "C" bool NvDsInferParseCustomEfficientNMS (std::vector<NvDsInferLayerInfo> const &outputLayersInfo, NvDsInferNetworkInfo const &networkInfo, NvDsInferParseDetectionParams const &detectionParams, std::vector<NvDsInferObjectDetectionInfo> &objectList) { if(outputLayersInfo.size() != 4) { std::cerr << "Mismatch in the number of output buffers." << "Expected 4 output buffers, detected in the network :" << outputLayersInfo.size() << std::endl; return false; } const char* log_enable = std::getenv("ENABLE_DEBUG"); int* p_keep_count = (int *) outputLayersInfo[3].buffer; //int* p_keep_count = (int *) outputLayersInfo[0].buffer; //float* p_bboxes = (float *) outputLayersInfo[1].buffer; float* p_bboxes = (float *) outputLayersInfo[0].buffer; //NvDsInferDims inferDims_p_bboxes = outputLayersInfo[1].inferDims; NvDsInferDims inferDims_p_bboxes = outputLayersInfo[0].inferDims; int numElements_p_bboxes=inferDims_p_bboxes.numElements; float* p_scores = (float *) outputLayersInfo[2].buffer; //unsigned int* p_classes = (unsigned int *) outputLayersInfo[3].buffer; unsigned int* p_classes = (unsigned int *) outputLayersInfo[1].buffer; const float threshold = detectionParams.perClassThreshold[0]; float max_bbox=0; for (int i=0; i < numElements_p_bboxes; i++) { if ( max_bbox < p_bboxes[i] ) max_bbox=p_bboxes[i]; } if (p_keep_count[0] > 0) { assert (!(max_bbox < 2.0)); for (int i = 0; i < p_keep_count[0]; i++) { if ( p_scores[i] < threshold) continue; assert((unsigned int) p_classes[i] < detectionParams.numClassesConfigured); NvDsInferObjectDetectionInfo object; object.classId = (int) p_classes[i]; object.detectionConfidence = p_scores[i]; object.left=p_bboxes[4*i]; object.top=p_bboxes[4*i+1]; object.width=(p_bboxes[4*i+2] - object.left); object.height= (p_bboxes[4*i+3] - object.top); if(log_enable != NULL && std::stoi(log_enable)) { std::cout << "label/conf/ x/y w/h -- " << p_classes[i] << " " << p_scores[i] << " " << object.left << " " << object.top << " " << object.width << " "<< object.height << " " << std::endl; } object.left=CLIP(object.left, 0, networkInfo.width - 1); object.top=CLIP(object.top, 0, networkInfo.height - 1); object.width=CLIP(object.width, 0, networkInfo.width - 1); object.height=CLIP(object.height, 0, networkInfo.height - 1); objectList.push_back(object); } } return true; CHECK_CUSTOM_PARSE_FUNC_PROTOTYPE(NvDsInferParseCustomEfficientNMS); }

mchi · November 18, 2022, 8:48am

YoloV7 + DeepStream sample with CUDA accelerated parser
YoloV7+DeepStream perf vs YoloV7+trtexec perf
YoloV7 QAT