Running Yolov5 Model in triton inference server with GRPC mode to work with Deepstream

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.1.1
• JetPack Version (valid for Jetson only) 5.0.2
• TensorRT Version 8.4
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) LD_PRELOAD=/home/user/Neo/deepstream_python_apps/apps/deepstream-occupancy/person-head-detection/nvdsinfer_custom_impl_Yolo/ ./tritonserver --model-repository=/home/user/triton_model_repo --backend-directory=/opt/nvidia/deepstream/deepstream/lib/triton_backends/ --allow-grpc=1

Hi i am trying to triton inference server in grpc mode to access model in deepstream using nvinferserver.
I tried the above command to run yolov5 model and it was running perfectly, but whenever the detections appears the deepstream pipeline giving error and stopping.

I am getting the below only error even with the debug set to 3, i am running python code here.

0:00:23.925216093 2033209 0xfffee0079240 DEBUG v4l2bufferpool gstv4l2bufferpool.c:2077:gst_v4l2_buffer_pool_process:<nvv4l2decoder0:pool:sink> process buffer 0xfffec80151a8Segmentation fault (core dumped)

Please find the code and model, custom parser (20.0 KB)
config_nvdsanalytics.txt (3.6 KB)
yolov5_pgie_nvinferserver_grpc_config.txt (2.0 KB)
crowdhuman_yolov5m.cfg (9.2 KB)
crowd_labels.txt (11 Bytes) (822.9 KB) (51.1 MB)

In nvdsparsebbox_Yolo.cpp of nvdsinfer_custom_impl_Yolo, please change this code
const NvDsInferLayerInfo& counts = outputLayersInfo[0];
const NvDsInferLayerInfo& boxes = outputLayersInfo[1];
const NvDsInferLayerInfo& scores = outputLayersInfo[2];
const NvDsInferLayerInfo& classes = outputLayersInfo[3];
const NvDsInferLayerInfo& counts = outputLayersInfo[3];
const NvDsInferLayerInfo& boxes = outputLayersInfo[0];
const NvDsInferLayerInfo& scores = outputLayersInfo[2];
const NvDsInferLayerInfo& classes = outputLayersInfo[1];
then rebuild.

1 Like

But the same code works with deepstream nvinfer and nvinferserver without running triton as separate service.
and i am not able to find the line const NvDsInferLayerInfo& counts = outputLayersInfo[3] in this.

in grpc mode. output layers sequence maybe will change. please refer to this code to get the corresponding pointer.

1 Like

okay will try this

@fanzh this worked, i was using yolov5m conversion from darknet weights to tensorrt engine so it was not working.
Now i used latest code from the custom parser repo which uses onnx to trt engine.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.