YOLOV4-DS-TRITON/got an error about input unmatch

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Hi!
Now I will deploy the model to my Jetson TX2 by deepstream through triton inference server.
I am using a Jetson TX2 with deep-stream 5-0.

  1. got engine file
    followed the steps of GitHub - Tianxiaomo/pytorch-YOLOv4: PyTorch ,ONNX and TensorRT implementation of YOLOv4,
python demo_darknet2onnx.py /workspace/pytorch-YOLOv4/cfg/yolov4.cfg /workspace/pytorch-YOLOv4/yolov4.weights /workspace/pytorch-YOLOv4/data/dog.jpg 1

/usr/src/tensorrt/bin/trtexec --onnx=yolov4_1_3_608_608_static.onnx --explicitBatch --saveEngine=yolov4_1_3_608_608_static.engine --workspace=2048 --fp16

  1. config triton info
    source1_primary_yoloV4_original.txt (3.1 KB)
    config_infer_primary_yoloV4_original.txt (2.4 KB)
    config.pbtxt (568 Bytes)

  2. NvDsInferParseCustomYoloV4

git clone https://github.com/NVIDIA-AI-IOT/yolov4_deepstream.git
/yolov4_deepstream/deepstream_yolov4/nvdsinfer_custom_impl_Yolo
  1. got error as following:

Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
** INFO: <bus_callback:167>: Pipeline running

ERROR: TRTIS: TrtServerRequest failed to create inference request providerV2, trtis_err_str:INVALID_ARG, err_msg:unexpected shape for input ‘input’ for model ‘yolov4_original’. Expected [1,3,608,608], got [3,608,608]
ERROR: TRTIS failed to create request for model: yolov4_original version:-1
ERROR: TRT-IS run failed to create request for model: yolov4_original
ERROR: TRT-IS failed to run inference on model yolov4_original, nvinfer error:NVDSINFER_TRTIS_ERROR
0:00:14.451512879 20883 0x7e886e5530 WARN nvinferserver gstnvinferserver.cpp:519:gst_nvinfer_server_push_buffer:<primary_gie> error: inference failed with unique-id:1
ERROR from primary_gie: inference failed with unique-id:1
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinferserver/gstnvinferserver.cpp(519): gst_nvinfer_server_push_buffer (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInferServer:primary_gie
ERROR: TRTIS: TrtServerRequest failed to create inference request providerV2, trtis_err_str:INVALID_ARG, err_msg:unexpected shape for input ‘input’ for model ‘yolov4_original’. Expected [1,3,608,608], got [3,608,608]
ERROR: TRTIS failed to create request for model: yolov4_original version:-1
ERROR: TRT-IS run failed to create request for model: yolov4_original
ERROR: TRT-IS failed to run inference on model yolov4_original, nvinfer error:NVDSINFER_TRTIS_ERROR
0:00:14.451512879 20883 0x7e886e5530 WARN nvinferserver gstnvinferserver.cpp:519:gst_nvinfer_server_push_buffer:<primary_gie> error: inference failed with unique-id:1
ERROR from primary_gie: inference failed with unique-id:1
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinferserver/gstnvinferserver.cpp(519): gst_nvinfer_server_push_buffer (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInferServer:primary_gie
ERROR: TRTIS: TrtServerRequest failed to create inference request providerV2, trtis_err_str:INVALID_ARG, err_msg:unexpected shape for input ‘input’ for model ‘yolov4_original’. Expected [1,3,608,608], got [3,608,608]
ERROR: TRTIS failed to create request for model: yolov4_original version:-1
ERROR: TRT-IS run failed to create request for model: yolov4_original
ERROR: TRT-IS failed to run inference on model yolov4_original, nvinfer error:NVDSINFER_TRTIS_ERROR
0:00:14.453875925 20883 0x7e886e5530 WARN nvinferserver gstnvinferserver.cpp:519:gst_nvinfer_server_push_buffer:<primary_gie> error: inference failed with unique-id:1
ERROR from primary_gie: inference failed with unique-id:1
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinferserver/gstnvinferserver.cpp(519): gst_nvinfer_server_push_buffer (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInferServer:primary_gie
ERROR: TRTIS: TrtServerRequest failed to create inference request providerV2, trtis_err_str:INVALID_ARG, err_msg:unexpected shape for input ‘input’ for model ‘yolov4_original’. Expected [1,3,608,608], got [3,608,608]
ERROR: TRTIS failed to create request for model: yolov4_original version:-1
ERROR: TRT-IS run failed to create request for model: yolov4_original

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

5

Hi,
thank you for your response. I had not finished my issue yet at the time you responded. Sorry about that

6

Hi @wanty7788,

We recommend you to post your concern on DeepStream forum. You may get better help here.

Thank you.

8

Soga~ Thank you

1 Like