Error in converting caffe xilinx yolo v3 model to Tensorrt

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description:
I am trying to convert a yolo v3 model built on caffe -xilinx to tensorrt using trtexec ,
I have used this command for conversion ,

caffe_bvlc/TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1/TensorRT-7.2.3.4/bin/trtexec --deploy=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/test.prototxt --model=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/trainval.caffemodel --output=layer117-conv --batch=64 --saveEngine=output.trt.

Error:
While I am doing this , I have resulted in the following error,
&&&& RUNNING TensorRT.trtexec # caffe_bvlc/TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1/TensorRT-7.2.3.4/bin/trtexec --deploy=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/test.prototxt --model=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/trainval.caffemodel --output=layer117-conv --batch=64 --saveEngine=output.trt
[09/29/2021-09:38:36] [I] === Model Options ===
[09/29/2021-09:38:36] [I] Format: Caffe
[09/29/2021-09:38:36] [I] Model: /home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/trainval.caffemodel
[09/29/2021-09:38:36] [I] Prototxt: /home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/test.prototxt
[09/29/2021-09:38:36] [I] Output: layer117-conv
[09/29/2021-09:38:36] [I] === Build Options ===
[09/29/2021-09:38:36] [I] Max batch: 64
[09/29/2021-09:38:36] [I] Workspace: 16 MiB
[09/29/2021-09:38:36] [I] minTiming: 1
[09/29/2021-09:38:36] [I] avgTiming: 8
[09/29/2021-09:38:36] [I] Precision: FP32
[09/29/2021-09:38:36] [I] Calibration:
[09/29/2021-09:38:36] [I] Refit: Disabled
[09/29/2021-09:38:36] [I] Safe mode: Disabled
[09/29/2021-09:38:36] [I] Save engine: output.trt
[09/29/2021-09:38:36] [I] Load engine:
[09/29/2021-09:38:36] [I] Builder Cache: Enabled
[09/29/2021-09:38:36] [I] NVTX verbosity: 0
[09/29/2021-09:38:36] [I] Tactic sources: Using default tactic sources
[09/29/2021-09:38:36] [I] Input(s)s format: fp32:CHW
[09/29/2021-09:38:36] [I] Output(s)s format: fp32:CHW
[09/29/2021-09:38:36] [I] Input build shapes: model
[09/29/2021-09:38:36] [I] Input calibration shapes: model
[09/29/2021-09:38:36] [I] === System Options ===
[09/29/2021-09:38:36] [I] Device: 0
[09/29/2021-09:38:36] [I] DLACore:
[09/29/2021-09:38:36] [I] Plugins:
[09/29/2021-09:38:36] [I] === Inference Options ===
[09/29/2021-09:38:36] [I] Batch: 64
[09/29/2021-09:38:36] [I] Input inference shapes: model
[09/29/2021-09:38:36] [I] Iterations: 10
[09/29/2021-09:38:36] [I] Duration: 3s (+ 200ms warm up)
[09/29/2021-09:38:36] [I] Sleep time: 0ms
[09/29/2021-09:38:36] [I] Streams: 1
[09/29/2021-09:38:36] [I] ExposeDMA: Disabled
[09/29/2021-09:38:36] [I] Data transfers: Enabled
[09/29/2021-09:38:36] [I] Spin-wait: Disabled
[09/29/2021-09:38:36] [I] Multithreading: Disabled
[09/29/2021-09:38:36] [I] CUDA Graph: Disabled
[09/29/2021-09:38:36] [I] Separate profiling: Disabled
[09/29/2021-09:38:36] [I] Skip inference: Disabled
[09/29/2021-09:38:36] [I] Inputs:
[09/29/2021-09:38:36] [I] === Reporting Options ===
[09/29/2021-09:38:36] [I] Verbose: Disabled
[09/29/2021-09:38:36] [I] Averages: 10 inferences
[09/29/2021-09:38:36] [I] Percentile: 99
[09/29/2021-09:38:36] [I] Dump refittable layers:Disabled
[09/29/2021-09:38:36] [I] Dump output: Disabled
[09/29/2021-09:38:36] [I] Profile: Disabled
[09/29/2021-09:38:36] [I] Export timing to JSON file:
[09/29/2021-09:38:36] [I] Export output to JSON file:
[09/29/2021-09:38:36] [I] Export profile to JSON file:
[09/29/2021-09:38:36] [I]
[09/29/2021-09:38:36] [I] === Device Information ===
[09/29/2021-09:38:36] [I] Selected Device: NVIDIA GeForce RTX 3080
[09/29/2021-09:38:36] [I] Compute Capability: 8.6
[09/29/2021-09:38:36] [I] SMs: 68
[09/29/2021-09:38:36] [I] Compute Clock Rate: 1.77 GHz
[09/29/2021-09:38:36] [I] Device Global Memory: 10014 MiB
[09/29/2021-09:38:36] [I] Shared Memory per SM: 100 KiB
[09/29/2021-09:38:36] [I] Memory Bus Width: 320 bits (ECC disabled)
[09/29/2021-09:38:36] [I] Memory Clock Rate: 9.501 GHz
[09/29/2021-09:38:36] [I]
[libprotobuf ERROR google/protobuf/text_format.cc:298] Error parsing text-format ditcaffe.NetParameter: 2027:23: Message type “ditcaffe.LayerParameter” has no field named “deephi_resize_param”.
[09/29/2021-09:38:36] [E] [TRT] CaffeParser: Could not parse deploy file
[09/29/2021-09:38:36] [E] Failed to parse caffe model or prototxt, tensors blob not found
[09/29/2021-09:38:36] [E] Parsing model failed
[09/29/2021-09:38:36] [E] Engine creation failed
[09/29/2021-09:38:36] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # caffe_bvlc/TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1/TensorRT-7.2.3.4/bin/trtexec --deploy=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/test.prototxt --model=/home/vvsa/raji/dk_yolov3_cityscapes_256_512_0.9_5.46G_1.4/float/trainval.caffemodel --output=layer117-conv --batch=64 --saveEngine=output.trt

Environment

TensorRT Version: 7.2.3.4
GPU Type: RTX 3080
Nvidia Driver Version: 470.63.01
CUDA Version: 11.1
CUDNN Version: 8.1.0
Operating System + Version: x86_64 + 18.04
Python Version (if applicable): 3.6.9 (using virtual environment)

I have attached my prototxt and caffemodel.
test.prototxt (39.9 KB)
trainval.caffemodel (9.0 MB)

How to get through this?

2

Hi , UFF and Caffe Parser have been deprecated from TensorRT 7 onwards, hence request you to try ONNX parser.
Please check the below link for the same.

Thanks!

4

Hi ,
I converted my caffe xilinx model to onnx , then I use onnx parser to convert my onnx model to tensorrt.While doing so , I resulted in this error,

ERROR: builtin_op_importers.cpp:516 In function importConv:
[6] Assertion failed: nchan == -1 || kernelWeights.shape.d[1] * ngroup == nchan
[09/30/2021-09:34:59] [E] Failed to parse onnx file
[09/30/2021-09:34:59] [E] Parsing model failed
[09/30/2021-09:34:59] [E] Engine creation failed
[09/30/2021-09:34:59] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # caffe_bvlc/TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1/TensorRT-7.2.3.4/bin/trtexec --onnx=/home/vvsa/output.onnx --batch=64 --int8 --saveEngine=raji/caffe.plan --workspace=256

The environment setup is same as I mentioned in the above issue.
How to get through this??

5

Hi,

Could you share us onnx model to try from our end for better help.

Thank you.

6

Hi ,
I have attached my onnx model here.

Note: It is converted to onnx from a yolov3 model built on caffe-xilinx

output.onnx (9.0 MB)

7

Hi,

We could reproduce the same issue. Looks like concat is broken. The input has C=52, but the Concat only has one input, while the next Conv expects the input to have C=78.
Please check.

image
image893×818 22.6 KB

Thank you.

8

Hi ,
If concat layer is broken means there should be problems in onnx conversion right

How to solve this issue?

9

Yes, We need to correct the dimension.
Following may be helpful to you to modify onnx.
https://docs.nvidia.com/deeplearning/tensorrt/onnx-graphsurgeon/docs/index.html