Driveworks dnn custom model for objects detection (yolo)

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
DRIVE OS Linux 5.1.6.1 and DriveWorks 2.2
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.8.0.10363
other

Host Machine Version
native Ubuntu 18.04
other

Hi,
I have a problem with implementation custom model dnn (yolo v4 or yolov3) for example: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights

I try convert yolo model (cfg, weights) to onnx model - GitHub - Tianxiaomo/pytorch-YOLOv4: PyTorch ,ONNX and TensorRT implementation of YOLOv4
I installed onnx=1.6.0 and torch=1.2.0 or 1.3.0 or 1.10.2 and I can convert to onnx model

python3 demo_darknet2onnx.py cfg/yolov4.cfg data/coco.names yolov4.weights data/dog.jpg -1

but I have a problem with convert onnx model to tensorrt model using tensorRT_optimization

/usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=onnx --onnxFile=yolov4.onnx --out=yolov4.bin

I have a lot of problems, for example:

Initializing network optimizer on model yolov4.onnx.
----------------------------------------------------------------
Input filename:   yolov4.onnx
ONNX IR version:  0.0.7
Opset version:    10
Producer name:    darknet to ONNX example
Producer version: 
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.7) than this parser was built against (0.0.3).
While parsing node number 201 [Resize]:
ERROR: ModelImporter.cpp:147 In function importNode:
[8] No importer registered for op: Resize
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov4.onnx.
----------------------------------------------------------------
Input filename:   yolov4.onnx
ONNX IR version:  0.0.7
Opset version:    9
Producer name:    pytorch
Producer version: 1.10
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.7) than this parser was built against (0.0.3).
While parsing node number 0 [Conv]:
ERROR: ModelImporter.cpp:288 In function importModel:
[5] Assertion failed: tensors.count(input_name)
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    9
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
WARNING: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Successfully casted down to INT32.
While parsing node number 215 [Gather]:
ERROR: onnx2trt_utils.hpp:277 In function convert_axis:
[8] Assertion failed: axis >= 0 && axis < nbDims
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    8
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
While parsing node number 214 [Cast]:
ERROR: builtin_op_importers.cpp:704 In function importCast:
[8] Assertion failed: inputs.at(0).is_tensor()
Error: DW_FILE_INVALID: Unable to parse given onnx file.

Can you help what I doing wrong?

I tried do this simple tutorial: GitHub - ShayNvidia/dnnWebinar: NVIDIA DriveWorks for Drive
and I have a bin model and it working fine.

Hi,

Have you tried to generate the ONNX model with ir_version = 0.0.3?

It looks like these are compatibility issues.
It will be good if you can try the ONNX version we have tested to see if the same issue.

Thanks.

Hi @AastaLLL ,
I want to generate the ONNX model with ir_version = 0.0.3 but if I use torch=1.10.2 the ir_version was 0.0.7, if torch=1.2.0 the ir_version was 0.0.4, if torch=1.1.0 the ir_version was 0.0.4, if torch=1.0.0 I can’t convert model to onnx.
If the solution is to generate = ir_0.0.3, I would like to generate this version but I don’t know how, maybe other libraries affect it but I don’t know what.
Thanks for tip but I will be try find the solution, help will be appreciated.

Hi,

Would you mind applying some extra experiments to the testing of IR 0.0.4 + Opset 9 setting first?

It seems the error is related to the output format of ONNX.
Could you try if the below workaround can help with your use case?

Thanks.

I tried a lot of opset version and this is an oputput to convert bin file (onxx->bin)

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    9
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
WARNING: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Successfully casted down to INT32.
While parsing node number 2 [Gather]:
ERROR: onnx2trt_utils.hpp:277 In function convert_axis:
[8] Assertion failed: axis >= 0 && axis < nbDims
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    10
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
WARNING: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
Successfully casted down to INT32.
While parsing node number 2 [Gather]:
ERROR: onnx2trt_utils.hpp:277 In function convert_axis:
[8] Assertion failed: axis >= 0 && axis < nbDims
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    7
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
While parsing node number 1 [Cast]:
ERROR: builtin_op_importers.cpp:704 In function importCast:
[8] Assertion failed: inputs.at(0).is_tensor()
Error: DW_FILE_INVALID: Unable to parse given onnx file.

or

Initializing network optimizer on model yolov3.onnx.
----------------------------------------------------------------
Input filename:   yolov3.onnx
ONNX IR version:  0.0.4
Opset version:    8
Producer name:    pytorch
Producer version: 1.2
Domain:           
Model version:    0
Doc string:       
----------------------------------------------------------------
WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
While parsing node number 1 [Cast]:
ERROR: builtin_op_importers.cpp:704 In function importCast:
[8] Assertion failed: inputs.at(0).is_tensor()
Error: DW_FILE_INVALID: Unable to parse given onnx file.

Hi,

Thanks for the testing.

For the first testing (IR0.0.4 + Opset9), have you applied the patch shared above?
It uses tensor.view(.) to change the output format and is expected to fix the axis error you met.

Thanks.

Hi,
I used this code to export to onnx:

#!/usr/bin/env python3

import torch
from torch import onnx

input = torch.rand(2,64,4)

class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        n, hw, c = x.shape
        # Split the hw dimension into 8*8
        h = 8
        w = hw // h
        x = x.view(n, h, w, c)
        # NHWC -> NCHW is pretty common.
        return x.permute(0, 3, 1, 2)

# Opset version 9 is required by Jetson Xavier, which has TRT6.
torch.onnx.export(Model(), input, "yolov3.onnx", opset_version=9)

Hi,

Thanks for the confirmation.

It looks like the YOLOv3 is not supported by the TensorRT 5.
Could you use other networks for your use case?

For example, the below model should work on TensorRT 5.1:

Thanks.

Hi,
Thanks for replay, but I still have a problem with conversion to bin file.
If I use clasification model (for example: resnet-101) it’s works(I’m very happy),

but if I want use detections model (ssd_mobilenet_v1_coco.uff or ssd_mobilenet_v2_coco.uff or ssd_inception_v2_coco.uff) I can’t convert to tensorrt model:

/usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=bboxes,coverage --uffFile=ssd_mobilenet_v1_coco.uff --inputBlobs=ssd_coco_labels.txt --inputDims=3x480x960
Initializing network optimizer on model ssd_mobilenet_v1_coco.uff.
UffParser: Validator error: Postprocessor: Unsupported operation _NMS_TRT
Error: DW_INTERNAL_ERROR: Unable to parse uff file.

or

/usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=bboxes,coverage --uffFile=ssd_inception_v2_coco.uff --inputBlobs=data0,data1 --inputDims=3x480x960,1x1x10
Initializing network optimizer on model ssd_inception_v2_coco.uff.
UffParser: Validator error: GridAnchor: Unsupported operation _GridAnchor_TRT
Error: DW_INTERNAL_ERROR: Unable to parse uff file.

and I try change inputDims and it doesn’t work.

Hi,

Thanks for the testing.

NMS and GridAnchor should include in our TensorRT plugin library.
Could you share the output of the below command with us?

$ ldd /usr/local/driveworks/tools/dnn/tensorRT_optimization

Thanks.

Hi,
on host system:

ldd /usr/local/driveworks/tools/dnn/tensorRT_optimization
	linux-vdso.so.1 (0x00007fffec2ab000)
	libnvinfer.so.5 => /usr/local/driveworks/tools/dnn/../../targets/x86_64-Linux/lib/libnvinfer.so.5 (0x00007f32fb8d6000)
	libnvparsers.so.5 => /usr/local/driveworks/tools/dnn/../../targets/x86_64-Linux/lib/libnvparsers.so.5 (0x00007f32fb3a9000)
	libnvonnxparser.so.0 => /usr/local/driveworks/tools/dnn/../../targets/x86_64-Linux/lib/libnvonnxparser.so.0 (0x00007f32fac9d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f32faa95000)
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007f32f936f000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f32f916b000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f32f8f4c000)
	libcudart.so.10.2 => /usr/local/cuda/lib64/libcudart.so.10.2 (0x00007f32f8cce000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f32f8945000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f32f85a7000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f32f838f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f32f7f9e000)
	libcudnn.so.7 => /usr/local/driveworks/tools/dnn/../../targets/x86_64-Linux/lib/libcudnn.so.7 (0x00007f32e03e4000)
	libcublas.so.10 => /usr/local/cuda/lib64/libcublas.so.10 (0x00007f32dc12d000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3304bb4000)
	libcublasLt.so.10 => /usr/local/cuda/lib64/libcublasLt.so.10 (0x00007f32da298000)

on target system:

nvidia@tegra-ubuntu:~$ ldd /usr/local/driveworks/tools/dnn/tensorRT_optimization
	linux-vdso.so.1 (0x0000007f8e75d000)
	libnvdla_compiler.so => /usr/lib/libnvdla_compiler.so (0x0000007f8e2e4000)
	libnvinfer.so.5 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libnvinfer.so.5 (0x0000007f84dc2000)
	libnvparsers.so.5 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libnvparsers.so.5 (0x0000007f84a83000)
	libcudnn.so.7 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libcudnn.so.7 (0x0000007f6add3000)
	libnvonnxparser.so.0 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libnvonnxparser.so.0 (0x0000007f6a9b0000)
	libnvinfer_plugin.so.5 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libnvinfer_plugin.so.5 (0x0000007f6a6bd000)
	libcublas.so.10 => /usr/lib/aarch64-linux-gnu/libcublas.so.10 (0x0000007f66030000)
	libnvToolsExt.so.1 => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libnvToolsExt.so.1 (0x0000007f66018000)
	libnvmedia_ipp.so => /usr/lib/libnvmedia_ipp.so (0x0000007f65cd7000)
	libcuda.so.1 => /usr/lib/libcuda.so.1 (0x0000007f64dd9000)
	libnvmedia.so => /usr/lib/libnvmedia.so (0x0000007f64d49000)
	libcudart.so.10.2 => /usr/local/cuda-10.2/targets/aarch64-linux/lib/libcudart.so.10.2 (0x0000007f64cd6000)
	libglfw.so => /usr/local/driveworks/tools/dnn/../../targets/aarch64-Linux/lib/libglfw.so (0x0000007f64ca7000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f64c7b000)
	libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f64b52000)
	libXext.so.6 => /usr/lib/aarch64-linux-gnu/libXext.so.6 (0x0000007f64b32000)
	libXxf86vm.so.1 => /usr/lib/aarch64-linux-gnu/libXxf86vm.so.1 (0x0000007f64b1d000)
	libXinerama.so.1 => /usr/lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007f64b0a000)
	libXrandr.so.2 => /usr/lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007f64af0000)
	libEGL_nvidia.so.0 => /usr/lib/aarch64-linux-gnu/libEGL_nvidia.so.0 (0x0000007f649d5000)
	libdrm.so.2 => /usr/lib/libdrm.so.2 (0x0000007f649a3000)
	libudev.so.1 => /lib/aarch64-linux-gnu/libudev.so.1 (0x0000007f64979000)
	libusb-1.0.so.0 => /lib/aarch64-linux-gnu/libusb-1.0.so.0 (0x0000007f64953000)
	libnv_extimgdev.so => /usr/lib/libnv_extimgdev.so (0x0000007f64918000)
	libnv_embstatsplugin.so => /usr/lib/libnv_embstatsplugin.so (0x0000007f64903000)
	librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f648ec000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f648d7000)
	libGLESv2_nvidia.so.2 => /usr/lib/aarch64-linux-gnu/libGLESv2_nvidia.so.2 (0x0000007f648ab000)
	libXi.so.6 => /usr/lib/aarch64-linux-gnu/libXi.so.6 (0x0000007f6488d000)
	libXcursor.so.1 => /usr/lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007f64874000)
	libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f646e1000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f64627000)
	libgomp.so.1 => /usr/lib/aarch64-linux-gnu/libgomp.so.1 (0x0000007f645ea000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f645c6000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f6446d000)
	/lib/ld-linux-aarch64.so.1 (0x0000007f8e732000)
	libnvos.so => /usr/lib/libnvos.so (0x0000007f6444d000)
	libEGL.so.1 => /usr/lib/aarch64-linux-gnu/libEGL.so.1 (0x0000007f6442c000)
	libcublasLt.so.10 => /usr/lib/aarch64-linux-gnu/libcublasLt.so.10 (0x0000007f6220c000)
	libnvrm.so => /usr/lib/libnvrm.so (0x0000007f621c7000)
	libnvmedia_isc.so => /usr/lib/libnvmedia_isc.so (0x0000007f621ad000)
	libnvrm_gpu.so => /usr/lib/libnvrm_gpu.so (0x0000007f6215f000)
	libnvrm_graphics.so => /usr/lib/libnvrm_graphics.so (0x0000007f62140000)
	libnvidia-fatbinaryloader.so.425.00 => /usr/lib/libnvidia-fatbinaryloader.so.425.00 (0x0000007f620e5000)
	libnvscibuf.so.1 => /usr/lib/libnvscibuf.so.1 (0x0000007f620a7000)
	libnvscisync.so.1 => /usr/lib/libnvscisync.so.1 (0x0000007f62089000)
	libnvtegrahv.so => /usr/lib/libnvtegrahv.so (0x0000007f62077000)
	libnvm_vicsi_v3.so => /usr/lib/libnvm_vicsi_v3.so (0x0000007f6204f000)
	libnvdc.so => /usr/lib/libnvdc.so (0x0000007f62023000)
	libnvtvmr.so => /usr/lib/libnvtvmr.so (0x0000007f61f95000)
	libnvparser.so => /usr/lib/libnvparser.so (0x0000007f61f56000)
	libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f61f26000)
	libXrender.so.1 => /usr/lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007f61f0b000)
	libnvidia-glsi.so.425.00 => /usr/lib/libnvidia-glsi.so.425.00 (0x0000007f61e66000)
	libnvll.so => /usr/lib/libnvll.so (0x0000007f61e49000)
	libXfixes.so.3 => /usr/lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007f61e33000)
	libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/libGLdispatch.so.0 (0x0000007f61d07000)
	libnvscicommon.so => /usr/lib/libnvscicommon.so (0x0000007f61cef000)
	libnvsciipc.so => /usr/lib/libnvsciipc.so (0x0000007f61cd5000)
	libnvimp.so => /usr/lib/libnvimp.so (0x0000007f61cc0000)
	libnvddk_2d_v2.so => /usr/lib/libnvddk_2d_v2.so (0x0000007f61c9b000)
	libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f61c88000)
	libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f61c71000)
	libnvidia-rmapi-tegra.so.425.00 => /usr/lib/libnvidia-rmapi-tegra.so.425.00 (0x0000007f61c30000)
	libnvrm_interop_gpu.so => /usr/lib/libnvrm_interop_gpu.so (0x0000007f61c1c000)
	libnvivc.so => /usr/lib/libnvivc.so (0x0000007f61c09000)
	libnvddk_vic.so => /usr/lib/libnvddk_vic.so (0x0000007f61beb000)
	libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f61bc7000)

Hi,

It looks like the plugin library is well-linked in your environment.
But the input argument tensorRT_optimization looks incorrect.

You can find the corresponding argument in the below link:
https://github.com/dusty-nv/jetson-inference/blob/L4T-R32.2/c/detectNet.cpp#L425

Would you mind running the below command to see if it works?

$ /usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=NMS,NMS_1 --uffFile=ssd_inception_v2_coco.uff --inputBlobs=Input --inputDims=3,300,300
$ /usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=Postprocessor,Postprocessor_1 --uffFile=ssd_mobilenet_v1_coco.uff --inputBlobs=Input --inputDims=3,300,300

Thanks.

Hi,
I try above command and output is:

nvidia@tegra-ubuntu:~/Downloads$ /usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=NMS,NMS_1 --uffFile=ssd_inception_v2_coco.uff --inputBlobs=Input --inputDims=3,300,300
Error: DW_INVALID_ARGUMENT: An input is supposed to have three dimensions: CxHxW

if I change --inputDims from 3,300,300 to --inputDims=3x300x300, I have this output:

nvidia@tegra-ubuntu:~/Downloads$ /usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=Postprocessor,Postprocessor_1 --uffFile=ssd_mobilenet_v1_coco.uff --inputBlobs=Input --inputDims=3x300x300
Initializing network optimizer on model ssd_mobilenet_v1_coco.uff.
UffParser: Validator error: Postprocessor: Unsupported operation _NMS_TRT
Error: DW_INTERNAL_ERROR: Unable to parse uff file.

nvidia@tegra-ubuntu:~/Downloads$ /usr/local/driveworks/tools/dnn/tensorRT_optimization --modelType=uff --outputBlobs=NMS,NMS_1 --uffFile=ssd_inception_v2_coco.uff --inputBlobs=Input --inputDims=3x300x300
Initializing network optimizer on model ssd_inception_v2_coco.uff.
UffParser: Validator error: GridAnchor: Unsupported operation _GridAnchor_TRT
Error: DW_INTERNAL_ERROR: Unable to parse uff file.

Dear @Matthew_Pimot,
Please confirm if you are sticking DRIVE SW 10 or you can move to new DW + DOS release? We notice on DRIVE OS 5.2.6 + DW 4.0 release, customer could run yolo v3 model. It is recommended to test your model with trtexec tool in TensorRT before using DW.
Please see the discussion in Yolov3 Implementation error in inference
Sample object detector tracker YOLOV3 model error inference if it helps.