tensorrt inference error while load onnx model

Unsupported ONNX data type: UINT8(2)
[TensorRT] ERROR: Network must have at least one output

anybody can help

Hi,

TensorRT only supports FP32, FP16, INT32, and INT8: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/FoundationalTypes/DataType.html#tensorrt.DataType

UINT8 is not supported as mentioned there. Can you try to use int8 or int32 instead of uint8 before converting your model to ONNX?

Alternatively, you could try to use the ONNX API to convert the UINT8 nodes to INT8 or INT32 after training/converting to ONNX, but these could potentially create incorrect results if not handled properly.

For example, the range of UINT8 is [0, 255], and INT8 is [-127, 128], so if you just converted UINT8->INT8, those values between [128-255] may propogate incorrectly through the network after being casted. INT32 might be a safer bet as it will cover the range of UINT8 (and more).

However, if the casting to INT8/INT32 was done in the original DL framework before training time, the model would likely be fine with this, as it would be trained/learn with these types.

Thanks.

after use int8, I also have the same problem “[TensorRT] ERROR: Network must have at least one output”.

I use C++ parse the onnx model, get the problem “[8] Assertion failed: axis>=0 && axis<nbDims”

pytorch:1.2.0
torchvision:0.4.0
tensorrt:6.0.1.5

FYR

Hi,

There may be a couple fixes for the output error.

First can you check that the model was parsed correctly? parser->parse… should return True/False depending on whether it parsed the model successfully. Example: https://github.com/NVIDIA/TensorRT/blob/07ed9b57b1ff7c24664388e5564b17f7ce2873e5/samples/opensource/sampleINT8API/sampleINT8API.cpp#L481-L483

If the model was parsed successfully, then the output error has a pretty simple fix. Either mark the output with output_names=… in torch.onnx.export() when exporting your model to ONNX from PyTorch. Or you can do this after the fact in TensorRT by doing something like:

network->markOutput(network->getLayer(network->getNbLayers() - 1)->get_output(0))

If the parser didn’t successfully parse the model (returned False), then maybe there is some unsupported op or something on the model. A common one from PyTorch could be something like the view() function, as mentioned here: https://github.com/onnx/onnx-tensorrt/issues/125#issuecomment-502931336

yes, https://github.com/onnx/onnx-tensorrt/issues/125#issuecomment-502931336 solve the problem.

thanks.

Glad it worked!

How can I change the UINT8 to INT32 in tensorflow ssd inception v2 model.?
Is there any workaround for this model.

I got the below error when i run ./trtexec --onxx=inception.onnx comand
Error repeated for boht custom trained and also standard ssd inception v2 model

WARNING: ONNX model has a newer ir_version (0.0.5) than this parser was built against (0.0.3).
Unsupported ONNX data type: UINT8 (2)
ERROR: ModelImporter.cpp:54 In function importInput:
[8] Assertion failed: convert_dtype(onnx_tensor_type.elem_type(), &trt_dtype)

Any suggestions for this??
I am using jetson nano with below specifications.

sudo apt-cache show nvidia-jetpack
Package: nvidia-jetpack
Version: 4.3-b134
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-container-csv-cuda (= 10.0.326-1), libopencv-python (= 4.1.1-2-gd5a58aa75), libvisionworks-sfm-dev (= 0.90.4), libvisionworks-dev (= 1.6.0.500n), libvisionworks-samples (= 1.6.0.500n), libnvparsers6 (= 6.0.1-1+cuda10.0), libcudnn7-doc (= 7.6.3.28-1+cuda10.0), libcudnn7-dev (= 7.6.3.28-1+cuda10.0), libnvinfer-samples (= 6.0.1-1+cuda10.0), libnvinfer-bin (= 6.0.1-1+cuda10.0), nvidia-container-csv-cudnn (= 7.6.3.28-1+cuda10.0), libvisionworks-tracking-dev (= 0.88.2), vpi-samples (= 0.1.0), tensorrt (= 6.0.1.10-1+cuda10.0), libopencv (= 4.1.1-2-gd5a58aa75), libnvinfer-doc (= 6.0.1-1+cuda10.0), libnvparsers-dev (= 6.0.1-1+cuda10.0), libcudnn7 (= 7.6.3.28-1+cuda10.0), libnvidia-container0 (= 0.9.0beta.1), cuda-toolkit-10-0 (= 10.0.326-1), nvidia-container-csv-visionworks (= 1.6.0.500n), graphsurgeon-tf (= 6.0.1-1+cuda10.0), libopencv-samples (= 4.1.1-2-gd5a58aa75), python-libnvinfer-dev (= 6.0.1-1+cuda10.0), libnvinfer-plugin-dev (= 6.0.1-1+cuda10.0), libnvinfer-plugin6 (= 6.0.1-1+cuda10.0), nvidia-container-toolkit (= 1.0.1-1), libnvinfer-dev (= 6.0.1-1+cuda10.0), libvisionworks (= 1.6.0.500n), libopencv-dev (= 4.1.1-2-gd5a58aa75), nvidia-l4t-jetson-multimedia-api (= 32.3.1-20191209225816), vpi-dev (= 0.1.0), vpi (= 0.1.0), python3-libnvinfer (= 6.0.1-1+cuda10.0), python3-libnvinfer-dev (= 6.0.1-1+cuda10.0), opencv-licenses (= 4.1.1-2-gd5a58aa75), nvidia-container-csv-tensorrt (= 6.0.1.10-1+cuda10.0), libnvinfer6 (= 6.0.1-1+cuda10.0), libnvonnxparsers-dev (= 6.0.1-1+cuda10.0), libnvonnxparsers6 (= 6.0.1-1+cuda10.0), uff-converter-tf (= 6.0.1-1+cuda10.0), nvidia-docker2 (= 2.2.0-1), libvisionworks-sfm (= 0.90.4), libnvidia-container-tools (= 0.9.0beta.1), nvidia-container-runtime (= 3.1.0-1), python-libnvinfer (= 6.0.1-1+cuda10.0), libvisionworks-tracking (= 0.88.2)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.3-b134_arm64.deb
Size: 29742
SHA256: 1fd73e258509822b928b274f61a413038a29c3705ee8eef351a914b9b1b060ce
SHA1: a7c4ab8b241ab1d2016d2c42f183c295e66d67fe
MD5sum: de856bb9607db87fd298faf7f7cc320f
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8