maybe related to https://devtalk.nvidia.com/default/topic/1047625/jetson-tx2/running-a-pytorch-network-converted-to-onnx-with-tensorrt-on-the-tx2/1 and fix should be available in TensorRT5.1 and the package in JetPack4.2.1.
Can you please share me link to jetpack 4.2.1, we are not able to find it.
Is this also applicable to Nano ?
The link only gives jetpack 4.2, whose disc image size is 5505865KB on windows machine. This same jetpack is running on my nano and this one is giving error as reported above .
Can you please cross check as 4.2.1 seem to be not available to public.
If it is available, please share exact link.
Gentle Reminder to Nvidia team !
Please share the link for 4.2.1, as this fix is badly required for us.
Gentle Reminder !
Please provide link for Jetpack 4.2.1 .
If you couldn’t find the link at that time, it probably wasn’t released yet. You can find currently find links to download 4.2.1, 4.2.2, 4.2.3, and various other archived versions at https://developer.nvidia.com/embedded/downloads.
I also met this problem that I convert the saved_model.pb from uff_ssd of python sample to onnx.
My command is
python3 -m tf2onnx.convert --opset 10 --fold_const --saved-model ./workspace/models/ssd_inception_v2_coco_2017_11_17/saved_model --output MODEL.onnx
Next, I parser this .onnx by parser.parse() then getting the error of
Unsupported ONNX data type: UINT8 (2).
Get model onnx path. /home/chieh/Downloads/TensorRT-22.214.171.124/samples/python/onnx_ssd/utils/../workspace/models/ssd_inception_v2_coco_2017_11_17/ssd_inception_v2_coco_2017_11_17.onnx TensorRT inference engine settings: * Inference precision - DataType.FLOAT * Max batch size - 64 Loading ONNX file from path /home/chieh/Downloads/TensorRT-126.96.36.199/samples/python/onnx_ssd/utils/../workspace/models/ssd_inception_v2_coco_2017_11_17/ssd_inception_v2_coco_2017_11_17.onnx... onnx_file_path /home/chieh/Downloads/TensorRT-188.8.131.52/samples/python/onnx_ssd/utils/../workspace/models/ssd_inception_v2_coco_2017_11_17/ssd_inception_v2_coco_2017_11_17.onnx Beginning ONNX file parsing Unsupported ONNX data type: UINT8 (2) ERROR: Failed to parse the ONNX file. In node -1 (importInput): UNSUPPORTED_NODE: Assertion failed: convertDtype(onnxDtype.elem_type(), &trtDtype) Traceback (most recent call last): File "voc_evaluation.py", line 495, in <module> parsed['trt_engine_datatype'], parsed['max_batch_size']) File "/home/chieh/Downloads/TensorRT-184.108.40.206/samples/python/onnx_ssd/utils/inference.py", line 119, in __init__ engine_utils.save_engine(self.trt_engine, trt_engine_path) File "/home/chieh/Downloads/TensorRT-220.127.116.11/samples/python/onnx_ssd/utils/engine.py", line 185, in save_engine buf = engine.serialize() AttributeError: 'NoneType' object has no attribute 'serialize'
Indeed, I checked the input of model which is type: uint8[?,?,?,3].
However, the frozen_inference_graph.pb can convert to .uff, and then successfully build the engine and do inference in Uff_ssd sample.
Is there any method to solve it?
Or should we directly rebuild the architecture and train the model from scratch again?
TensorRT version: 18.104.22.168
Cuda version: 10.2
Cudnn version: 7.6.5
I’m having exact same problem as Chieh. I need to migrate to using ONNX rather than deprecated UFF.
I too ran into this problem and it appears as though all of the models (except the quantized ones) in the TensorFlow detection model zoo repo contain input layers with the datatype of unint8. TensorRT is not compatible with this datatype (you know that already). However, the models at ONNX Model Zoo all have input layers with a datatype of float32. Also, in this blog post, Speed up TensorFlow Inference on GPUs with TensorRT, the SavedModels (i.e., protobuf) files that come with the examples also have float32 input layers. I do not know their source though. From my preliminary testing, I was able to convert from pb --> onnx --> trt engine for the ONNX Model Zoo files and the ones posted on the devblog page.
So, with that being said, I think the solution is to do as Chieh suggested. Either re-build the model architecture to have a input layer with a float32 datatype or download any of the ONNX Model Zoo models. In both cases, however, I think you will have to re-train no matter what. Unless someone else responds with a better solution, I think that is the only way (for now).
Any solutions to this problem??
WARNING: ONNX model has a newer ir_version (0.0.5) than this parser was built against (0.0.3).
Unsupported ONNX data type: UINT8 (2)
ERROR: ModelImporter.cpp:54 In function importInput:
 Assertion failed: convert_dtype(onnx_tensor_type.elem_type(), &trt_dtype)
[05/29/2020-10:13:46] [E] Failed to parse onnx file
[05/29/2020-10:13:46] [E] Parsing model failed
[05/29/2020-10:13:46] [E] Engine could not be created
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=inception_standard.onnx
Even I have the same problem on converting my model to onnx. Any solutions?
Unsupported ONNX data type: UINT8 (2)
ERROR: batch:1:191 In function importInput:
 Assertion failed: convertDtype(onnxDtype.elem_type(), &trtDtype)
[06/29/2020-16:30:09] [E] Failed to parse onnx file
[06/29/2020-16:30:09] [E] Parsing model failed
[06/29/2020-16:30:09] [E] Engine creation failed
[06/29/2020-16:30:09] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=/home/stackfusion/Downloads/train_batch_shape.onnx --shapes=input_3:1x200x200x3
Same problem for me when trying to read in onnx file with TensorRT onnx parser. This seems odd, since on https://github.com/onnx/onnx/blob/master/docs/IR.md it specifically says it supports uint8 tensor element types. Is there something unique about “UINT8 (2)” or bug in parser?
---------------------------------------------------------------- Input filename: ../resources/mars-small128_batch_1.onnx ONNX IR version: 0.0.6 Opset version: 11 Producer name: tf2onnx Producer version: 1.6.3 Domain: Model version: 0 Doc string: ---------------------------------------------------------------- Unsupported ONNX data type: UINT8 (2) ERROR: images:0:188 In function importInput:  Assertion failed: convertDtype(onnxDtype.elem_type(), &trtDtype
Looks like the “(2)” in UINT8 (2) is just the enum value of the type. I wrote a python script to edit the type of the input layer (as well as set the batch size), but this created another error:
map/while/strided_slice: out of bounds slice, input dimensions = [128,64,3], start = [0,0,3], size = [128,64,3], stride = [1,1,-1]. Layer map/while/strided_slice failed validation
So I’m investigating that now.
In case anyone is interested, here’s my python script for changing the input node type and the network batch size of an onnx model file.
import onnx def change_input_datatype(model, typeNdx): # values for typeNdx # 1 = float32 # 2 = uint8 # 3 = int8 # 4 = uint16 # 5 = int16 # 6 = int32 # 7 = int64 inputs = model.graph.input for input in inputs: input.type.tensor_type.elem_type = typeNdx dtype = input.type.tensor_type.elem_type def change_input_batchsize(model, batchSize): inputs = model.graph.input for input in inputs: dim1 = input.type.tensor_type.shape.dim dim1.dim_value = batchSize #print("input: ", input) # uncomment to see input layer details def change_output_batchsize(model, batchSize): outputs = model.graph.output for output in outputs: dim1 = output.type.tensor_type.shape.dim dim1.dim_value = batchSize #print("output: ", output) #uncomment to see output layer details onnx_model = onnx.load(<path to your original onnx model file>) change_input_datatype(onnx_model, 1) change_input_batchsize(onnx_model, 1) change_output_batchsize(onnx_model, 1) onnx.save(onnx_model, <path to your edited onnx model file>)
I used to have the same problem when I tried to convert ssd_mobilenet_v3.pb -> onnx -> TensorRT engine.
It is no problem to convert to ONNX, the model still works in the onnx runtime. The problem is the TensorRT ONNX parser, which does not support UINT8.
So I replaced the input type in the tensorflow frozen graph from UINT8->Float32. I described it with code example here
Did you solve this issue??
TRT cannot support UINT8.
It is the final conclusion.
We can not even port standard object detection models to tensorRT.
Did you successfully inferenced any tensorflow object detection models in tensorRT c++ version??
I already changed to use torch because uff will be deprecated in next version and other converting tool which still cannot support a lot of ops.
Of course, it depends on your model how it is complicated. :))
Hello, so could you help with my model?
I have same issue with onnx2trt for ssd mobilenet model?