# TensorRT 5.0.2.6 onnx run error

**URL:** <https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578>\
**Category:** TensorRT\
**Created:** [November 19, 2018, 2:36pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578 "2018-11-19T14:36:18Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![soroor.shekarizade](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@soroor.shekarizade](https://forums.developer.nvidia.com/u/soroor.shekarizade)\
**Post date:** [November 19, 2018, 2:36pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/1 "2018-11-19T14:36:18Z")

</div>

Hi,  
I’ve upgraded tensorrt to verion 5.0.2.6 and now i can’t run the SampleONNXMnist sample also I can’t run my own onnx model and i got core dumped error  
you can see the error below:

ERROR: Network must have at least one output  
sample\_onnx\_mnist: sampleOnnxMNIST.cpp:64: void onnxToTRTModel(const string&, unsigned int, nvinfer1::IHostMemory\*&): Assertion `engine’ failed.  
Aborted (core dumped)

Can any one help me?

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [November 19, 2018, 4:17pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/2 "2018-11-19T16:17:19Z")

</div>

Hello,

What platform (windows/linux) are you on? I’m able to use the TensorRT 18.11 container and seems to run mnist sample fine.

```auto
NVIDIA Release 18.11 (build 817536)

NVIDIA TensorRT 5.0.2 (c) 2016-2018, NVIDIA CORPORATION. All rights reserved.
Container image (c) 2018, NVIDIA CORPORATION. All rights reserved.

https://developer.nvidia.com/tensorrt

To install Python sample dependencies, run /opt/tensorrt/python/python_setup.sh

root@2ca610df4db7:/workspace# ls
README.md tensorrt
root@2ca610df4db7:/workspace# cd tensorrt/
TensorRT-Release-Notes.pdf bin/ data/ doc/ python/ samples/
root@2ca610df4db7:/workspace# cd tensorrt/samples/
Makefile getDigits/ sampleFasterRCNN/ sampleINT8API/ sampleMNISTAPI/ sampleNMT/ sampleSSD/ trtexec/
Makefile.config python/ sampleGoogleNet/ sampleMLP/ sampleMovieLens/ sampleOnnxMNIST/ sampleUffMNIST/
common/ sampleCharRNN/ sampleINT8/ sampleMNIST/ sampleMovieLensMPS/ samplePlugin/ sampleUffSSD/
root@2ca610df4db7:/workspace# cd tensorrt/samples/sampleOnnxMNIST/
root@2ca610df4db7:/workspace/tensorrt/samples/sampleOnnxMNIST# ls
Makefile README sampleOnnxMNIST.cpp
root@2ca610df4db7:/workspace/tensorrt/samples/sampleOnnxMNIST# cat README
This sample demonstrates conversion of an MNIST network in ONNX format to
a TensorRT network. The network used in this sample can be found at https://github.com/onnx/models/tree/master/mnist
(model.onnx)
root@2ca610df4db7:/workspace/tensorrt/samples/sampleOnnxMNIST# make
../Makefile.config:5: CUDA_INSTALL_DIR variable is not specified, using /usr/local/cuda by default, use CUDA_INSTALL_DIR=<cuda_directory> to change.
../Makefile.config:8: CUDNN_INSTALL_DIR variable is not specified, using $CUDA_INSTALL_DIR by default, use CUDNN_INSTALL_DIR=<cudnn_directory> to change.
:
Compiling: sampleOnnxMNIST.cpp
Linking: ../../bin/sample_onnx_mnist_debug
:
Compiling: sampleOnnxMNIST.cpp
Linking: ../../bin/sample_onnx_mnist
# Copy every EXTRA_FILE of this sample to bin dir
root@2ca610df4db7:/workspace/tensorrt/samples/sampleOnnxMNIST# ../../bin/sample_onnx_mnist
----------------------------------------------------------------
Input filename: ../../data/mnist/mnist.onnx
ONNX IR version: 0.0.3
Opset version: 1
Producer name: CNTK
Producer version: 2.4
Domain:
Model version: 1
Doc string:
----------------------------------------------------------------
 ----- Parsing of ONNX model ../../data/mnist/mnist.onnx is Done ----

---------------------------

@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@#-:.-=@@@@@@@@@@@@@@
@@@@@%= . *@@@@@@@@@@@@@
@@@@% .:+%%% *@@@@@@@@@@@@@
@@@@+=#@@@@@# @@@@@@@@@@@@@@
@@@@@@@@@@@% @@@@@@@@@@@@@@
@@@@@@@@@@@: *@@@@@@@@@@@@@@
@@@@@@@@@@- .@@@@@@@@@@@@@@@
@@@@@@@@@: #@@@@@@@@@@@@@@@
@@@@@@@@: +*%#@@@@@@@@@@@@
@@@@@@@% :+*@@@@@@@@
@@@@@@@@#*+--.:: +@@@@@@
@@@@@@@@@@@@@@@@#=:. +@@@@@
@@@@@@@@@@@@@@@@@@@@ .@@@@@
@@@@@@@@@@@@@@@@@@@@#. #@@@@
@@@@@@@@@@@@@@@@@@@@# @@@@@
@@@@@@@@@%@@@@@@@@@@- +@@@@@
@@@@@@@@#-@@@@@@@@*. =@@@@@@
@@@@@@@@ .+%%%%+=. =@@@@@@@
@@@@@@@@ =@@@@@@@@
@@@@@@@@*=: :--*@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Prob 0 0.0000 Class 0:
 Prob 1 0.0000 Class 1:
 Prob 2 0.0000 Class 2:
 Prob 3 1.0000 Class 3: **********
 Prob 4 0.0000 Class 4:
 Prob 5 0.0000 Class 5:
 Prob 6 0.0000 Class 6:
 Prob 7 0.0000 Class 7:
 Prob 8 0.0000 Class 8:
 Prob 9 0.0000 Class 9:

root@2ca610df4db7:/workspace/tensorrt/samples/sampleOnnxMNIST#

```

---

<div class="post-metadata">

**Author:** ![soroor.shekarizade](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@soroor.shekarizade](https://forums.developer.nvidia.com/u/soroor.shekarizade)\
**Post date:** [November 19, 2018, 4:32pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/3 "2018-11-19T16:32:06Z")

</div>

Thank you for your response.  
I use Ubuntu 18 and upgrade tensorrt to 5.0.2.6 also I installed onnx-tensorrt to run the yolo-onnx model in python. now I want to run the yolo-onnx in c++ framework.  
It seems because of the onnxIR version, as u see it is ONNX IR version: 0.0.3  
and in the TensorRT-Developer-Guide was mentioned that only support ONNX IR version 7!  
And now. I don’t know how to convert the onnx model to version 7!

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [November 19, 2018, 4:56pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/4 "2018-11-19T16:56:23Z")

</div>

Hello,

In general, the ONNX Parser is designed to be backward compatible, therefore, a model file produced by an earlier version of ONNX exporter should not cause a problem.

In case where it’s not compatible, convert the earlier ONNX model file into a later supported version. For more information on this subject, see ONNX Model Opset Version Converter. [https://github.com/onnx/onnx/blob/master/docs/OpsetVersionConverter.md](https://github.com/onnx/onnx/blob/master/docs/OpsetVersionConverter.md)

For more details, please reference: [Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_onnx_c)

---

<div class="post-metadata">

**Author:** ![bpinaya](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bpinaya/32/12691_2.png) [@bpinaya](https://forums.developer.nvidia.com/u/bpinaya)\
**Post date:** [November 20, 2018, 9:13am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/5 "2018-11-20T09:13:25Z")

</div>

Hi there, trying to build the sampleOnnxMNIST sample on TensorRT5.0.2GA is not working for me. I’ve installed trt via the deb package, and built onnx-tensorrt from source via their git, but when trying to make the package I get the following:

```auto
../Makefile.config:5: CUDA_INSTALL_DIR variable is not specified, using /usr/local/cuda by default, use CUDA_INSTALL_DIR=<cuda_directory> to change.
../Makefile.config:8: CUDNN_INSTALL_DIR variable is not specified, using $CUDA_INSTALL_DIR by default, use CUDNN_INSTALL_DIR=<cudnn_directory> to change.
:
Compiling: sampleOnnxMNIST.cpp
sampleOnnxMNIST.cpp: In function ‘void onnxToTRTModel(const string&, unsigned int, nvinfer1::IHostMemory*&)’:
sampleOnnxMNIST.cpp:45:63: error: cannot convert ‘nvinfer1::INetworkDefinition’ to ‘nvinfer1::INetworkDefinition*’ for argument ‘1’ to ‘nvonnxparser::IParser* nvonnxparser::{anonymous}::createParser(nvinfer1::INetworkDefinition*, nvinfer1::ILogger&)’
     auto parser = nvonnxparser::createParser(*network, gLogger);
                                                               ^
../Makefile.config:172: recipe for target '../../bin/dchobj/sampleOnnxMNIST.o' failed
make: *** [../../bin/dchobj/sampleOnnxMNIST.o] Error 1

```

Any ideas what am I missing out?

---

<div class="post-metadata">

**Author:** ![soroor.shekarizade](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@soroor.shekarizade](https://forums.developer.nvidia.com/u/soroor.shekarizade)\
**Post date:** [November 20, 2018, 10:40am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/6 "2018-11-20T10:40:56Z")

</div>

Hi,  
it’s exactly my problem while I could run this sample on trt 5 before upgrading to 5.0.2!

---

<div class="post-metadata">

**Author:** ![bpinaya](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bpinaya/32/12691_2.png) [@bpinaya](https://forums.developer.nvidia.com/u/bpinaya)\
**Post date:** [November 20, 2018, 11:20am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/7 "2018-11-20T11:20:29Z")

</div>

Note on this, I still encounter this error, but in the TensorRT container it works, I’ve tried to find the dockerfile to see if I did any version mismatch but can’t find it.

The container has this TRT version:

```auto
ii tensorrt 5.0.2.6-1+cuda10.0 amd64 Meta package of TensorRT

```

The `pip list` output:

```auto
Package Version   
------------------- ----------
absl-py 0.6.1     
appdirs 1.4.3     
astor 0.7.1     
atomicwrites 1.2.1     
attrs 18.2.0    
certifi 2018.10.15
chardet 3.0.4     
decorator 4.3.0     
gast 0.2.0     
graphsurgeon 0.3.2     
grpcio 1.16.1    
h5py 2.8.0     
idna 2.7       
Keras-Applications 1.0.6     
Keras-Preprocessing 1.0.5     
Mako 1.0.7     
Markdown 3.0.1     
MarkupSafe 1.0       
more-itertools 4.3.0     
numpy 1.15.3    
onnx 1.3.0     
pathlib2 2.3.2     
Pillow 5.3.0     
pip 18.1      
pluggy 0.8.0     
protobuf 3.6.1     
py 1.7.0     
pycuda 2018.1.1  
pytest 3.9.3     
pytools 2018.5.2  
requests 2.20.1    
setuptools 40.5.0    
six 1.11.0    
tensorboard 1.12.0    
tensorflow 1.12.0    
tensorflow-gpu 1.12.0    
tensorrt 5.0.2.6   
termcolor 1.1.0     
torch 0.4.1     
torchvision 0.2.1     
typing 3.6.6     
typing-extensions 3.6.6     
uff 0.5.5     
urllib3 1.24.1    
Werkzeug 0.14.1    
wget 3.2       
wheel 0.32.2

```

And it’s on Python 3.5.2.

The python API doesn’t work so no luck for [GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt).  
I can just use the c libraries but that issue I’ve put above still bothers me on native.

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [November 20, 2018, 4:25pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/8 "2018-11-20T16:25:34Z")

</div>

Hello,

Is there a reason why installing [GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt) in addition to standalone TensorRT? TensorRT already includes an ONNX parser.

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [November 20, 2018, 4:29pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/9 "2018-11-20T16:29:44Z")

</div>

Hello regarding the dockerfile, from [NGC.nvidia.com](http://NGC.nvidia.com), you can look at the “layer” of the image you are pulling, which should give you an idea how the image is constructed.

---

<div class="post-metadata">

**Author:** ![bpinaya](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bpinaya/32/12691_2.png) [@bpinaya](https://forums.developer.nvidia.com/u/bpinaya)\
**Post date:** [November 21, 2018, 9:42am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/10 "2018-11-21T09:42:06Z")

</div>

Hi @NVES, I thought we needed to install onnx-tensorrt to have the onnx parser working. I uninstalled onnx-tensorrt, reinstalled TRT5GA and tried to build the samples and it’s still failing on the ONNX part.

```auto
Compiling: sampleINT8API.cpp
g++ -Wall -std=c++11 -I"/usr/local/cuda/include" -I"/usr/local/include" -I"../include" -I"../common" -I"/usr/local/cuda/include" -I"../../include" -D_REENTRANT -g -c -o ../../bin/dchobj/sampleINT8API.o sampleINT8API.cpp
sampleINT8API.cpp: In member function ‘bool sampleINT8API::build()’:
sampleINT8API.cpp:448:102: error: cannot convert ‘nvinfer1::INetworkDefinition’ to ‘nvinfer1::INetworkDefinition*’ for argument ‘1’ to ‘nvonnxparser::IParser* nvonnxparser::{anonymous}::createParser(nvinfer1::INetworkDefinition*, nvinfer1::ILogger&)’
     auto parser = SampleUniquePtr<nvonnxparser::IParser>(nvonnxparser::createParser(*network, gLogger));
                                                                                                      ^
../Makefile.config:172: recipe for target '../../bin/dchobj/sampleINT8API.o' failed
make[1]: *** [../../bin/dchobj/sampleINT8API.o] Error 1
make[1]: Leaving directory '/home/bpinaya/Documents/tensorrt/samples/sampleINT8API'
Makefile:37: recipe for target 'all' failed
make: *** [all] Error 2

```

I checked on the layers of the docker image but there is a script /nvidia/build-scripts/installTRT.sh that I can’t find.

On TRT 4, building the samples (onnx) worked from scratch, I think I’ll try again on a clean install but it’d be awesome if you release the dockerfiles eventually. I’ll update this if I have luck on a clean build.

---

<div class="post-metadata">

**Author:** ![bpinaya](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bpinaya/32/12691_2.png) [@bpinaya](https://forums.developer.nvidia.com/u/bpinaya)\
**Post date:** [November 21, 2018, 1:11pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/11 "2018-11-21T13:11:01Z")

</div>

Ok, so if I understand correctly: With tensorrt we can use the c++ parser API to parse ONNX models, without having to install [GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt) right?  
But for the python API we need to install it?  
From the container you linked I can start python and type:

```auto
import onnx

```

And that works of course but trying to import:

```auto
import onnx_tensorrt.backend as backend

```

results in:

```auto
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'onnx_tensorrt'

```

I guess I’m confused as of the purpose of [GitHub - onnx/onnx-tensorrt: ONNX-TensorRT: TensorRT backend for ONNX](https://github.com/onnx/onnx-tensorrt).

My goal is to take the network I have (trained in pytorch) latter parsed to ONNX to tensorrt.  
I’ve serialized networks before, but just from caffe, loading it from ONNX is where I seem to be struggling. Thanks for your time!!

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [November 21, 2018, 5:29pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/12 "2018-11-21T17:29:02Z")

</div>

Correct, you get the onnx parser with standard TRT install. For more details, reference [https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import\_onnx\_python](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_onnx_python)

---

<div class="post-metadata">

**Author:** ![bpinaya](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/bpinaya/32/12691_2.png) [@bpinaya](https://forums.developer.nvidia.com/u/bpinaya)\
**Post date:** [November 22, 2018, 9:09am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/13 "2018-11-22T09:09:50Z")

</div>

You are right, from a clean setup all the samples build correctly, I think it was an issue of onnx-tensorrt that messed things up.  
Thanks for the help!

---

<div class="post-metadata">

**Author:** ![599251371](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@599251371](https://forums.developer.nvidia.com/u/599251371)\
**Post date:** [December 6, 2018, 8:13am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/14 "2018-12-06T08:13:57Z")

</div>

I want to install keras on Nvidia Jetson Xavier,but the version of CUDA is 10.0. It’s a pity,I can’t install keras sucessfully.I don’t know how to install it on Xavier, and I have installed tensorflow sucessfully. Can anyone help me?

---

<div class="post-metadata">

**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\
**Post date:** [December 6, 2018, 6:48pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/15 "2018-12-06T18:48:15Z")

</div>

> [@](#):
>
> I want to install keras on Nvidia Jetson Xavier,but the version of CUDA is 10.0. It’s a pity,I can’t install keras sucessfully.I don’t know how to install it on Xavier, and I have installed tensorflow sucessfully. Can anyone help me?

Hello, Please post your question on [Jetson AGX Xavier - NVIDIA Developer Forums](https://devtalk.nvidia.com/default/board/326/jetson-agx-xavier/) , you’ll get support coverage there.

---

<div class="post-metadata">

**Author:** ![599251371](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@599251371](https://forums.developer.nvidia.com/u/599251371)\
**Post date:** [December 7, 2018, 1:59am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/16 "2018-12-07T01:59:00Z")

</div>

Thanks for your reply!

---

<div class="post-metadata">

**Author:** ![1763398711](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@1763398711](https://forums.developer.nvidia.com/u/1763398711)\
**Post date:** [August 6, 2019, 3:26am UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/17 "2019-08-06T03:26:54Z")

</div>

recipe for target ‘…/…/bin/sample\_mnist\_debug’ failed

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [October 12, 2021, 7:47pm UTC](https://forums.developer.nvidia.com/t/tensorrt-5-0-2-6-onnx-run-error/67578/18 "2021-10-12T19:47:59Z")

</div>

Closed on October 12, 2021, 7:47pm UTC


