Convert onnx to tensorrt plain

I want to convert from onnxt to tensorrt plain like mentioned this https://devblogs.nvidia.com/nvidia-serves-deep-learning-inference/ in the section ‘ONNX Models’. I tried install docker from this article https://github.com/onnx/onnx-tensorrt

cp /path/to/TensorRT-3.0.*.tar.gz .
docker build -t onnx_tensorrt .

I download TensorRT-3.0.*.tar.gz . from this source https://developer.nvidia.com/nvidia-tensorrt-download

but during installation I get follow error

CMakeFiles/nvonnxparser_plugin.dir/build.make:77: recipe for target 'CMakeFiles/nvonnxparser_plugin.dir/nvonnxparser_plugin_generated_Split.cu.o' failed
nvcc fatal   : Unsupported gpu architecture 'compute_75'
CMake Error at nvonnxparser_plugin_generated_Split.cu.o.cmake:207 (message):
  Error generating
  /opt/onnx2trt/build/CMakeFiles/nvonnxparser_plugin.dir//./nvonnxparser_plugin_generated_Split.cu.o


make[2]: *** [CMakeFiles/nvonnxparser_plugin.dir/nvonnxparser_plugin_generated_ResizeNearest.cu.o] Error 1
make[2]: *** [CMakeFiles/nvonnxparser_plugin.dir/nvonnxparser_plugin_generated_Split.cu.o] Error 1
CMakeFiles/Makefile2:180: recipe for target 'CMakeFiles/nvonnxparser_plugin.dir/all' failed
make[1]: *** [CMakeFiles/nvonnxparser_plugin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
The command '/bin/sh -c rm -rf build/ &&     mkdir build &&     cd build &&     cmake .. &&     make -j$(nproc) &&     make install &&     ldconfig &&     cd .. &&     python setup.py build &&     python setup.py install &&     rm -rf ./build/' returned a non-zero code: 2

Also I tried convert from onnx to caffe2, and also have error mentioned in this article https://devtalk.nvidia.com/default/topic/1046420/tensorrt/the-tensortrt-server-can-t-run-a-pretrain-model-after-converting-from-onnx-to-caffe2/

P.S. I have this error on macos and ubuntu 16.04

y.glushenkov@ml-test-env:/tmp/models_example$ nvidia-smi
Fri Jan 18 07:43:02 2019       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.78       Driver Version: 410.78       CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:00:05.0 Off |                  N/A |
| 35%   55C    P2    76W / 250W |   7344MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      7293      C   trtserver                                    841MiB |
|    0     18379      C   /home/a.eryomin/anaconda3/bin/python        6493MiB |
+-----------------------------------------------------------------------------+

P.P.S gpu info mentioned above from ubuntu

y.glushenkov@ml-test-env:/tmp/models_example$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

And also I have the same error on another server

/y.glushenkov@ml-dev-env:~$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
y.glushenkov@ml-dev-env:~$ nvidia-smi
Fri Jan 18 08:05:12 2019       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.54                 Driver Version: 396.54                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:00:05.0 Off |                  N/A |
| 26%   47C    P2    62W / 250W |    715MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:00:06.0 Off |                  N/A |
| 21%   30C    P8    18W / 250W |   1231MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 108...  Off  | 00000000:00:07.0 Off |                  N/A |
| 21%   21C    P8    16W / 250W |   7603MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     31014      C   java                                         701MiB |
|    1     23532      C   /home/a.eryomin/anaconda3/bin/python        1221MiB |
|    2      3746      C   /home/a.eryomin/anaconda3/bin/python        7593MiB |
+-----------------------------------------------------------------------------+
y.glushenkov@ml-dev-env:~$

On the both servers GPU are 1080TI

I solve this issue by removing “75” (line 120) from CMakeLists.txt :-)

e.g.

list(APPEND GPU_ARCHS
    35
    53
    61
    70
#     75
    )

Hi zwzhuang,

now I have next error:)

/usr/bin/ld: warning: libcublas.so.9.1, needed by /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libcudart.so.9.1, needed by /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: onnx2trt: hidden symbol `cudaMemcpyAsync' in /usr/local/cuda/lib64/libcudart_static.a(libcudart_static.a.o) is referenced by DSO
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [onnx2trt] Error 1
CMakeFiles/onnx2trt.dir/build.make:105: recipe for target 'onnx2trt' failed
CMakeFiles/Makefile2:295: recipe for target 'CMakeFiles/onnx2trt.dir/all' failed
make[1]: *** [CMakeFiles/onnx2trt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Linking CXX shared library libtrt_onnxify.so
[100%] Built target trt_onnxify
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
The command '/bin/sh -c rm -rf build/ &&     mkdir build &&     cd build &&     cmake .. &&     make -j$(nproc) &&     make install &&     ldconfig &&     cd .. &&     python setup.py build &&     python setup.py install &&     rm -rf ./build/' returned a non-zero code: 2

Do you know how can I fixed it?

This issues is still actual.