Installing ONNX library on my Jetson Xavier

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi everyone,

After bein amazed by the performance of my SSD-inception-v2 model optimized with TRT and INT8-Calibration, I wanted to go back from where I started and so try to get up to those performance with some YOLO models. I’ve seen different topics talking about optimizing YOLOV3 with TensorRT but it seems to be harder than I thought. I was working on this repo : GitHub - jkjung-avt/tensorrt_demos: TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet and when I wanted to test out the yolov3_onnx project, I saw that ONNX was not installed on my Xavier but the problem is none of my pip3 install command are working. Does anyone here have a way to install onnx on an embeded platform such as my Xavier ?

Have a nice day :)

2

What errors do you get when trying to install the onnx package? This is what I do to install it:

$ sudo apt-get install python3-pip libprotoc-dev protobuf-compiler
$ pip3 install onnx --verbose
4 Likes
3

Don’t really know which part is the real problem here since it seems like there is different problems … But all linked to protobuf at a first sight I would say ? Sorry for the long error list

Building wheels for collected packages: onnx
  Building wheel for onnx (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9tzfjt08/onnx/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9tzfjt08/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-20oatpbw
       cwd: /tmp/pip-install-9tzfjt08/onnx/
  Complete output (180 lines):
  fatal: not a git repository (or any of the parent directories): .git
  running bdist_wheel
  running build
  running build_py
  running create_version
  running cmake_build
  -- The C compiler identification is GNU 7.4.0
  -- The CXX compiler identification is GNU 7.4.0
  -- Check for working C compiler: /usr/bin/cc
  -- Check for working C compiler: /usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/bin/c++
  -- Check for working CXX compiler: /usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Looking for pthread.h
  -- Looking for pthread.h - found
  -- Looking for pthread_create
  -- Looking for pthread_create - not found
  -- Looking for pthread_create in pthreads
  -- Looking for pthread_create in pthreads - not found
  -- Looking for pthread_create in pthread
  -- Looking for pthread_create in pthread - found
  -- Found Threads: TRUE
  CMake Warning at /usr/share/cmake-3.10/Modules/FindProtobuf.cmake:455 (message):
    Protobuf compiler version 3.8.0 doesn't match library version 3.0.0
  Call Stack (most recent call first):
    CMakeLists.txt:127 (find_package)
  
  
  -- Found Protobuf: /usr/lib/aarch64-linux-gnu/libprotobuf.so;-lpthread (found version "3.0.0")
  Generated: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto
  Generated: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
  CMake Warning at CMakeLists.txt:394 (find_package):
    By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
    asked CMake to find a package configuration file provided by "pybind11",
    but CMake did not find one.
  
    Could not find a package configuration file provided by "pybind11"
    (requested version 2.2) with any of the following names:
  
      pybind11Config.cmake
      pybind11-config.cmake
  
    Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
    "pybind11_DIR" to a directory containing one of the above files.  If
    "pybind11" provides a separate development package or SDK, be sure it has
    been installed.
  
  
  --
  -- ******** Summary ********
  --   CMake version         : 3.10.2
  --   CMake command         : /usr/bin/cmake
  --   System                : Linux
  --   C++ compiler          : /usr/bin/c++
  --   C++ compiler version  : 7.4.0
  --   CXX flags             :  -Wnon-virtual-dtor
  --   Build type            : Release
  --   Compile definitions   :
  --   CMAKE_PREFIX_PATH     :
  --   CMAKE_INSTALL_PREFIX  : /usr/local
  --   CMAKE_MODULE_PATH     :
  --
  --   ONNX version          : 1.6.0
  --   ONNX NAMESPACE        : onnx
  --   ONNX_BUILD_TESTS      : OFF
  --   ONNX_BUILD_BENCHMARKS : OFF
  --   ONNX_USE_LITE_PROTO   : OFF
  --   ONNXIFI_DUMMY_BACKEND : OFF
  --   ONNXIFI_ENABLE_EXT    : OFF
  --
  --   Protobuf compiler     : /usr/local/bin/protoc
  --   Protobuf includes     : /usr/include
  --   Protobuf libraries    : /usr/lib/aarch64-linux-gnu/libprotobuf.so;-lpthread
  --   BUILD_ONNX_PYTHON     : ON
  --     Python version      :
  --     Python executable   : /usr/bin/python3
  --     Python includes     : /usr/include/python3.6m
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build
  Scanning dependencies of target gen_onnx_proto
  Scanning dependencies of target onnxifi_dummy
  Scanning dependencies of target onnxifi_loader
  [  1%] Running gen_proto.py on onnx/onnx.in.proto
  [  3%] Building C object CMakeFiles/onnxifi_dummy.dir/onnx/onnxifi_dummy.c.o
  [  4%] Building C object CMakeFiles/onnxifi_loader.dir/onnx/onnxifi_loader.c.o
  /tmp/pip-install-9tzfjt08/onnx/onnx/onnxifi_dummy.c: In function ‘onnxGetExtensionFunctionAddress’:
  /tmp/pip-install-9tzfjt08/onnx/onnx/onnxifi_dummy.c:173:21: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
             *function = &onnxGetExtensionFunctionAddress;
                       ^
  /tmp/pip-install-9tzfjt08/onnx/onnx/onnxifi_dummy.c:176:21: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
             *function = &onnxSetIOAndRunGraph;
                       ^
  Processing /tmp/pip-install-9tzfjt08/onnx/onnx/onnx.in.proto
  Writing /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto
  Writing /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto3
  generating /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx_pb.py
  [  6%] Linking C static library libonnxifi_loader.a
  [  8%] Running C++ protocol buffer compiler on /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto
  [  9%] Linking C shared library libonnxifi_dummy.so
  [  9%] Built target onnxifi_loader
  Scanning dependencies of target onnxifi_wrapper
  [ 11%] Building C object CMakeFiles/onnxifi_wrapper.dir/onnx/onnxifi_wrapper.c.o
  [ 11%] Built target onnxifi_dummy
  Failed to generate mypy stubs: No module named google.protobuf.descriptor_pb2
  [ 11%] Built target gen_onnx_proto
  [ 13%] Running gen_proto.py on onnx/onnx-operators.in.proto
  Processing /tmp/pip-install-9tzfjt08/onnx/onnx/onnx-operators.in.proto
  Writing /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
  Writing /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto3
  generating /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx_operators_pb.py
  [ 14%] Running C++ protocol buffer compiler on /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
  Failed to generate mypy stubs: No module named google.protobuf.descriptor_pb2
  Scanning dependencies of target onnx_proto
  [ 18%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o
  [ 18%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o
  [ 19%] Linking C shared module libonnxifi.so
  In file included from /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.pb.cc:4:0:
  /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
   #include <google/protobuf/port_def.inc>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  In file included from /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.pb.cc:4:0:
  /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
   #include <google/protobuf/port_def.inc>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  CMakeFiles/onnx_proto.dir/build.make:108: recipe for target 'CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o' failed
  make[2]: *** [CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o] Error 1
  make[2]: *** Waiting for unfinished jobs....
  CMakeFiles/onnx_proto.dir/build.make:84: recipe for target 'CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o' failed
  make[2]: *** [CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o] Error 1
  CMakeFiles/Makefile2:242: recipe for target 'CMakeFiles/onnx_proto.dir/all' failed
  make[1]: *** [CMakeFiles/onnx_proto.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....
  [ 19%] Built target onnxifi_wrapper
  Makefile:129: recipe for target 'all' failed
  make: *** [all] Error 2
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 334, in <module>
      'backend-test-tools = onnx.backend.test.cmd_tools:main',
    File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 144, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 204, in run
      self.run_command('build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 209, in run
      self.run_command('cmake_build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 203, in run
      subprocess.check_call(build_args)
    File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/usr/bin/cmake', '--build', '.', '--', '-j', '8']' returned non-zero exit status 2.
  ----------------------------------------
  ERROR: Failed building wheel for onnx
  Running setup.py clean for onnx
Failed to build onnx
Installing collected packages: onnx
    Running setup.py install for onnx ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-9tzfjt08/onnx/setup.py'"'"'; __file__='"'"'/tmp/pip-install-9tzfjt08/onnx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2plkpoyb/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/user/.local/include/python3.6m/onnx
         cwd: /tmp/pip-install-9tzfjt08/onnx/
    Complete output (125 lines):
    fatal: not a git repository (or any of the parent directories): .git
    running install
    running build
    running build_py
    running create_version
    running cmake_build
    CMake Warning at /usr/share/cmake-3.10/Modules/FindProtobuf.cmake:455 (message):
      Protobuf compiler version 3.8.0 doesn't match library version 3.0.0
    Call Stack (most recent call first):
      CMakeLists.txt:127 (find_package)
    
    
    Generated: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.proto
    Generated: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.proto
    CMake Warning at CMakeLists.txt:394 (find_package):
      By not providing "Findpybind11.cmake" in CMAKE_MODULE_PATH this project has
      asked CMake to find a package configuration file provided by "pybind11",
      but CMake did not find one.
    
      Could not find a package configuration file provided by "pybind11"
      (requested version 2.2) with any of the following names:
    
        pybind11Config.cmake
        pybind11-config.cmake
    
      Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
      "pybind11_DIR" to a directory containing one of the above files.  If
      "pybind11" provides a separate development package or SDK, be sure it has
      been installed.
    
    
    --
    -- ******** Summary ********
    --   CMake version         : 3.10.2
    --   CMake command         : /usr/bin/cmake
    --   System                : Linux
    --   C++ compiler          : /usr/bin/c++
    --   C++ compiler version  : 7.4.0
    --   CXX flags             :  -Wnon-virtual-dtor
    --   Build type            : Release
    --   Compile definitions   :
    --   CMAKE_PREFIX_PATH     :
    --   CMAKE_INSTALL_PREFIX  : /usr/local
    --   CMAKE_MODULE_PATH     :
    --
    --   ONNX version          : 1.6.0
    --   ONNX NAMESPACE        : onnx
    --   ONNX_BUILD_TESTS      : OFF
    --   ONNX_BUILD_BENCHMARKS : OFF
    --   ONNX_USE_LITE_PROTO   : OFF
    --   ONNXIFI_DUMMY_BACKEND : OFF
    --   ONNXIFI_ENABLE_EXT    : OFF
    --
    --   Protobuf compiler     : /usr/local/bin/protoc
    --   Protobuf includes     : /usr/include
    --   Protobuf libraries    : /usr/lib/aarch64-linux-gnu/libprotobuf.so;-lpthread
    --   BUILD_ONNX_PYTHON     : ON
    --     Python version      :
    --     Python executable   : /usr/bin/python3
    --     Python includes     : /usr/include/python3.6m
    -- Configuring done
    -- Generating done
    -- Build files have been written to: /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build
    [  9%] Built target onnxifi_loader
    [  9%] Built target onnxifi_dummy
    [  9%] Built target gen_onnx_proto
    [ 14%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o
    [ 14%] Building CXX object CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o
    [ 16%] Built target onnxifi_wrapper
    In file included from /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.pb.cc:4:0:
    /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-ml.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
     #include <google/protobuf/port_def.inc>
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    In file included from /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.pb.cc:4:0:
    /tmp/pip-install-9tzfjt08/onnx/.setuptools-cmake-build/onnx/onnx-operators-ml.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
     #include <google/protobuf/port_def.inc>
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    CMakeFiles/onnx_proto.dir/build.make:84: recipe for target 'CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o' failed
    make[2]: *** [CMakeFiles/onnx_proto.dir/onnx/onnx-ml.pb.cc.o] Error 1
    make[2]: *** Waiting for unfinished jobs....
    CMakeFiles/onnx_proto.dir/build.make:108: recipe for target 'CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o' failed
    make[2]: *** [CMakeFiles/onnx_proto.dir/onnx/onnx-operators-ml.pb.cc.o] Error 1
    CMakeFiles/Makefile2:242: recipe for target 'CMakeFiles/onnx_proto.dir/all' failed
    make[1]: *** [CMakeFiles/onnx_proto.dir/all] Error 2
    Makefile:129: recipe for target 'all' failed
    make: *** [all] Error 2
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 334, in <module>
        'backend-test-tools = onnx.backend.test.cmd_tools:main',
      File "/usr/local/lib/python3.6/dist-packages/setuptools/__init__.py", line 144, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.6/dist-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 209, in run
        self.run_command('cmake_build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-9tzfjt08/onnx/setup.py", line 203, in run
        subprocess.check_call(build_args)
      File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/usr/bin/cmake', '--build', '.', '--', '-j', '8']' returned non-zero exit status 2.
4

It looks like it is missing pybind11 - try running this first:

$ pip3 install pybind11
5

Done but still a lot of errors … Does this one means something to you ?

fatal error: google/protobuf/port_def.inc: No such file or directory

Ps: Thanks for the quick reply :)

6

Hmm what package version of libprotoc-dev and protobuf-compiler do you have installed? Mine shows: 3.0.0.9.1ubuntu1

From this similar issue, it appears related to protobuf version: https://github.com/BVLC/caffe/issues/6774

7

Hi,

Same for me … :

libprotoc-dev is already the newest version (3.0.0-9.1ubuntu1).
protobuf-compiler is already the newest version (3.0.0-9.1ubuntu1).

Going to check your link, thanks

1 Like
8

Here you find ONNX compiled for Xavier: Release onnxruntime GPU TensorRT · domcross/Jetson-Xavier-AGX-stuff · GitHub

Note: this is my unofficial build

9

Hi, thanks for your answer :) I pip installed your wheel but my “import onnx” still doesn’t work… Anything else to do ?

10

Maybe some requirements are missing that I already have installed for the build-process?

As far as I can tell at minimum you need:
six (1.14.0)
numpy (1.18.1)
protobuf (3.11.3)

11

Built protobuf from source with it’s latest release ( 3.11.4) and it worked, thanks for your help !

12

Now I got it- my build is “onnxruntime” only while you want to use “onnx” - sorry for the confusion.

13

Moreover, they both can be also built from sources:
onnx from GitHub - onnx/onnx: Open standard for machine learning interoperability

onnxruntime - from here using git checkout ebf2374

Reference: pdf slides