JetPack 6.1 doesn't support OnnxRuntime

Autonomous Machines Jetson & Embedded Systems Jetson Orin NX
1

I have a fresh image of the 6.1 Jetpack installed, and I want to run the OnnxRuntime to do inferencing.

$ python
Python 3.11.11 (main, Jan  5 2025, 05:21:43) [GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import onnxruntime as ort

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "<stdin>", line 1, in <module>
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/onnxruntime/__init__.py", line 23, in <module>
    from onnxruntime.capi._pybind_state import ExecutionMode  # noqa: F401
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/onnxruntime/capi/_pybind_state.py", line 32, in <module>
    from .onnxruntime_pybind11_state import *  # noqa
Traceback (most recent call last):
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/numpy/core/_multiarray_umath.py", line 44, in __getattr__
    raise ImportError(msg)
ImportError: 
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.2.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/onnxruntime/__init__.py", line 58, in <module>
    raise import_capi_exception
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/onnxruntime/__init__.py", line 23, in <module>
    from onnxruntime.capi._pybind_state import ExecutionMode  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jetson/Documents/GlaDOS/.venv/lib/python3.11/site-packages/onnxruntime/capi/_pybind_state.py", line 32, in <module>
    from .onnxruntime_pybind11_state import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError

This is a bit weird, as from version 1.20 of ortruntime, the numpy v1 issue should be fixed.

Im using the onnxruntime_gpu-1.20.0-cp311-cp311-linux_aarch64.whl wheel from:
https://pypi.jetson-ai-lab.dev/jp6/cu126/onnxruntime-gpu/1.20.0

If I try a sanity check, and just use the base CPU version of the OnnxRuntime installed with Pip, I get this weird error!

/opt/rh/gcc-toolset-12/root/usr/include/c++/12/bits/stl_vector.h:1123: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _Tp = unsigned int; _Alloc = std::allocator<unsigned int>; reference = unsigned int&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
Aborted (core dumped)

Is there a working wheel for 6.1 available?

4

Hi,

Could you try to downgrade NumPy to version<2.0 to see if it can work?
Thanks.

5

@AastaLLL

Yes, this seems to fix the issue.

This is a bit weird, as this issue should be fixed:

6

Hi,

Not sure if this is related to the environment when generating the ONNXRuntime package.
If you prefer NumPy 2.x, could you try to build ONNXRuntime from the source to see if it can work?

Thanks.