Onnx runtime gpu on jetson nano in c++

user165035 · April 19, 2022, 12:45pm

I am trying to install onnx runtime gpu version for jetson nano as the link:
https://elinux.org/Jetson_Zoo#ONNX_Runtime
I successfully installed, but i could not fins these files
libonnxruntime.so and libonnxruntime.so.1.12.0 and also onnxruntime_c_api.h, so i couls not run the code and load the model

As onnx does not have any release for aarch64 gou version, i tried merging their onnxruntime-linux-aarch64-1.11.0.tgz and the built gpu of jetson zoo, but did not work. The onnxruntime-linux-aarch64 provied by onnx works on jetson without gpu and very slow

How can i get onnx runtime gpu with c++ in jetson?

AastaLLL · April 20, 2022, 2:39am

Hi,

The package is for python users.

We are checking the C++based library internally.
Will share more information with you later.

Thanks.

AastaLLL · April 20, 2022, 2:49am

Hi,

Based on the discussion below, it required some manual steps for the C++ library.
Please build it from the source and copy the required header file as the suggestion in the comment.

github.com/microsoft/onnxruntime

Build on Jetson Xavier NX does not produce c++ API include and libs

opened 06:12PM - 14 Oct 21 UTC

isra60

stale

**Describe the bug** I have a Jetson Xavier NX and I want to use the c++ API of… the onnxruntime, so instead of using the standard wheel from the Jetson Model Zoo (https://elinux.org/Jetson_Zoo#ONNX_Runtime) I need to build the entire onnxruntime again with the option of build the shared libs As reference I'm using this tutorial https://onnxruntime.ai/docs/build/eps.html#nvidia-jetson-tx1tx2nanoxavier **System information** Jetpack 4.6 L4T **To Reproduce** This is my command line to build onnxruntime Note the use of **--build_shared_lib** which I think is the proper way to obtain the include and libs for the c++ API > ./build.sh --config Release --update --build --parallel --build_wheel --build_shared_lib --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu --tensorrt_home /usr/lib/aarch64-linux-gnu After the long building, I have a path like this **onnxruntime/build/Linux/Release** but in that folder I don't found any reference to the C++ API I mean something like **include/onnxruntime/core/session/onnxruntime_cxx_api.h** Also, do you know if the wheel provided by NVIDIA is built with the TensorRT option??

Thanks.

user165035 · April 20, 2022, 8:47am

hi,

I am trying to build ONNX runtime from source now, but got this error

CMake Error at /usr/local/share/cmake-3.23/Modules/CMakeDetermineCUDACompiler.cmake:311 (message):
CMAKE_CUDA_ARCHITECTURES must be valid if set.
Call Stack (most recent call first):
CMakeLists.txt:1697 (enable_language)
as jetpack 4.61 is supposed to have cuda installed, any idea of this problem?

user165035 · April 20, 2022, 6:48pm

fixed by removing the checking of CMAKE_CUDA_ARCHITECTURES condition in line 311 of file CMakeDetermineCUDACompiler.cmake

system · May 11, 2022, 6:20am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.