RuntimeError installing Torch TensorRT on Jetson Xavier NX

I am trying to install torch-tensorrt for Python on my Jetson Xavier NX with Jetpack 5.0.2. Followed the instructions and have successfully run everything up until python3 setup.py install --use-cxx11-abi which ran all the way until it got to “Allowing ninja to set a default number of workers” which it hung on for quite some time until eventually erroring out with the output listed below. Any advice would be much appreciated.

Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/4] c++ -MMD -MF /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_classes.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -I/home/nvidia/TensorRT/pytorch_tensorrt/csrc -I/home/nvidia/TensorRT/pytorch_tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TRTorch/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-Torch-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-tensorrt/external/tensorrt/include -I/home/nvidia/TensorRT/py/../ -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/TH -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.4/include -I/usr/include/python3.8 -c -c /home/nvidia/TensorRT/py/torch_tensorrt/csrc/tensorrt_classes.cpp -o /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_classes.o -Wno-deprecated -Wno-deprecated-declarations -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
FAILED: /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_classes.o
c++ -MMD -MF /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_classes.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -I/home/nvidia/TensorRT/pytorch_tensorrt/csrc -I/home/nvidia/TensorRT/pytorch_tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TRTorch/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-Torch-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-tensorrt/external/tensorrt/include -I/home/nvidia/TensorRT/py/../ -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/TH -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.4/include -I/usr/include/python3.8 -c -c /home/nvidia/TensorRT/py/torch_tensorrt/csrc/tensorrt_classes.cpp -o /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_classes.o -Wno-deprecated -Wno-deprecated-declarations -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[2/4] c++ -MMD -MF /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_backend.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -I/home/nvidia/TensorRT/pytorch_tensorrt/csrc -I/home/nvidia/TensorRT/pytorch_tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TRTorch/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-Torch-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-tensorrt/external/tensorrt/include -I/home/nvidia/TensorRT/py/../ -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/TH -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.4/include -I/usr/include/python3.8 -c -c /home/nvidia/TensorRT/py/torch_tensorrt/csrc/tensorrt_backend.cpp -o /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/tensorrt_backend.o -Wno-deprecated -Wno-deprecated-declarations -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
[3/4] c++ -MMD -MF /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/register_tensorrt_classes.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -I/home/nvidia/TensorRT/pytorch_tensorrt/csrc -I/home/nvidia/TensorRT/pytorch_tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TRTorch/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-Torch-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-tensorrt/external/tensorrt/include -I/home/nvidia/TensorRT/py/../ -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/TH -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.4/include -I/usr/include/python3.8 -c -c /home/nvidia/TensorRT/py/torch_tensorrt/csrc/register_tensorrt_classes.cpp -o /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/register_tensorrt_classes.o -Wno-deprecated -Wno-deprecated-declarations -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
[4/4] c++ -MMD -MF /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/torch_tensorrt_py.o.d -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -UNDEBUG -I/home/nvidia/TensorRT/pytorch_tensorrt/csrc -I/home/nvidia/TensorRT/pytorch_tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TRTorch/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-Torch-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-TensorRT/external/tensorrt/include -I/home/nvidia/TensorRT/py/../bazel-tensorrt/external/tensorrt/include -I/home/nvidia/TensorRT/py/../ -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/TH -I/home/nvidia/.local/lib/python3.8/site-packages/torch/include/THC -I/usr/local/cuda-11.4/include -I/usr/include/python3.8 -c -c /home/nvidia/TensorRT/py/torch_tensorrt/csrc/torch_tensorrt_py.cpp -o /home/nvidia/TensorRT/py/build/temp.linux-aarch64-3.8/torch_tensorrt/csrc/torch_tensorrt_py.o -Wno-deprecated -Wno-deprecated-declarations -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1013"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=1 -std=c++14
In file included from /home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/utils/python_arg_parser.h:63,
                 from /home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/jit/python/pybind_utils.h:26,
                 from /home/nvidia/TensorRT/py/torch_tensorrt/csrc/torch_tensorrt_py.cpp:9:
/home/nvidia/.local/lib/python3.8/site-packages/torch/include/torch/csrc/utils/python_strings.h:106:19: warning: ‘pybind11::object PyObject_FastGetAttrString(PyObject*, const char*)’ defined but not used [-Wunused-function]
  106 | static py::object PyObject_FastGetAttrString(PyObject* obj, const char* name) {
      |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1865, in _run_ninja_build
    subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "setup.py", line 368, in <module>
    setup(
  File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 144, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 222, in run
    install.run(self)
  File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 65, in run
    orig.install.run(self)
  File "/usr/lib/python3.8/distutils/command/install.py", line 589, in run
    self.run_command('build')
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/home/nvidia/.local/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 778, in build_extensions
    build_ext.build_extensions(self)
  File "/home/nvidia/.local/lib/python3.8/site-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 208, in build_extension
    _build_ext.build_extension(self, ext)
  File "/usr/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 593, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1544, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/home/nvidia/.local/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1881, in _run_ninja_build
    raise RuntimeError(message) from e

Hi,

Could you share how you compile it in detail?
In general, please follow the instructions mentioned in the below link to build Torch-TRT.

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.