Failed running build_wheel.py for TensorRT-LLM

ZXZhang · January 9, 2026, 3:55am

I tried to deploy TensorRT-LLM according to the document below.

I ran into some problems when I reached 1.3.

I tried to solve them, but I couldn’t fix the issues when running the build_wheel.py.

I tried the following：

Enter this website and choose v0.12.0-jetson.

Download the zip and copy it to my device.

unzip TensorRT-LLM-0.12.0-jetson.zip
mv TensorRT-LLM-0.12.0-jetson TensorRT-LLM
cd TensorRT-LLM
git init
git remote add origin https://github.com/NVIDIA/TensorRT-LLM.git
git fetch origin v0.12.0-jetson
git checkout FETCH_HEAD --force
git submodule update --init --recursive

sudo cp /etc/apt/sources.list.d/nvidia-l4t-apt-source.list /etc/apt/sources.list.d/nvidia-l4t-apt-source.list.bak

echo "deb https://repo.download.nvidia.com/jetson/common r36.4 main" | sudo tee /etc/apt/sources.list.d/nvidia-l4t-apt-source.list

echo "deb https://repo.download.nvidia.com/jetson/t234 r36.4 main" | sudo tee -a /etc/apt/sources.list.d/nvidia-l4t-apt-source.list

sudo apt update
sudo apt install -y tensorrt libnvinfer-dev libnvinfer-plugin-dev libcudnn9-dev
sudo apt install -y cmake build-essential ccache
echo "export PATH=/usr/local/cuda/bin:\$PATH" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=/usr/local/cuda/lib64:\$LD_LIBRARY_PATH" >> ~/.bashrc
source ~/.bashrc

python3 scripts/build_wheel.py --clean --cuda_architectures 87 -DENABLE_MULTI_DEVICE=0 -DTENSORRT_ROOT=/usr --build_type Release --benchmarks --use_ccache

My error when running the build_wheel.py:

collect2: error: ld returned 1 exit status
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'
gmake[3]: *** [benchmarks/CMakeFiles/bertBenchmark.dir/build.make:114: benchmarks/bertBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3957: benchmarks/CMakeFiles/bertBenchmark.dir/all] Error 2
gmake[2]: *** Waiting for unfinished jobs....
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IConcatenationLayer::setAxis(int)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createNetwork()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createNetwork()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)'
collect2: error: ld returned 1 exit status
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'
gmake[3]: *** [benchmarks/CMakeFiles/gptSessionBenchmark.dir/build.make:114: benchmarks/gptSessionBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3917: benchmarks/CMakeFiles/gptSessionBenchmark.dir/all] Error 2
collect2: error: ld returned 1 exit status
gmake[3]: *** [benchmarks/CMakeFiles/gptManagerBenchmark.dir/build.make:114: benchmarks/gptManagerBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3997: benchmarks/CMakeFiles/gptManagerBenchmark.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3884: benchmarks/CMakeFiles/benchmarks.dir/rule] Error 2
gmake: *** [Makefile:1154: benchmarks] Error 2
Traceback (most recent call last):
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 412, in <module>
    main(**vars(args))
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 206, in main
    build_run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake --build . --config Release --parallel 8 --target tensorrt_llm nvinfer_plugin_tensorrt_llm th_common bindings benchmarks  executorWorker  ' returned non-zero exit status 2.

Environment：

TensorRT Version: 10.3.0.30
CUDA Version: 12.6, V12.6.68
Operating System + Version: jetpack (6.2.1)
Python Version (if applicable): 3.10.12

Thanks to anyone who can help solve my problem.

carolyuu · January 9, 2026, 4:18am

*** Please note that this reply is generated by LLM automatically ***

Based on the provided context, it appears that the question is asking about the deployment of TensorRT LLM on Jetson Orin devices. However, the context does not provide specific information about TensorRT LLM deployment on Jetson Orin devices.

From the provided context, we can infer that Jetson Orin devices are supported by NVIDIA’s JetPack SDK, which includes the TensorRT library. TensorRT is a software development kit (SDK) that provides a set of tools and libraries for optimizing and deploying deep learning models on NVIDIA GPUs.

The provided link to the NVIDIA/TensorRT-LLM GitHub repository suggests that TensorRT LLM is a Python library that provides an easy-to-use API for defining and deploying large language models (LLMs) on NVIDIA GPUs. The library also includes components for creating Python and C runtimes that can orchestrate the inference execution in a performant way.

To deploy TensorRT LLM on Jetson Orin devices, you would likely need to:

Install the JetPack SDK on your Jetson Orin device.
Install the TensorRT LLM library using pip or another package manager.
Define and train your LLM model using the TensorRT LLM API.
Optimize and deploy the model on your Jetson Orin device using the TensorRT library.

However, without more specific information about the deployment process, it is difficult to provide a more detailed answer.

Comparison with other Jetson devices:

The provided context does not provide specific information about the deployment of TensorRT LLM on other Jetson devices. However, based on the fact that Jetson Orin devices are supported by the JetPack SDK, which includes the TensorRT library, it is likely that TensorRT LLM can also be deployed on other Jetson devices that are supported by the JetPack SDK.

In particular, the Jetson AGX Orin series, Jetson Orin NX series, and Jetson Orin Nano series are all supported by the JetPack SDK, and therefore may be compatible with the TensorRT LLM library. However, the specific deployment process and performance characteristics may vary depending on the device and its configuration.

Missing information:

To provide a more detailed answer, more information is needed about the specific deployment process and requirements for TensorRT LLM on Jetson Orin devices. This may include information about the specific Jetson device model, the version of the JetPack SDK and TensorRT library, and any specific configuration or optimization requirements for the deployment.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

AastaLLL · January 9, 2026, 8:10am

Hi

Could you try the container shared in the link below?

Thanks.

ZXZhang · January 9, 2026, 8:28am

Of course, I referred to the first link, but as I mentioned above, it didn’t work.

ZXZhang · January 12, 2026, 1:58am

I tried 1.3 from the link again and encountered the following error:

My Command：

git config --global http.postBuffer 524288000
git clone https://github.com/NVIDIA/TensorRT-LLM.git
cd TensorRT-LLM
git checkout v0.12.0-jetson
git lfs pull
python3 scripts/build_wheel.py --clean --cuda_architectures 87 -DENABLE_MULTI_DEVICE=0 --build_type Release --benchmarks --use_ccache

My Error：

/usr/bin/ld: warning: libnvdla_compiler.so, needed by /usr/lib/aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libnvdla_compiler.so, needed by /usr/lib/aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libnvdla_compiler.so, needed by /usr/lib/aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setUseSoftMaxOptz(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setUseSoftMaxOptz(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setUseSoftMaxOptz(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReshape(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReshape(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReshape(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addSlice(nvdla::ITensor*, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::SliceLayerMode)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addSlice(nvdla::ITensor*, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::SliceLayerMode)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addSlice(nvdla::ITensor*, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::SliceLayerMode)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'
/usr/bin/ld: /usr/bin/ld/usr/lib/aarch64-linux-gnu/libnvinfer.so: : undefined reference to `/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IConcatenationLayer::setAxis(int)nvdla::IConcatenationLayer::setAxis(int)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IConcatenationLayer::setAxis(int)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld/usr/bin/ld: : /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `/usr/bin/ldnvdla::destroyWisdom(nvdla::IWisdom*): '
/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'
/usr/bin/ld/usr/bin/ld: : /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `: undefined reference to `nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)'
/usr/bin/ld/usr/bin/ld: : /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld/usr/bin/ld: : /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `: undefined reference to `nvdla::createNetwork()nvdla::createNetwork()'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createNetwork()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `/usr/bin/ld: nvdla::IProfile::setCanGenerateLayerwiseStats(bool)/usr/lib/aarch64-linux-gnu/libnvinfer.so'
: undefined reference to `nvdla::IProfile::setCanGenerateLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/bin/ld: undefined reference to `: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)nvdla::ISoftMaxLayer::setAxis(int)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)'
/usr/bin/ld/usr/bin/ld: : /usr/lib/aarch64-linux-gnu/libnvinfer.so/usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'
'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
gmake[3]: *** [benchmarks/CMakeFiles/bertBenchmark.dir/build.make:114: benchmarks/bertBenchmark] Error 1
gmake[3]: *** [benchmarks/CMakeFiles/gptSessionBenchmark.dir/build.make:114: benchmarks/gptSessionBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3957: benchmarks/CMakeFiles/bertBenchmark.dir/all] Error 2
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [CMakeFiles/Makefile2:3917: benchmarks/CMakeFiles/gptSessionBenchmark.dir/all] Error 2
gmake[3]: *** [benchmarks/CMakeFiles/gptManagerBenchmark.dir/build.make:114: benchmarks/gptManagerBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3997: benchmarks/CMakeFiles/gptManagerBenchmark.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3884: benchmarks/CMakeFiles/benchmarks.dir/rule] Error 2
gmake: *** [Makefile:1154: benchmarks] Error 2
Traceback (most recent call last):
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 412, in <module>
    main(**vars(args))
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 206, in main
    build_run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake --build . --config Release --parallel 8 --target tensorrt_llm nvinfer_plugin_tensorrt_llm th_common bindings benchmarks  executorWorker  ' returned non-zero exit status 2.
evoc@tegra-ubuntu:~/TensorRT-LLM$

ZXZhang · January 12, 2026, 5:58am

OK，I think I understand what the problem is.

The first step ：Installing CUDA, cuDNN and TensorRT is wrong.

My JetPack version is JetPack 6.2.

And JetPack 6.2 should includes CUDA 12.6, TensorRT 10.3, cuDNN 9.3.

But the JetPack I installed didn’t include them.

Can you give me their installation commands?

AastaLLL · January 12, 2026, 7:03am

Hi,

Is the container an option for you?
If yes, could you share which kind of error you met with the container?

Thanks.

ZXZhang · January 12, 2026, 7:09am

Just like I said above, maybe CUDA 12.6, TensorRT 10.3 and cuDNN 9.3 are not installed correctly.

Can you give me their installation commands?

Thanks!

ZXZhang · January 13, 2026, 12:50am

I think I found the right command：

sudo apt update -y 

# install CUDA12.6+cuDNN9.3+TensorRT10.3
sudo apt install -y cuda-toolkit-12-6 libcudnn9-cuda-12 libcudnn9-dev-cuda-12 tensorrt tensorrt-libs tensorrt-dev

sudo ln -sf /usr/local/cuda-12.6 /usr/local/cuda

sudo ldconfig

vim ~/.bashrc

#Paste the following three lines here
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export CUDA_HOME=/usr/local/cuda

source ~/.bashrc

# Verify Installed Version
# CUDA 12.6 
nvcc -V

# cuDNN9.3
dpkg -l | grep libcudnn9
# It is considered successful if both lines starting with 'ii' are version 9.3.

# TensorRT10.3
dpkg -l | grep tensorrt
# If these three lines all start with 'ii' and the version is 10.3, it means it was successful.

ZXZhang · January 13, 2026, 1:14am

I encountered a similar error when I ran the build_wheel.py.

collect2: error: ld returned 1 exit status
gmake[3]: *** [benchmarks/CMakeFiles/gptSessionBenchmark.dir/build.make:114: benchmarks/gptSessionBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3917: benchmarks/CMakeFiles/gptSessionBenchmark.dir/all] Error 2
[100%] Linking CXX executable gptManagerBenchmark
/usr/bin/ld: warning: libnvdla_compiler.so, needed by /usr/lib/aarch64-linux-gnu/libnvinfer.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setUseSoftMaxOptz(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReshape(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addSlice(nvdla::ITensor*, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::Weights, nvdla::SliceLayerMod
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyNetwork(nvdla::INetwork*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createWisdom()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IConcatenationLayer::setAxis(int)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanCompressStructuredSparseWeights(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addReduce(nvdla::ITensor*, nvdla::PoolingType, nvdla::Weights)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IPoolingLayer::setPoolingPaddingInclusionType(nvdla::PoolingPaddingInclusionType)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::destroyWisdom(nvdla::IWisdom*)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateDetailedLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addTranspose(nvdla::ITensor*, nvdla::Dims4)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::createNetwork()'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::IProfile::setCanGenerateLayerwiseStats(bool)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::ISoftMaxLayer::setAxis(int)'
/usr/bin/ld: /usr/lib/aarch64-linux-gnu/libnvinfer.so: undefined reference to `nvdla::INetwork::addResize(nvdla::ITensor*, nvdla::ResizeMode, nvdla::Weights)'
collect2: error: ld returned 1 exit status
gmake[3]: *** [benchmarks/CMakeFiles/gptManagerBenchmark.dir/build.make:114: benchmarks/gptManagerBenchmark] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:3997: benchmarks/CMakeFiles/gptManagerBenchmark.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:3884: benchmarks/CMakeFiles/benchmarks.dir/rule] Error 2
gmake: *** [Makefile:1154: benchmarks] Error 2
Traceback (most recent call last):
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 412, in <module>
    main(**vars(args))
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 206, in main
    build_run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake --build . --config Release --parallel 8 --target tensorrt_llm nvinfer_plugin_tensorrt_llm th_common bindings benchmarks  executorWorker  ' returned non-zero exitatus 2.

It seems like I don’t have libnvdla_compiler.so and libnvdla_runtime.so.

How to fix it if anybody konws?

ZXZhang · January 13, 2026, 2:20am

OK,I tried to solve it using the method from the following link.

https://forums.developer.nvidia.com/t/libnvdla-compiler-so-not-found/323265

Download the package and install.

sudo dpkg -i --force-depends ./nvidia-l4t-dla-compiler_36.4.3-20250107174145_arm64.deb

sudo apt install -f -y

dpkg -l | grep nvidia-l4t-dla-compiler

ls /usr/lib/aarch64-linux-gnu/tegra/libnvdla*
# I can see libnvdla_compiler.so and libnvdla_runtime.so

But when I try to run the build_wheel.py, a new error occurred.

[100%] Built target nvinfer_plugin_tensorrt_llm
[100%] Building CXX object tensorrt_llm/executor_worker/CMakeFiles/executorWorker.dir/executorWorker.cpp.o
[100%] Linking CXX executable executorWorker
[100%] Built target executorWorker
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pybind11-stubgen in /home/evoc/.local/lib/python3.10/site-packages (2.5.5)
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/evoc/.local/lib/python3.10/site-packages/pybind11_stubgen/__main__.py", line 4, in <module>
    main()
  File "/home/evoc/.local/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 320, in main
    run(
  File "/home/evoc/.local/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 359, in run
    QualifiedName.from_str(module_name), importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
ImportError: libnvdla_compiler.so: cannot open shared object file: No such file or directory
Failed to build pybind11 stubgen: Command '"/usr/bin/python3" -m pybind11_stubgen -o . bindings' returned non-zero exit status 1.
* Building wheel...
Traceback (most recent call last):
  File "/home/evoc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    main()
  File "/home/evoc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
  File "/home/evoc/.local/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
    return _build_backend().build_wheel(
  File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
  File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
    self.run_setup()
  File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 267, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 93, in <module>
    sanity_check()
  File "setup.py", line 48, in sanity_check
    raise ImportError(
ImportError: The `bindings` module does not exist. Please check the package integrity. If you are attempting to use the pip development mode (editable installation), please execute `build_wheels.py` first, and then run `pip install -e .`.

ERROR Backend subprocess exited when trying to invoke build_wheel
Traceback (most recent call last):
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 412, in <module>
    main(**vars(args))
  File "/home/evoc/TensorRT-LLM/scripts/build_wheel.py", line 325, in main
    build_run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '"/usr/bin/python3" -m build /home/evoc/TensorRT-LLM --skip-dependency-check --no-isolation --wheel --outdir "/home/evoc/TensorRT-LLM/build"' returned non-zero exit status 1.

ZXZhang · January 13, 2026, 2:36am

Finally!!! It succeeded!!!

OK,try this way:

sudo ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so /usr/lib/aarch64-linux-gnu/
sudo ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so /usr/lib/aarch64-linux-gnu/

ls /usr/lib/aarch64-linux-gnu/libnvdla*.so
# I can see /usr/lib/aarch64-linux-gnu/libnvdla_compiler.so  /usr/lib/aarch64-linux-gnu/libnvdla_runtime.so

sudo ldconfig && source ~/.bashrc

export LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu:$LD_LIBRARY_PATH

python3 scripts/build_wheel.py --clean --cuda_architectures 87 -DENABLE_MULTI_DEVICE=0 --build_type Release --benchmarks --use_ccache
# This time it succeeded.
# Successfully built tensorrt_llm-0.12.0-cp310-cp310-linux_aarch64.whl

system · February 3, 2026, 5:40am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Inquiry on any updated support for tensorrt-llm support nvidia orin AGX? Jetson AGX Orin tensorrt , generative_ai , llama	4	255	June 3, 2025
TensorRT-LLM for Jetson Jetson AGX Orin generative_ai	11	3876	July 7, 2025
TensorRT-LLM for jetson errors Jetson AGX Orin generative_ai , paligemma , kosmos-2 , llama	14	936	January 16, 2025
Can I use TensorRT-LLM in Jetson AGX orin? Jetson AGX Orin nvbugs , generative_ai	3	773	July 15, 2024
Can TensorRT-LLM be used on Jetson Orin NX with JetPack 6.1? Jetson Orin NX tensorrt , generative_ai	6	412	December 17, 2024
Tensort-RT LLM Support for Jetson Jetson Orin Nano generative_ai	1	30	January 22, 2026
TensorRT-LLM issue while installing in jetson orin nano developer kit Jetson Orin Nano tensorrt , generative_ai	5	215	March 17, 2025
Error in Tensorrt_llm's whl file Jetson AGX Orin tensorrt	17	408	July 16, 2025
Jetson R35.3.1 TensorRT-LLM build fails Jetson TX2 generative_ai	2	100	October 22, 2025
How to install nvidia-tensorrt? Jetson AGX Orin tensorrt	7	13504	May 17, 2023

Failed running build_wheel.py for TensorRT-LLM

Related topics