TensorRT-8.6.1 (GA): aarch64 sbsa packages contain x86-64 stub libraries

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

TensorRT 8.6 distribution for Linux includes stub static libraries which are used to disable dependencies on cuBLAS and cuDNN libraries:

  • libcublasLt_static_stub_trt.a
  • libcublas_static_stub_trt.a
  • libcudnn_static_stub_trt.a

The problem is that these libraries are built for x86-64 (intel) despite being packaged for aarch64 sbsa (arm) hosts.

Environment

TensorRT Version: 8.6.1 GA

Relevant Files

Reproducer/analyzer script is at: GitHub - sergeev917/tensorrt-stub-wrong-arch: Reproducer for tensorrt packaging issue

Steps To Reproduce

  • Download nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0_1.0-1_arm64.deb
  • Run “./reproduce.sh --repo-deb-path …/nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0_1.0-1_arm64.deb”

Result:

-: OK
INFO: unpacking into /tmp/tmp.g2wsJEdImQ
INFO: unpacking /tmp/tmp.g2wsJEdImQ/var/nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0/libnvinfer-dev_8.6.1.6-1+cuda12.0_arm64.deb into /tmp/tmp.g2wsJEdImQ
/tmp/tmp.g2wsJEdImQ/usr/lib/aarch64-linux-gnu/stubs/libcublasLt_static_stub_trt/stub.o   Advanced Micro Devices X86-64
/tmp/tmp.g2wsJEdImQ/usr/lib/aarch64-linux-gnu/stubs/libcublas_static_stub_trt/stub.o   Advanced Micro Devices X86-64
/tmp/tmp.g2wsJEdImQ/usr/lib/aarch64-linux-gnu/stubs/libcudnn_static_stub_trt/stub.o   Advanced Micro Devices X86-64
All stubs are checked, unpacked files remain in /tmp/tmp.g2wsJEdImQ!
3

Hi,
Please refer to the installation steps from the below link if in case you are missing on anything

Also, we suggest you to use TRT NGC containers to avoid any system dependency related issues.

Thanks!

5

Hello, @AakankshaS!

Please refer to the installation steps from the below link if in case you are missing on anything

This guide is not relevant to the described issue. In short, SBSA/ARM packages contain some static libraries for X86 architecture and not ARM architecture. I don’t see how this could be an intended outcome. Correct me, if I’m wrong.

To be more precise, nv-tensorrt-local-repo-ubuntu2004-8.6.1-cuda-12.0_1.0-1_arm64.deb is the “repository” DEB package (i.e. contains other DEB packages) which can be downloaded from https://developer.nvidia.com/nvidia-tensorrt-8x-download. The problematic package inside is libnvinfer-dev_8.6.1.6-1+cuda12.0_arm64.deb – it contains /usr/lib/aarch64-linux-gnu/stubs/*.a libraries which are built for X86.

The reproducer script shows that issue in details.

Idea behind these static stub libraries is described here:

Stubbed static libraries for cuDNN, cuBLAS, and cuBLASLt are provided in $(TRT_LIB_DIR)/stubs. When statically linking TensorRT with no requirement for cuDNN, cuBLAS, or cuBLASLt, the stubbed library can be used to reduce CPU and GPU memory usage.