Static linking of nvinfer, undefined reference onnx2trt::xxx

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

When trying to link tensorrt 5.1.2.2 statically and using the onnx parser I get many undefined references like the following

/usr/lib/x86_64-linux-gnu/libnvonnxparser_static.a(NvOnnxParser.cpp.o):(.data.rel.ro._ZTVN8onnx2trt21TypeSerializingPluginE[_ZTVN8onnx2trt21TypeSerializingPluginE]+0x48): 
undefined reference to `onnx2trt::PluginAdapter::initialize

And indeed, scanning the static libs in /usr/lib/x86_64-linux-gnu and /usr/local/cuda/lib64, the definitions cannot be found in any static lib provided by the packages libnvinfer-dev.
Building with shared libraries compiles successfully, however.

Which libraries are required here?

1 Like
2

Some help will be very appreciated about static linking of tensorRT
The version that I use is 7.2.3.4 and I experience similar linking issues.
Some “undefined reference” issues are fixed by adding CUDA-11 libraries but I think it might be the wrong approach.

"nvrtc"
"cublas"
"onnx_proto"
"protobuf"
"protobuf-lite"
"myelin_compiler_static"
"myelin_executor_static"
"myelin_pattern_library_static"
     "myelin_pattern_runtime_static"
     "nvinfer_plugin_static"
     "nvinfer_static"
     "nvparsers_static"
     "nvonnxparser_static"

The error trace that I receive is:

undefined reference to `cublasLtCreate'
/home/david/workspace/CU11/vs_nextgen_isp/CMake/../runtime/linux_x64/TensorRT-7.2.3.4/libmyelin_compiler_static.a(dtree-cost.o): In function `myelin::ir::cublas_heuristic_search_helper(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, bool, cudaDataType_t, bool, bool, int, long, long, long, myelin::errors_t&, cublasLtContext*&, myelin::ir::tactic_attribute_t&, bool)':
dtree-cost.cpp:(.text+0x185c): undefined reference to `cublasLtMatmulDescCreate'
dtree-cost.cpp:(.text+0x187a): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x188c): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x189e): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x18b0): undefined reference to `cublasLtMatmulDescDestroy'
dtree-cost.cpp:(.text+0x18c2): undefined reference to `cublasLtMatmulPreferenceDestroy'
dtree-cost.cpp:(.text+0x1908): undefined reference to `cublasLtMatmulDescSetAttribute'
dtree-cost.cpp:(.text+0x1926): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x1938): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x194a): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x195c): undefined reference to `cublasLtMatmulDescDestroy'
dtree-cost.cpp:(.text+0x196e): undefined reference to `cublasLtMatmulPreferenceDestroy'
dtree-cost.cpp:(.text+0x22a8): undefined reference to `cublasLtMatmulDescSetAttribute'
dtree-cost.cpp:(.text+0x22da): undefined reference to `cublasLtMatrixLayoutCreate'
dtree-cost.cpp:(.text+0x2308): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x2333): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x2365): undefined reference to `cublasLtMatrixLayoutCreate'
dtree-cost.cpp:(.text+0x2383): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x2395): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x23a7): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x23b9): undefined reference to `cublasLtMatmulDescDestroy'
dtree-cost.cpp:(.text+0x23cb): undefined reference to `cublasLtMatmulPreferenceDestroy'
dtree-cost.cpp:(.text+0x243e): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x2450): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x2462): undefined reference to `cublasLtMatrixLayoutDestroy'
dtree-cost.cpp:(.text+0x2474): undefined reference to `cublasLtMatmulDescDestroy'
dtree-cost.cpp:(.text+0x2486): undefined reference to `cublasLtMatmulPreferenceDestroy'
dtree-cost.cpp:(.text+0x287d): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x28a8): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x28d3): undefined reference to `cublasLtMatrixLayoutCreate'
dtree-cost.cpp:(.text+0x28f9): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x2924): undefined reference to `cublasLtMatrixLayoutSetAttribute'
dtree-cost.cpp:(.text+0x29a7): undefined reference to `cublasLtMatmulPreferenceCreate'
dtree-cost.cpp:(.text+0x2b95): undefined reference to `cublasLtMatmulPreferenceSetAttribute'
dtree-cost.cpp:(.text+0x2be6): undefined reference to `cublasLtMatmulAlgoGetHeuristic'
dtree-cost.cpp:(.text+0x3088): undefined reference to `cublasLtMatmulAlgoConfigGetAttribute'
dtree-cost.cpp:(.text+0x30a1): undefined reference to `cublasLtMatmulAlgoConfigGetAttribute'
dtree-cost.cpp:(.text+0x30ba): undefined reference to `cublasLtMatmulAlgoConfigGetAttribute'
dtree-cost.cpp:(.text+0x30d3): undefined reference to `cublasLtMatmulAlgoConfigGetAttribute'
dtree-cost.cpp:(.text+0x30ec): undefined reference to `cublasLtMatmulAlgoConfigGetAttribute'`

Thanks

OS: Ubuntu 18.04
CUDA: 11.1
cudnn: 8.1.1.33

3

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

6

How does the ONNX model is relevant to this discussion?
The issue is with static linking with tensorRT. When I compile dynamically (linking with so) the program compiles successfully.

7

In my case, the original issue was caused by passing the linked libraries in an incorrect order. The order in which STATIC libraries are passed to the linker matters :/ This issue should be fixed by playing with the order of the libraries. In my case the following order works:

libnvonnxparser_static.a, libprotobuf-lite.a, libonnx_proto.a, libnvcaffe_parser.a, libnvparsers_static.a, libnvinfer_static.a, libnvinfer_plugin_static.a, libprotobuf.a, libmyelin_compiler_static.a, libmyelin_executor_static.a, libmyelin_pattern_library_static.a, libmyelin_pattern_runtime_static.a

Shared libraries don’t exhibit this behavior in Ubuntu, that’s why dynamic linking works.

So this is generic and unrelated to TensorRT.

Check out

https://bnikolic.co.uk/blog/gnu-ld-as-needed