How to infer the .trt/.engine model on Drive AGX Orin through cross-compile?

Please provide the following info (tick the boxes after creating this topic):
Software Version
[√] DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[√] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
[√] DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.3.10904
[√] other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[√] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Hi,
I originally had a C++ file and a trt model that could be inferred in the Windows environment, but now I want to infer the trt model on orin through cross compilation.

My steps are as follows:

  1. Convert the onnx model to a. trt model on the Orin target through trtexec;

  2. Copy the TRT model to the Docker on the host side;

  3. A sampleShuZiRen directory project has been added under/usr/src/tensorrt/samples on the host side. The corresponding Makefile has been written in this directory, and the C++file that can infer trt in the Windows environment has been copied here;

  4. Then compile using “sudo make TARGET=aarch64”.

However, the following error occurred. Is there a corresponding documentation to explain it?

aarch64-linux-gnu-g++ -MM -MF …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.d -MP -MT …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o -Wall -Wno-deprecated-declarations -std=c++14 -I"/usr/include/aarch64-linux-gnu" -I"/usr/include/aarch64-linux-gnu" -I"/usr/local/cuda/targets/aarch64-linux/include" -I"…/common" -I"/usr/local/cuda/include" -I"/usr/local/cuda/include" -I"…/include" -I"…/…/include" -I"…/…/parsers/onnxOpenSource" -D_REENTRANT -DTRT_STATIC=0 …/common/sampleEngines.cpp
Compiling: …/common/sampleEngines.cpp
aarch64-linux-gnu-g++ -Wall -Wno-deprecated-declarations -std=c++14 -I"/usr/include/aarch64-linux-gnu" -I"/usr/include/aarch64-linux-gnu" -I"/usr/local/cuda/targets/aarch64-linux/include" -I"…/common" -I"/usr/local/cuda/include" -I"/usr/local/cuda/include" -I"…/include" -I"…/…/include" -I"…/…/parsers/onnxOpenSource" -D_REENTRANT -DTRT_STATIC=0 -g -c -o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o …/common/sampleEngines.cpp
Linking: …/…/bin/sample_shu_zi_ren_debug
aarch64-linux-gnu-g++ -o …/…/bin/sample_shu_zi_ren_debug -L"…/lib/stubs" -L"…/…/lib/stubs" -L"/usr/lib/aarch64-linux-gnu/stubs" -L"/usr/lib/aarch64-linux-gnu" -L"/usr/lib/aarch64-linux-gnu/stubs" -L"/usr/lib/aarch64-linux-gnu" -L"/usr/local/cuda/targets/aarch64-linux/lib/stubs" -L"/usr/local/cuda/targets/aarch64-linux/lib" -L"/usr/local/cuda/lib" -Wl,-rpath-link=“/usr/local/cuda/lib” -L"/usr/local/cuda/lib64" -Wl,-rpath-link=“/usr/local/cuda/lib64” -L"…/lib" -L"…/…/lib" -L"…/…/lib" -Wl,-rpath-link=“…/…/lib” -L"/usr/lib/x86_64-linux-gnu/lib" -Wl,-rpath-link=“/usr/lib/x86_64-linux-gnu/lib” -L"" -Wl,-rpath-link=“” -L…/…/bin -Wl,–start-group -lnvinfer -lnvparsers -lnvinfer_plugin -lnvonnxparser -lcudart -lcublas -lcudnn -lrt -ldl -lpthread …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/sampleShuZiRen.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleInference.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleUtils.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleOptions.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/logger.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/getOptions.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleReporting.o …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o -Wl,–end-group
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleInference.o: in function sample::timeDeserialize(sample::InferenceEnvironment&, sample::SystemOptions const&)': /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleInference.cpp:1216: undefined reference to createRuntime()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleInference.o: in function sample::getLayerInformation[abi:cxx11](nvinfer1::ICudaEngine*, nvinfer1::IExecutionContext*, nvinfer1::LayerInformationFormat)': /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleInference.cpp:1307: undefined reference to createRuntime()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o: in function sample::(anonymous namespace)::CaffeBufferShutter::~CaffeBufferShutter()': /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleEngines.cpp:56: undefined reference to shutdownCaffeParser()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o: in function sample::(anonymous namespace)::UffBufferShutter::~UffBufferShutter()': /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleEngines.cpp:64: undefined reference to shutdownUffParser()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: …/…/bin/dchobj/sampleShuZiRen/sampleShuZiRen/…/common/sampleEngines.o: in function sample::LazilyDeserializedEngine::get()': /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleEngines.cpp:110: undefined reference to createRuntime()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: /usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:114: undefined reference to createRuntime()' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: ../../bin/dchobj/sampleShuZiRen/sampleShuZiRen/../common/sampleEngines.o: in function sample::modelToNetwork(sample::ModelOptions const&, sample::BuildOptions const&, nvinfer1::INetworkDefinition&, std::ostream&, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >)‘:
/usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:252: undefined reference to sampleCreateCaffeParser()' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: /usr/src/tensorrt/samples/sampleShuZiRen/../common/sampleEngines.cpp:278: undefined reference to sampleCreateUffParser()’
/usr/lib/gcc-cross/aarch64-linux-gnu/9/…/…/…/…/aarch64-linux-gnu/bin/ld: /usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:312: undefined reference to createONNXParser(nvinfer1::INetworkDefinition&)' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: ../../bin/dchobj/sampleShuZiRen/sampleShuZiRen/../common/sampleEngines.o: in function sample::modelToBuildEnv(sample::ModelOptions const&, sample::BuildOptions const&, sample::SystemOptions&, sample::BuildEnvironment&, std::ostream&)‘:
/usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:1250: undefined reference to createBuilder()' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: ../../bin/dchobj/sampleShuZiRen/sampleShuZiRen/../common/sampleEngines.o: in function sample::dumpRefittable(nvinfer1::ICudaEngine&)’:
/usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:1367: undefined reference to createRefitter(nvinfer1::ICudaEngine&)' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: ../../bin/dchobj/sampleShuZiRen/sampleShuZiRen/../common/sampleEngines.o: in function sample::timeRefit(nvinfer1::INetworkDefinition const&, nvinfer1::ICudaEngine&, bool)':
/usr/src/tensorrt/samples/sampleShuZiRen/…/common/sampleEngines.cpp:1536: undefined reference to createRefitter(nvinfer1::ICudaEngine&)' /usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: ../../bin/dchobj/sampleShuZiRen/sampleShuZiRen/../common/sampleEngines.o: in function samplesCommon::updateTimingCacheFile(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, nvinfer1::ITimingCache const
)‘:
/usr/src/tensorrt/samples/sampleShuZiRen/…/common/common.h:1183: undefined reference to `createBuilder()’
collect2: error: ld returned 1 exit status
make[1]: *** […/Makefile.config:363: …/…/bin/sample_shu_zi_ren_debug] Error 1
make[1]: Leaving directory ‘/usr/src/tensorrt/samples/sampleShuZiRen’
make: *** [Makefile:85: all] Error 2

Dear @AlanWang,
I could cross compile the shipped TRT samples with make TARGET=aarch64.

TRT.txt (223.0 KB)

Were you able to compile the TRT shipped samples and notice issue with only your sample?

@SivaRamaKrishnaNV
Thank you very much for your answer.
I could cross compile the shipped TRT samples with make TARGET=aarch64 too.
But, as I mentioned earlier, I want to infer a trt model on Orin through cross compilation. The shipped TRT samples use the onnx model for inference,this is a different place.
So, is there any documentation to infer with trt model?

Dear @AlanWang,
I see you are using the functions(files) used by trtexec. I suspect some issue in your makefile as you are able to compile trtexec and other TRT samples.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.