I made this change in the Triton_Server.Dockerfile
cp deb_file/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.6.0 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.8.5.3 Attaching the new Dockerfile I am using
Triton_Server.Dockerfile.txt (1.7 KB)
I don’t think this is the correct way to do it since it is just copying the x86 nvinfer file to aarch64 directory. However, the file might still be incompatible due to architectural differences. Can you please share the correct Triton_Server.Dockerfile to be used for setting it up on the Jetson?
Please find the error I received after making the change as mentioned above -
nvidia@ubuntu:~/ocr_nvidia/NVIDIA-Optical-Character-Detection-and-Recognition-Solution/triton$ sudo bash setup_triton_server.sh onnx_models/ocdnet.onnx onnx_models/ocrnet.onnx onnx_models/character_list.txt
[sudo] password for nvidia:
Set input size to HxW: onnx_models/ocdnet.onnxxonnx_models/ocrnet.onnx and max batch size: onnx_models/character_list.txt for ocd tensorRT engine
[+] Building 1225.4s (16/17) docker:default
=> [internal] load build definition from Triton_Server.Dockerfile 0.0s
=> => transferring dockerfile: 1.82kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for nvcr.io/nvidia/tritonserver:23.02-py3 1.0s
=> [ 1/13] FROM nvcr.io/nvidia/tritonserver:23.02-py3@sha256:2ab9e1cbd88a3e7fc29c6e393ac50e78604efc4ca67e8da9fa1fd94afe3fda7b 0.0s
=> [internal] load build context 3.7s
=> => transferring context: 246.25MB 3.6s
=> CACHED [ 2/13] RUN mkdir -p /opt/nvocdr && cd /opt/nvocdr 0.0s
=> CACHED [ 3/13] RUN mkdir -p /opt/nvocdr/onnx_model 0.0s
=> CACHED [ 4/13] COPY NVIDIA-Optical-Character-Detection-and-Recognition-Solution/triton/models/ocdnet.onnx /opt/nvocdr/onnx_model/ocdnet.onnx 0.0s
=> CACHED [ 5/13] COPY NVIDIA-Optical-Character-Detection-and-Recognition-Solution/triton/models/ocrnet.onnx /opt/nvocdr/onnx_model/ocrnet.onnx 0.0s
=> CACHED [ 6/13] COPY NVIDIA-Optical-Character-Detection-and-Recognition-Solution/triton/models/character_list /opt/nvocdr/onnx_model/character_list 0.0s
=> CACHED [ 7/13] RUN ls -l /opt/nvocdr/onnx_model/ 0.0s
=> [ 8/13] RUN wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/8.6.0/local_repos/nv-tensorrt-local-repo-ubuntu2004-8.6.0-cuda-11.8_1.0-1_amd64.deb && dpkg-deb -xv nv-tensorrt-local-repo-ubuntu2004-8.6.0-cuda-11.8_1.0-1_amd64.deb debs && cd debs/var/nv-tensorrt-lo 288.8s
=> [ 9/13] RUN apt-get update && apt-get install libgl1-mesa-glx --yes && apt-get install libopencv-dev --yes 519.9s
=> [10/13] COPY NVIDIA-Optical-Character-Detection-and-Recognition-Solution /opt/nvocdr/ocdr 45.1s
=> [11/13] RUN cd /opt/nvocdr/ocdr/triton && python3 -m pip install --upgrade pip && python3 -m pip install nvidia-pyindex && python3 -m pip install -r requirements-pip.txt 318.0s
=> ERROR [12/13] RUN cd /opt/nvocdr/ocdr/triton && make -j8 46.7s
------
> [12/13] RUN cd /opt/nvocdr/ocdr/triton && make -j8:
2.608 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.609 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.609 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.612 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.613 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.619 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.624 if [ ! -d ./chobj ]; then mkdir -p ./chobj; fi
2.860 g++ -MM -MF chobj/OCDNetEngine.d -MP -MT chobj/OCDNetEngine.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/OCDNetEngine.cpp
2.861 g++ -MM -MF chobj/MemManager.d -MP -MT chobj/MemManager.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/MemManager.cpp
2.866 g++ -MM -MF chobj/TRTEngine.d -MP -MT chobj/TRTEngine.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/TRTEngine.cpp
2.872 g++ -MM -MF chobj/nvOCDR.d -MP -MT chobj/nvOCDR.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/nvOCDR.cpp
2.878 g++ -MM -MF chobj/RectEngine.d -MP -MT chobj/RectEngine.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/RectEngine.cpp
2.880 g++ -MM -MF chobj/OCRNetEngine.d -MP -MT chobj/OCRNetEngine.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/OCRNetEngine.cpp
2.881 g++ -MM -MF chobj/kernel.d -MP -MT chobj/kernel.o -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ ../src/kernel.cu
2.887 g++: warning: ../src/kernel.cu: linker input file unused because linking not done
2.900 Compiling: ../src/kernel.cu
2.900 /usr/local/cuda/bin/nvcc -m64 -ccbin g++ --compiler-options '-fPIC' -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -c -o chobj/kernel.o ../src/kernel.cu
3.366 Compiling: ../src/MemManager.cpp
3.368 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/MemManager.o ../src/MemManager.cpp
4.423 Compiling: ../src/TRTEngine.cpp
4.424 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/TRTEngine.o ../src/TRTEngine.cpp
5.494 Compiling: ../src/OCRNetEngine.cpp
5.495 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/OCRNetEngine.o ../src/OCRNetEngine.cpp
6.895 Compiling: ../src/RectEngine.cpp
6.895 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/RectEngine.o ../src/RectEngine.cpp
7.410 Compiling: ../src/nvOCDR.cpp
7.411 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/nvOCDR.o ../src/nvOCDR.cpp
7.453 Compiling: ../src/OCDNetEngine.cpp
7.455 g++ -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -c -o chobj/OCDNetEngine.o ../src/OCDNetEngine.cpp
14.49 ../src/nvOCDR.cpp: In member function 'void nvocdr::nvOCDR::inferPatch(void*, void*, const Dims&, const Dims&, float, std::vector<std::vector<std::pair<nvocdr::Polygon, std::pair<std::__cxx11::basic_string<char>, float> > > >&)':
14.49 ../src/nvOCDR.cpp:451:105: warning: pointer of type 'void *' used in arithmetic [-Wpointer-arith]
14.49 451 | patchMaskMergeCUDA(oriImgBitMapDev.data(), mBuffMgr.mDeviceBuffer[ocdThresholdBufIdx].data()+(i*patch_w*patch_h), oriImgMaskDev.data(), mBuffMgr.mDeviceBuffer[ocdRawOutBufIdx].data()+(i*patch_w*patch_h*sizeof(float)), ocdOutputPatchshape, oriBinaryshape, overlapRate, col_idx, row_idx, num_col_cut, num_row_cut,mStream);
14.49 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~
14.49 ../src/nvOCDR.cpp:451:195: warning: pointer of type 'void *' used in arithmetic [-Wpointer-arith]
14.49 451 | patchMaskMergeCUDA(oriImgBitMapDev.data(), mBuffMgr.mDeviceBuffer[ocdThresholdBufIdx].data()+(i*patch_w*patch_h), oriImgMaskDev.data(), mBuffMgr.mDeviceBuffer[ocdRawOutBufIdx].data()+(i*patch_w*patch_h*sizeof(float)), ocdOutputPatchshape, oriBinaryshape, overlapRate, col_idx, row_idx, num_col_cut, num_row_cut,mStream);
14.49 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
22.89 Linking: nvocdr.cpython-38-x86_64-linux-gnu.so
22.89 g++ -O3 -Wall -shared -std=c++11 -I../include -I/usr/include/x86_64-linux-gnu/ -I../src -I/usr/local/cuda/include -I/usr/include/opencv4/ -fPIC -I/usr/include/python3.8 -I/usr/local/lib/python3.8/dist-packages/pybind11/include pybind.cpp -o nvocdr.cpython-38-x86_64-linux-gnu.so chobj/nvOCDR.o chobj/TRTEngine.o chobj/MemManager.o chobj/OCDNetEngine.o chobj/RectEngine.o chobj/OCRNetEngine.o chobj/kernel.o -L/usr/lib/x86_64-linux-gnu/ -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu/ -L. -lnvinfer -lcudart -lopencv_core -lopencv_imgproc -lopencv_imgcodecs -lcublas -lnvinfer_plugin
29.32 In file included from ../src/nvOCDR.h:10,
29.32 from pybind.h:6,
29.32 from pybind.cpp:2:
29.32 ../src/MemManager.h: In instantiation of 'nvocdr::GenericBuffer<AllocFunc, FreeFunc>::GenericBuffer(size_t, size_t) [with AllocFunc = nvocdr::DeviceAllocator; FreeFunc = nvocdr::DeviceFree; size_t = long unsigned int]':
29.32 pybind.cpp:54:60: required from here
29.32 ../src/MemManager.h:141:12: warning: 'nvocdr::GenericBuffer<nvocdr::DeviceAllocator, nvocdr::DeviceFree>::mItemSize' will be initialized after [-Wreorder]
29.32 141 | size_t mItemSize;
29.32 | ^~~~~~~~~
29.32 ../src/MemManager.h:140:22: warning: 'size_t nvocdr::GenericBuffer<nvocdr::DeviceAllocator, nvocdr::DeviceFree>::mCapacity' [-Wreorder]
29.32 140 | size_t mSize{0}, mCapacity{0};
29.32 | ^~~~~~~~~
29.32 ../src/MemManager.h:45:5: warning: when initialized here [-Wreorder]
29.32 45 | GenericBuffer(size_t size, size_t item_size)
29.32 | ^~~~~~~~~~~~~
46.24 /usr/bin/ld: skipping incompatible /usr/lib/gcc/aarch64-linux-gnu/9/../../../aarch64-linux-gnu/libnvinfer_plugin.so when searching for -lnvinfer_plugin
46.24 /usr/bin/ld: skipping incompatible /lib/aarch64-linux-gnu/libnvinfer_plugin.so when searching for -lnvinfer_plugin
46.24 /usr/bin/ld: skipping incompatible /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so when searching for -lnvinfer_plugin
46.24 /usr/bin/ld: skipping incompatible /lib/aarch64-linux-gnu/libnvinfer_plugin.so when searching for -lnvinfer_plugin
46.24 /usr/bin/ld: skipping incompatible /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so when searching for -lnvinfer_plugin
46.24 /usr/bin/ld: cannot find -lnvinfer_plugin
46.42 collect2: error: ld returned 1 exit status
46.42 make: *** [Makefile:88: nvocdr.cpython-38-x86_64-linux-gnu.so] Error 1
------
Triton_Server.Dockerfile:41
--------------------
39 |
40 | # step7: build nvocdr lib
41 | >>> RUN cd /opt/nvocdr/ocdr/triton && make -j8
42 |
43 | ENV PYTHONPATH="/opt/nvocdr/ocdr/triton:${PYTHONPATH}"
--------------------
ERROR: failed to solve: process "/bin/sh -c cd /opt/nvocdr/ocdr/triton && make -j8" did not complete successfully: exit code: 2
