Compatibility issues when running any Deepstream6.1 and later BodyPose 2d app on a Jetson Orin.
Impossible to fix this without NVIDIA recompiling a library that was not updated with Deepstream6.1
This had naturally worked with me previously on an x86 machine with an NVIDIA RTX3090 GPU.
The problem with aarch64 on the latest Jetpacks 5.0 and above is that I believe the library in /opt/nvidia/deepstream/deepstream6.1/lib/cvcore_libs/libnvcv_bodypose2d.so has not been updated to use the newer TensorRT installed in Jetpack 5.01 and above, compatible with the jetson Orin.
The error when running the bodypose2d app from graph-composer is the following:
The same error occurs regardless if the app was run from the host device or directly from the jetson.
/opt/nvidia/deepstream/deepstream/My-App/Container_config_jetson.yml
2022-08-17 09:54:15,335 - Composer - INFO - Running graph /opt/nvidia/deepstream/deepstream/My-App/App-BodyPose2d-test.yaml
[INFO] Trying to access target 'agx@192.168.1.176' over ssh
[INFO] Target access successful
Graphs: App-BodyPose2d-test.yaml
Target: /opt/nvidia/graph-composer/config/target_aarch64.yaml
===================================================================
Running App-BodyPose2d-test.yaml
===================================================================
[INFO] Writing manifest to /tmp/ds.App-BodyPose2d-test/manifest.yaml
2022-08-17 09:54:18,942 - Registry - INFO - Installing graph ...
2022-08-17 09:54:18,964 - Registry - INFO -
Requested target configuration:
arch: aarch64 os: linux distribution: ubuntu_20.04 cuda: 11.4 cudnn: 8.3.1.22 tensorrt: 8.4.0.11 deepstream: 6.1 vpi: 2.0.7
Selected extension variants:
StandardExtension
Version: 2.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 cuda: 11.4
MultimediaExtension
Version: 2.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04
NvDsInterfaceExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsBaseExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsMuxDemuxExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsInferenceExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsBodyPose2D
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 tensorrt: 8.4.0.2 deepstream: 6.1
NvDsOutputSinkExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsVisualizationExt
Version: 1.0.1 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsInferenceUtilsExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
NvDsSourceExt
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 deepstream: 6.1
Graph installed to output directory
2022-08-17 11:55:39.036 INFO gxf/gxe/gxe.cpp@230: Creating context
2022-08-17 11:55:39.063 ERROR gxf/std/extension_loader.cpp@109: Failed to load extension /tmp/ds.App-BodyPose2d-test/gxf/bodypose2d/libgxf_bodypose2d.so Error: libmyelin.so.1: cannot open shared object file: No such file or directory
2022-08-17 11:55:39.063 ERROR gxf/gxe/gxe.cpp@275: LoadExtensionManifest Error: GXF_EXTENSION_FILE_NOT_FOUND
*******************************************************************
End App-BodyPose2d-test.yaml
*******************************************************************
[INFO] Graph installation directory /tmp/ds.App-BodyPose2d-test and manifest /tmp/ds.App-BodyPose2d-test/manifest.yaml retained
2022-08-17 09:55:39 [92,568,635ms] [Error] [omni.kit.app.impl] [py stderr]: Connection to 192.168.1.176 closed.
Connection to 192.168.1.176 closed.2022-08-17 09:55:39 [92,568,635ms] [Error] [omni.kit.app.impl] [py stderr]:
The library libmyelin.so.1 requested by the graph-composer extension of bodypose2d is found in TensorRT7.2 or older, and was discontinued even in the oldest TensorRT version compatible with the oldest Jetson Orin JetPack version possible.
This basically makes it impossible to run this graph-composer/Deepstream BodyPose2D extension on a Jetson Orin without NVIDIA recompiling the /opt/nvidia/deepstream/deepstream6.1/lib/cvcore_libs/libnvcv_bodypose2d.so library that is asking for this older TensorRT library.
The library libnvcv_bodypose2d.so was clearly not updated with TensorRT 8.4 and newer that doesn’t need libmyelin.so.1 from TensorRT 7.2 and older
For Reference: TensorRT8.4 is the oldest version compatible with the Jetson Orin and JetPack 5.01
We can even see in the log:
NvDsBodyPose2D
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_20.04 tensorrt: 8.4.0.2 deepstream: 6.1
That the Bodypose2d extension is requesting tensorrt: 8.4.0.2 which does not come with libmyelin.so.1 anymore.
Finally, I tried to install a second version of tensorrt on the jetson to include this library. This removed the error, but came with another error requesting Cuda 11.1 ( TensorRT7.2 needs it to run)
And then it was again impossible to have two versions of CUDA running on the Orin without having very ugly system errors and driver compatibility problems.
Nvidia has not released the code for the library that needs to be recompiled, and so I cannot look at the code, but I feel like it should be something simple to change to make it compatible with JetPack5.01 and Deepstream 6.1 that it comes shipped with.
Thank you for your support.
