Libdcgm 3 triton compatibility

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) RTX 3080
• DeepStream Version 6.1.1
• TensorRT Version Default nvcr docker.
• NVIDIA GPU Driver Version (valid for GPU only) 515
• Issue Type( questions, new requirements, bugs) Compatibility breaks after apt update && apt upgrade -y
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Open nvcr.io/nvidia/deepstream:6.1.1-triton docker image.
apt update && apt upgrade -y
Try launching triton server fails

Found out that the upgrade process downloads libdcgm 3 and triton wants libdcgm 2.x

EDIT: “apt-mark hold datacenter-gpu-manager” fixes the issue, but probably the most ideal scenario would be that apt upgrade updated everything acordingly.

  1. could you share the output of “ldd /opt/tritonserver/bin/tritonserver” ?
  2. could you share the terminal log of starting tritonsever?
root@3ce247c65f6c:/opt/nvidia/deepstream/deepstream-6.1# ldd /opt/tritonserver/bin/tritonserver
        linux-vdso.so.1 (0x00007fff9b64e000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbbef8bf000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbbef8b9000)
        libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fbbef826000)
        libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fbbef550000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fbbef534000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbbef50f000)
        libre2.so.5 => /lib/x86_64-linux-gnu/libre2.so.5 (0x00007fbbef49e000)
        libb64.so.0d => /lib/x86_64-linux-gnu/libb64.so.0d (0x00007fbbef29b000)
        libtritonserver.so => /opt/tritonserver/lib/libtritonserver.so (0x00007fbbee6b3000)
        libcudart.so.11.0 => /usr/local/cuda/targets/x86_64-linux/lib/libcudart.so.11.0 (0x00007fbbee40e000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbbee22c000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbbee0db000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbbee0c0000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbbedece000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fbbeffb4000)
        libnuma.so.1 => /lib/x86_64-linux-gnu/libnuma.so.1 (0x00007fbbedec1000)
        libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fbbedeb8000)
        libcurl.so.4 => /lib/x86_64-linux-gnu/libcurl.so.4 (0x00007fbbede24000)
        libdcgm.so.2 => not found
        libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007fbbeddfb000)
        libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x00007fbbeddda000)
        librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x00007fbbeddba000)
        libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x00007fbbedd4a000)
        libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x00007fbbedd37000)
        libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007fbbedcea000)
        libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007fbbedc94000)
        liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007fbbedc83000)
        libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x00007fbbedc75000)
        libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x00007fbbedaf1000)
        libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007fbbed91b000)
        libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x00007fbbed8e4000)
        libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x00007fbbed8aa000)
        libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fbbed826000)
        libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007fbbed749000)
        libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007fbbed716000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007fbbed70f000)
        libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007fbbed700000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fbbed6e4000)
        libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fbbed6c7000)
        libgssapi.so.3 => /lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007fbbed682000)
        libbrotlicommon.so.1 => /lib/x86_64-linux-gnu/libbrotlicommon.so.1 (0x00007fbbed65d000)
        libp11-kit.so.0 => /lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007fbbed527000)
        libtasn1.so.6 => /lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007fbbed511000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007fbbed50a000)
        libheimntlm.so.0 => /lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007fbbed4fe000)
        libkrb5.so.26 => /lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007fbbed469000)
        libasn1.so.8 => /lib/x86_64-linux-gnu/libasn1.so.8 (0x00007fbbed3c2000)
        libhcrypto.so.4 => /lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007fbbed38a000)
        libroken.so.18 => /lib/x86_64-linux-gnu/libroken.so.18 (0x00007fbbed371000)
        libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007fbbed365000)
        libwind.so.0 => /lib/x86_64-linux-gnu/libwind.so.0 (0x00007fbbed33b000)
        libheimbase.so.1 => /lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007fbbed327000)
        libhx509.so.5 => /lib/x86_64-linux-gnu/libhx509.so.5 (0x00007fbbed2d9000)
        libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007fbbed1b0000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fbbed175000)
root@3ce247c65f6c:/opt/nvidia/deepstream/deepstream-6.1# tritonserver
tritonserver: error while loading shared libraries: libdcgm.so.2: cannot open shared object file: No such file or directory
root@3ce247c65f6c:/opt/nvidia/deepstream/deepstream-6.1# 

why do you need to do “apt upgrade -y”?

We do that as a common practice through our images so that we have the latest version of the packages

here is a workaround:

  1. wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/datacenter-gpu-manager_2.2.9_amd64.deb, and there are other versions: https://ubuntu.pkgs.org/20.04/cuda-amd64/datacenter-gpu-manager_2.2.9_amd64.deb.html
  2. dpkg -i datacenter-gpu-manager_2.2.9_amd64.deb
1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.