Deepstream + triton infer server

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU): Jetson
• DeepStream Version: 7.1
• JetPack Version (valid for Jetson only): 6.1
*• Issue Type( questions, new requirements, bugs): question

I have a docker file where I manually install deepstream-7.1 using apt. I am trying to use triton and I need the nvinferserver plugin which is missing.
I was following the README that can be found at the directory /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins/gst-nvinferserver inde the deepstream directory. I have followed the steps and built the library without any problem, but when I try gst-inspect-1.0 nvinferserver the plugin is not found. Inside the gst+deepstream plugins folder (/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream) i can see the library libnvdsgst_inferserver.so, so why I cannot see the plugin using gst-inspect?

This is the dockerfile (with just the main lines):

FROM nvcr.io/nvidia/l4t-jetpack:r36.4.0

RUN echo "deb https://repo.download.nvidia.com/jetson/common r36.4 main" | tee -a /etc/apt/sources.list.d/nvidia-l4t-apt-source.list && \
    echo "deb https://repo.download.nvidia.com/jetson/t234 r36.4 main" | tee -a /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
RUN apt-key adv --fetch-key https://repo.download.nvidia.com/jetson/jetson-ota-public.asc

RUN apt update
RUN apt install -y libtool deepstream-7.1 git-all python3-pip cmake unzip gcc python3-dev autoconf libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstrtspserver-1.0-dev libcairo2-dev wget \
    libjpeg-dev zlib1g-dev libtiff-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk redis-tools libgirepository1.0-dev \
    gstreamer1.0-x

RUN apt install -y libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev \
        libprotobuf-dev protobuf-compiler curl
RUN mkdir -p  /opt/tritonclient/
RUN wget https://github.com/triton-inference-server/server/releases/download/v2.49.0/tritonserver2.49.0-igpu.tar.gz
RUN tar xzf tritonserver2.49.0-igpu.tar.gz -C /opt/tritonclient/ --strip-components=2 tritonserver/clients/lib/ tritonserver/clients/include
RUN mkdir /opt/proto
ENV PB_REL="https://github.com/protocolbuffers/protobuf/releases"
RUN curl -LO $PB_REL/download/v21.12/protoc-21.12-linux-aarch_64.zip
RUN unzip protoc-21.12-linux-aarch_64.zip -d /opt/proto
RUN chmod -R +rx /opt/proto/
WORKDIR /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins/gst-nvinferserver
ENV CUDA_VER=12.6
RUN make && make install
ENV PATH=/opt/proto/bin/:$PATH

ENTRYPOINT ["bash"]

Can you share the log of below command line?

$ gst-inspect-1.0 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so

root@ac86d1b7c288:/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream# ls | grep inferser
libnvdsgst_inferserver.so

root@ac86d1b7c288:/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream# gst-inspect-1.0 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so

(gst-plugin-scanner:30): GStreamer-WARNING **: 09:20:45.230: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so’: libtritonserver.so: cannot open shared object file: No such file or directory

(gst-plugin-scanner:30): GStreamer-WARNING **: 09:20:45.232: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so’: librivermax.so.0: cannot open shared object file: No such file or directory
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 205)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 107)

(gst-inspect-1.0:29): GStreamer-WARNING **: 09:20:45.424: Failed to load plugin ‘/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so’: libtritonserver.so: cannot open shared object file: No such file or directory
Could not load plugin file: Opening module failed: libtritonserver.so: cannot open shared object file: No such file or directory

The error show can’t find triton server library. Can you install the library to: DEEPSTREAM_LIB_DIR=/opt/nvidia/deepstream/deepstream/lib?
Please refer: /opt/nvidia/deepstream/deepstream/samples/triton_backend_setup.sh

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.