Linking custom C++ Triton client shared library with DSExample

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
1

Hello,

• Hardware Platform (GPU)
• DeepStream Version
• TensorRT Version : cuda11.3-trt8.0.1.6-ga
• NVIDIA GPU Driver Version : 470.103.01
• Issue Type : questions

We have C++ based triton client shared library to communicate with Triton server. We have used this as a reference to create C++ based custom triton client. To use Gstreamer and Deepstream processing capabilities, We are planning to use gst-launch to invoke the custom triton client. To do that, We are trying to link the custom triton client shared library with DSExample which can be used in gst-launch.

We would like to know the standard approach to link the shared library with DSExample.

Does the shared library has to be a Gstreamer compliant to be linked with DSExample?

We have tried to link the shared library by providing path and library name to Make file like this, -L<Path to .so file> -l<lib_name>

If library name is libgst_try.so then we provide “gst_try” as lib name. With this approach, We are able to compile the DSExample but it is getting Blacklisted from GStreamer registry. Hence, DSExample element is not getting created.

We even tried to link static library .a file by providing the file name with its path like this, -L<Complete_path_to_static_lib>. This didn’t work as well.

We are exploring other possible approaches. Meanwhile, we would really appreciate the help with linking shared library to DSExample.

Thanks,
Hemang

4

Hi @hemangjethava ,

You can refer to gst-dsexample/Makefile, it’s using -L$(LIB_PATH) -l$(libname_without_lib_and_so), for exmaple, -L/usr/local/cuda-$(CUDA_VER)/lib64/ -lcudart .

No, I think you can link the Triton client libs, e.g. v2.20.0_ubuntu2004.clients.tar.gz under Releases · triton-inference-server/server · GitHub

do you see any error when gst-inspect your libs? for example
$ gst-inspect-1.0 /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so

5

Hi @mchi

We tried above command, It is basically not able to locate the .so at all.

$ /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-dsexample# gst-inspect-1.0 /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so

(gst-inspect-1.0:2734): GStreamer-WARNING **: 07:47:25.855: Failed to load plugin '/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so': libmylib.so: cannot open shared object file: No such file or directory
Could not load plugin file: Opening module failed: libmylib.so: cannot open shared object file: No such file or directory

We manually copied .so file to /usr/lib/ and tried gst-inspect again. It yields below error,

$ /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-dsexample# gst-inspect-1.0 /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so

(gst-inspect-1.0:4395): GStreamer-WARNING **: 07:51:42.832: Failed to load plugin '/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so': /usr/lib/libmylib.so: undefined symbol: _ZN9inference30_ModelConfig_default_instance_E
Could not load plugin file: Opening module failed: /usr/lib/libmylib.so: undefined symbol: _ZN9inference30_ModelConfig_default_instance_E

Let us know if we are missing anything here.

Thanks,
Hemang

6

Hi,

We are trying to link .so file with this approach only, yet it is getting blacklisted.

We are creating executable as well as shared lib using CMakelist.txt. Could that be reason? Like this,

add_library(mylib SHARED custom_triton_client.cc)
  install(
    TARGETS triton_client_exe
    RUNTIME DESTINATION bin
  )

We are able to execute the executable “triton_client_exe” with this approach.

Let me know if you think we are missing something here.

Thanks,
Hemang

7

I think,. this is failure is the reason that it is getting blacklisted. Could you check why there is undefined symbol?
And, as a Triton client, is it just linked to the Triton client libs, e.g. v2.20.0_ubuntu2004.clients.tar.gz under Releases · triton-inference-server/server · GitHub ?

$ /opt/nvidia/deepstream/deepstream-6.0/sources/gst-plugins/gst-dsexample# gst-inspect-1.0 /opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so

(gst-inspect-1.0:4395): GStreamer-WARNING **: 07:51:42.832: Failed to load plugin '/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libnvdsgst_dsexample.so': /usr/lib/libmylib.so: undefined symbol: _ZN9inference30_ModelConfig_default_instance_E
Could not load plugin file: Opening module failed: /usr/lib/libmylib.so: undefined symbol: _ZN9inference30_ModelConfig_default_instance_E
8

Hey @mchi

Nope, We haven’t used the Triton client libs directly. We actually referred the github repo to build the custom triton client lib and specifically customized this file as per our use-case. This simply takes one image to be sent to Triton server using gRPC and get the bbox responses to be overlayed on the image.

We are able to create the executable which works properly, Linking its shared lib with dsexample is not working.

Is it possible that triton client lib have some internal dependencies which are not getting linked?

I would really appreciate if you could describe the step wise instructions to link the shared library(be it custom or default one) with dsexample.

Note: We haven’t added any external dependencies to it apart from whatever was there in the Git repo for triton client to make/compile cpp based triton client executable and shared lib.

Thanks,
Hemang

9

Hey @mchi

We are unsure about that only. We are expecting if you would have came across errors like these and If we can get the reason behind that, We would be able to build the dsexample element.

Thanks,
Hemang

10

Seems this is your own library, isn’t it?

11

Hi @mchi

That’s correct, But that only contains the c++ image client example with some modifications.

We added below line in CMakelist to generate libmylib.so,

add_library(mylib SHARED image_client.cc)

Thus, This liblylib.so only contains the c++ based image client and its dependencies.

Thanks.

12

Hi,

Just to update here, We are able to link a simple print function from c++ triton client code to dsexample.

This was done by adding additional line in CMakelists.txt apart from below line to create shared library.

This was the change,

add_library(mylib SHARED image_client.cc)
set_property(TARGET mylib PROPERTY POSITION_INDEPENDENT_CODE ON)

We can conclude that the approach to link the triton client lib with dsexample is correct. We just need to add additional dependencies for triton client.

UPDATE: We are able to link triton client shared lib with dsexample and it is working now.

Thank you for the support.

Thanks,
Hemang

1 Like
14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.