Issue while Feeding Pytorch TRT 2D-U-net model into Deepstream 6 having same TRT version

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
#1

Hello,

We have trained our Pytorch Unet model and converted that into TRT using the torch2trt but while feeding that into the Deepstream- 6 we are facing the errors logged below we have converted our torch2trt model on the same machine having deepstream having similar TRT version

(pytorch) nkumar@u117282:~/deepstream_trt$ deepstream-segmentation-app -t infer deepstream_tao_apps/configs/unet_tao/pgie_unet_tao_config.txt deepstream_tao_apps/***********67.11704647123835-81.03094573969791.h264 

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.186: External plugin loader failed. This most likely means that the plugin loader helper binary was not found or could not be run. You might need to set the GST_PLUGIN_SCANNER environment variable if your setup is unusual. This should normally not be required though.

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.186: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.187: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_nvblender.so': libgstbadvideo-1.0.so.0: cannot open shared object file: No such file or directory

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.187: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
One element could not be created. Exiting.
Failed to create source bin. Exiting.

My configuration file looks like :

[property]

labelfile-path=unet_labels.txt
model-engine-file=../../models/unet/**_**_ICG_100_2switched.trt_ew
#CD_CD_ICG_100_swiched.trt
#CA_CD_ICGGrayscale.trt
#engine-create-func-name=NvDsInferYoloCudaEngineGet


#tlt-encoded-model=../../models/unet/CA_CD.etlt

net-scale-factor=0.00784313725490196
offsets=127.5;127.5;127.5
infer-dims=3;512;512
#tlt-model-key=CA_CD_ICG
network-type=2
num-detected-classes=3
model-color-format=1
segmentation-threshold=0.0
output-blob-names=softmax_1
segmentation-output-order=1
gie-unique-id=1

Environment

GPU Type: NVIDIA RTX A6000
Nvidia Driver Version: 510.60.02
Operating System : Ubuntu 21:
Python Version : 3.7

deepstream-app version 6.0.1, 
DeepStreamSDK 6.0.1, 
CUDA Driver Version: 11.6, 
CUDA Runtime Version: 11.4, 
TensorRT Version: 8.4, 
cuDNN Version: 8.4, 
libNVWarp360 Version: 2.0.1d3
#2

Hi,

This looks like a Deepstream related issue. We will move this post to the Deepstream forum.

Thanks!

#5
  • Ubuntu 18.04
  • GStreamer 1.14.5
  • NVIDIA driver 470.63.01
  • CUDA 11.4
  • TensorRT 8.0.1
#7

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

#8

Still waiting for a valuable response

#11

Sorry for the confusion, our team will do the investigation and provide suggestions soon. Thanks

#12

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi @nitinp14920914
Can you try this in DeepStream triton docker - nvcr.io/nvidia/deepstream:6.1-triton ?
If you are already using nvcr.io/nvidia/deepstream:6.1-triton, please run below command to check if all the dependent libraries of libnvdsgst_inferserver.so all exist,

$ ldd /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so

#14

Here’s my output for the Command mentioned i think i see all the libraries.

(base) nkxmxxar@u1172xxx:~/deep$ ldd /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so
	linux-vdso.so.1 (0x00007ffc549e3000)
	libcudart.so.11.0 => /usr/local/cuda-11.4/lib64/libcudart.so.11.0 (0x00007fe357c53000)
	libnvds_infer_server.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_infer_server.so (0x00007fe356a7f000)
	libnvds_meta.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_meta.so (0x00007fe356875000)
	libnvdsgst_meta.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvdsgst_meta.so (0x00007fe356670000)
	libnvdsgst_helper.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvdsgst_helper.so (0x00007fe356469000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe35643d000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe35641a000)
	libgstreamer-1.0.so.0 => /lib/x86_64-linux-gnu/libgstreamer-1.0.so.0 (0x00007fe3562d3000)
	libgstbase-1.0.so.0 => /lib/x86_64-linux-gnu/libgstbase-1.0.so.0 (0x00007fe356255000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fe35612c000)
	libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fe3560cc000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe355ee8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe355ecd000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe355cdb000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe358200000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe355cd1000)
	libtritonserver.so => not found
	libnvbufsurftransform.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvbufsurftransform.so (0x00007fe352291000)
	libnvbufsurface.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvbufsurface.so (0x00007fe352088000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe35206c000)
	libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007fe351d96000)
	libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fe351d03000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe351bb4000)
	libgstrtp-1.0.so.0 => /lib/x86_64-linux-gnu/libgstrtp-1.0.so.0 (0x00007fe351b8c000)
	libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fe351b86000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe351b13000)
	libffi.so.7 => /lib/x86_64-linux-gnu/libffi.so.7 (0x00007fe351b07000)
	libcuda.so.1 => /lib/x86_64-linux-gnu/libcuda.so.1 (0x00007fe3502c4000)
	libnppc.so.11 => /usr/local/cuda-11.4/lib64/libnppc.so.11 (0x00007fe34ff3d000)
	libnppig.so.11 => /usr/local/cuda-11.4/lib64/libnppig.so.11 (0x00007fe34de8f000)
	libnppidei.so.11 => /usr/local/cuda-11.4/lib64/libnppidei.so.11 (0x00007fe34d336000)
	libnvbuf_fdmap.so => ///opt/nvidia/deepstream/deepstream-6.0/lib/libnvbuf_fdmap.so (0x00007fe34d128000)
#15

this lib is missed

1 Like
#16

Is this one for Jetson platfom or DGPU ?

#17

both platform have this lib for nvinferserver/Triton

#18

Can you pl help me with any reference Installation Guide for same without using Container/Docker on the Host OS Ubuntu20/18.04 running platform having Dgpu Dual A6000

#19

Waiting for Response

#20

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

#21

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Hi @nitinp14920914
As asked in comment - Issue while Feeding Pytorch TRT 2D-U-net model into Deepstream 6 having same TRT version - #12 by mchi , can you use DeepStream triton docker - nvcr.io/nvidia/deepstream:6.1-triton ?

In this triton docker, as at
ds_triton_docker.txt (11.0 KB)
tached output log of below three commands, all the Triton relevant files, lib path env are all set correctly. If you don’t use DS Triton docker, e.g. installing DS outside of docker or using other DS non-Triton docker, you need to copy the relevant Triton files, set lib path env by himself.

ldd /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so
ls -l /opt/tritonserver/
env

#22
#23

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.