deepstream-test1-app running forever

502291755 · March 14, 2019, 12:18pm

hi,
I run the sample deepstream-test1-app in nvidia-docker container “nvcr.io/nvidia/deepstream:3.0-18.11”.

Nvidia driver version is 418.43.
GPU product name is GeForce GTX 1080 Ti.

But the sample output is as follow and never stop without any error:

root@63ed00bffff5:~/DeepStream_Release/sources/apps/sample_apps/deepstream-test1# ./deepstream-test1-app ../../../../samples/streams/sample_720p.h264 
Now playing: ../../../../samples/streams/sample_720p.h264
Running...

Can someone help me? Thank you!

Amycao · March 17, 2019, 8:07am

Hi,
Can you try deepstream-app -c /path to config/source30_720p_dec_infer-resnet_tiled_display_int8.txt
to see if it can run success? disable gie in the config,
[primary-gie]
enable=0

to make sure nvidia decoder works, since deepstream-test1 app use it

/* Use nvdec_h264 for hardware accelerated decode on GPU */
decoder = gst_element_factory_make (“nvdec_h264”, “nvh264-decoder”);

Amycao · March 18, 2019, 2:24am

or you can directly try GST_DEBUG=4 ./deepstream-test1-app …/…/…/…/samples/streams/sample_720p.h264
to get the log.

kfathima · May 7, 2019, 9:16pm

Hi
I was using Nvidia docker container deepstream:3.0-18.11 on DGX-1 and tried running the sample app as per your last comment
GST_DEBUG=4 ./deepstream-test1-app …/…/…/…/samples/streams/sample_720p.h264 I see following issues still,could pls help.

0:00:00.220803993 12647 0x70e600 INFO GST_TYPEFIND gsttypefind.c:71:gst_type_find_register: registering typefind function for avtype_yuv4mpegpipe
0:00:00.226775225 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstlibav.so” loaded
0:00:00.229939782 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgst1394.so” loaded
0:00:00.230932357 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstamrwbdec.so” loaded
0:00:00.231682847 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstbluez.so” loaded
0:00:00.232717483 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_spot.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.233235630 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_infer.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.233886883 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_tracker.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.234545185 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_multistream.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.235262417 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_videocodecs.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.236312200 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_bboxfilter.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.236937613 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_osd.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.237561832 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_eglglessink.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.239782422 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_aislemetadata.so” loaded
0:00:00.240312259 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_dewarper.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.240803785 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_vidconv.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.242032171 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_msgconv.so” loaded
0:00:00.242375237 12647 0x70e600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_msgbroker.so” loaded
0:00:00.242865646 12647 0x70e600 WARN GST_PLUGIN_LOADING gstplugin.c:748:_priv_gst_plugin_load_file_for_registry: module_open failed: libcuda.so.1: cannot open shared object file: No such file or directory

(gst-plugin-scanner:12647): 0:00:00.253225175 12646 0:00:00.253241560 12646 0:00:00.262073932 12646 0:00:00.287514118 12646 0:00:00.287524970 12646 0:00:00.287528514 12646 0:00:00.287546340 12646 0:00:00.287551350 12646 0:00:00.287555034 12646 0:00:00.287618289 12646 0:00:00.288353550 12646 0:00:00.288363827 12646 0:00:00.288459980 12646 0:00:00.289499692 12646 0:00:00.289510019 12646 0:00:00.289576605 12646 0:00:00.289605920 12646 0:00:00.289624990 12646 0:00:00.289637247 12646 0:00:00.289644324 12646 0:00:00.289651371 12646 0:00:00.289656371 12646 0:00:00.289661114 12646 One element could root@1c431e782872:~/Dee GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstnvdsexample.so’: libcuda.so.1: cannot open shared object file: No such file or directory
0x818600 INFO GST_REGISTRY gstregistry.c:1690:scan_and_update_registry: Registry cache changed. Writing new registry cache
0x818600 INFO GST_REGISTRY gstregistrybinary.c:368:priv_gst_registry_binary_write_cache: Building binary registry cache image
0x818600 INFO GST_REGISTRY gstregistrybinary.c:400:priv_gst_registry_binary_write_cache: Writing binary registry cache
0x818600 INFO GST_REGISTRY gstregistrybinary.c:261:gst_registry_binary_cache_finish: Wrote binary registry cache
0x818600 INFO GST_REGISTRY gstregistry.c:1699:scan_and_update_registry: Registry cache written successfully
0x818600 INFO GST_REGISTRY gstregistry.c:1758:ensure_current_registry: registry reading and updating done, result = 1
0x818600 INFO GST_INIT gst.c:724:init_post: GLib runtime version: 2.48.2
0x818600 INFO GST_INIT gst.c:726:init_post: GLib headers version: 2.48.1
0x818600 INFO GST_INIT gst.c:727:init_post: initialized GStreamer successfully
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:362:gst_element_factory_create: creating element “pipeline” named “dstest1-pipeline”
0x818600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstcoreelements.so” loaded
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:362:gst_element_factory_create: creating element “filesrc” named “file-source”
0x818600 INFO GST_ELEMENT_PADS gstelement.c:659:gst_element_add_pad:GstBaseSrc@0xba80e0 adding pad ‘src’
0x818600 INFO GST_PLUGIN_LOADING gstplugin.c:842:_priv_gst_plugin_load_file_for_registry: plugin “/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvideoparsersbad.so” loaded
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:362:gst_element_factory_create: creating element “h264parse” named “h264-parser”
0x818600 INFO GST_ELEMENT_PADS gstelement.c:659:gst_element_add_pad:GstBaseParse@0xbaca60 adding pad ‘sink’
0x818600 INFO GST_ELEMENT_PADS gstelement.c:659:gst_element_add_pad:GstBaseParse@0xbaca60 adding pad ‘src’
0x818600 INFO baseparse gstbaseparse.c:3942:gst_base_parse_set_pts_interpolation:GstH264Parse@0xbaca60 PTS interpolation: no
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:457:gst_element_factory_make: no such element factory “nvdec_h264”!
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:457:gst_element_factory_make: no such element factory “nvinfer”!
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:457:gst_element_factory_make: no such element factory “nvvidconv”!
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:457:gst_element_factory_make: no such element factory “nvosd”!
0x818600 INFO GST_ELEMENT_FACTORY gstelementfactory.c:457:gst_element_factory_make: no such element factory “nveglglessink”!
not be created. Exiting.
pStream_Release/sources/apps/sample_apps/deepstream-test1#

Amycao · May 8, 2019, 2:43am

HI, kfathima
from below log clip, you can not find libcuda.so.1, Do you install gpu driver? please install gpu driver and
try again
(gst-plugin-scanner:12647): GStreamer-WARNING **: Failed to load plugin ‘/usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst_multistream.so’: libcuda.so.1: cannot open shared object file: No such file or directory

kfathima · May 8, 2019, 10:36pm

Hi ,
Thanks for the reply
inside my deepstream docker container that is running on DGX-1 i did find the respective library but this seems to be different from the one on the DGX-1 server ,so

root@35ca98248b1c:~/DeepStream_Release/sources/apps/sample_apps/deepstream-test1# find /usr/ -name ‘libcuda.so.1’
/usr/local/cuda-10.0/compat/libcuda.so.1
-rw-r–r-- 1 root root 12125448 Sep 12 2018 libnvidia-ptxjitcompiler.so.410.48
lrwxrwxrwx 1 root root 34 Sep 12 2018 libnvidia-ptxjitcompiler.so.1 → libnvidia-ptxjitcompiler.so.410.48
lrwxrwxrwx 1 root root 29 Sep 12 2018 libnvidia-ptxjitcompiler.so → libnvidia-ptxjitcompiler.so.1
-rw-r–r-- 1 root root 292840 Sep 12 2018 libnvidia-fatbinaryloader.so.410.48
-rw-r–r-- 1 root root 15623896 Sep 12 2018 libcuda.so.410.48
lrwxrwxrwx 1 root root 17 Sep 12 2018 libcuda.so.1 → libcuda.so.410.48
lrwxrwxrwx 1 root root 12 Sep 12 2018 libcuda.so → libcuda.so.1

dgx1-1:~# find /usr/ -name ‘libcuda.so.1’
/usr/lib/i386-linux-gnu/libcuda.so.1
/usr/lib/x86_64-linux-gnu/libcuda.so.1
dgx1-1:~# ls -lrt /usr/lib/i386-linux-gnu/
total 21620
lrwxrwxrwx 1 root root 18 Jan 25 15:43 libOpenCL.so.1.0 → libOpenCL.so.1.0.0
lrwxrwxrwx 1 root root 16 Jan 25 15:43 libOpenCL.so.1 → libOpenCL.so.1.0
lrwxrwxrwx 1 root root 27 Jan 25 15:43 libnvidia-opencl.so.1 → libnvidia-opencl.so.384.183
lrwxrwxrwx 1 root root 18 Jan 25 15:43 libcuda.so.1 → libcuda.so.384.183
lrwxrwxrwx 1 root root 12 Jan 25 15:43 libcuda.so → libcuda.so.1
-rw-r–r-- 1 root root 30468 Jan 25 17:16 libOpenCL.so.1.0.0
-rw-r–r-- 1 root root 12273844 Jan 25 17:25 libcuda.so.384.183
-rw-r–r-- 1 root root 9827796 Jan 25 17:26 libnvidia-opencl.so.384.183

Setting env variables PATH LD_LIBRARY_PATH USE_CUDA_PATH within the container did not help either

Amycao · May 10, 2019, 5:31am

HI
Can you find plugin for example nvosd
gst-inspect-1.0 nvosd