Installing DeepStream v1.5 on Jetson TX2 with Jetpack 3.2

Hi,

I am currently trying to install the DeepStream SDK (v1.5) on a Jetson TX2 flashed with Jetpack 3.2. However, after following the instructions in the ReadMe, I try to run the sample application and get these error messages:

nvidia@tegra-ubuntu:~$ nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt
------------>  -----------------
** ERROR: <create_secondary_gie:105>: Failed to create 'secondary_gie_0'
** ERROR: <create_secondary_gie:235>: create_secondary_gie failed
** ERROR: <create_secondary_gie_bin:364>: create_secondary_gie_bin failed
** ERROR: <create_processing_instance:926>: create_processing_instance failed
** ERROR: <create_pipeline:1062>: create_pipeline failed
** ERROR: <main:384>: Failed to create pipeline
Quitting
App run failed

I then try to make from source and get these errors:

nvidia@tegra-ubuntu:~/nvgstiva-app_sources/nvgstiva-app$ make
cc -o nvgstiva-app nvgstiva_app_main.o src/nvgstiva_source_bin.o src/nvgstiva_metadata_pool.o src/nvgstiva_color_detector_bin.o src/nvgstiva_config_file_parser.o src/nvgstiva_common.o src/nvgstiva_split_bin.o src/nvgstiva_sink_bin.o src/nvgstiva_perf.o src/nvgstiva_app.o src/nvgstiva_dsexample.o src/nvgstiva_secondary_gie_bin.o src/nvgstiva_face_detector_bin.o src/nvgstiva_osd_bin.o src/nvgstiva_tracker_bin.o src/nvgstiva_primary_gie_bin.o -lgstnvivameta -lnvid_mapper `pkg-config --libs gstreamer-1.0`
/usr/bin/ld: cannot find -lnvid_mapper
collect2: error: ld returned 1 exit status
Makefile:37: recipe for target 'nvgstiva-app' failed
make: *** [nvgstiva-app] Error 1

I’ve found other topics with the same problems, but it seems like they were created a few months ago and were with DeepStream v1.0, which didn’t support Jetpack 3.2. Is there something else I’m missing?

Any help or pointers in the right direction would be greatly appreciated!

Do you have the following library below?

root@tx2:/usr/src/deep_stream# ls -la /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0
-rwxrwxr-x 1 root root 7744 May 17 03:24 /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0

root@tx2:/usr/src/deep_stream# ldd nvgstiva-app | grep nvid_mapper
libnvid_mapper.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0 (0x0000007f8b688000)

This library is not a part of any packages, but it was included in the nvidia_drivers tarball in JetPack 3.2.1. I would assume that it was also included in JetPack 3.2? But if it’s not, you can always extract it from JetPack 3.2.1 and copy over to your system.

On a side note, I have nothing but the problems with DeepStream 1.5, while 1.0 was working for me. I hope NVidia will release 2.0 soon to address the issues found in 1.5.

-albertr

Ok, the problems with DeepStream 1.5 on our system here were addressed, and it works as well as the old DeepStream 1.0. Looking forward to upcoming DeepStream 2.0.

-albertr

Hi albertr,

Thank you so much for your help! We have extracted the nvidia_drivers from the L4T installation.

After running the first command, we get as expected:

nvidia@tegra-ubuntu:~$ ls -la /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0
-rwxrwxr-x 1 root root 7744 Dec 22  2017 /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0

However, when we run the second command, we get:

nvidia@tegra-ubuntu:~$ ldd nvgstiva-app | grep nvid_mapper
ldd: ./nvgstiva-app: No such file or directory

I tried to go into the nvgstiva-app-sources directory and run it, and I get:

nvidia@tegra-ubuntu:~/nvgstiva-app_sources$ ldd nvgstiva-app | grep nvid_mapper
ldd: ./nvgstiva-app: not regular file

Any suggestions what might be going on? Thanks again for your help.

Did you compile nvgstiva-app from the source code? There’s also a pre-compiled binary included in DeepStream SDK which will be extracted to /usr/bin/nvgstiva-app when you untar the DeepStream tarball.

Could you do the following?

ls -la /usr/bin/nvgstiva-app
file /usr/bin/nvgstiva-app
ldd /usr/bin/nvgstiva-app

Also, don’t forget to execute ldconfig after extracting libraries from nvidia_drivers tarball.

-albertr

I run into the cannot find -lnvid_mapper error when I try to compile from nvgstiva-app_sources/nvgstiva-app.

When I run those commands I get this output:

nvidia@tegra-ubuntu:~$ ls -la /usr/bin/nvgstiva-app
-rwxrwxr-x 1 root root 110608 May 21 17:15 /usr/bin/nvgstiva-app
nvidia@tegra-ubuntu:~$ file /usr/bin/nvgstiva-app
/usr/bin/nvgstiva-app: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, stripped
nvidia@tegra-ubuntu:~$ ldd /usr/bin/nvgstiva-app
        linux-vdso.so.1 =>  (0x0000007f93e6a000)
        libgstreamer-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 (0x0000007f93d26000)
        libglib-2.0.so.0 => /lib/aarch64-linux-gnu/libglib-2.0.so.0 (0x0000007f93c1f000)
        libgobject-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0 (0x0000007f93bc3000)
        libgstnvivameta.so => /usr/lib/aarch64-linux-gnu/libgstnvivameta.so (0x0000007f93bb0000)
        libnvid_mapper.so.1.0.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so.1.0.0 (0x0000007f93b9e000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f93a57000)
        libgmodule-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgmodule-2.0.so.0 (0x0000007f93a42000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f93995000)
        libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f93982000)
        libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f93955000)
        /lib/ld-linux-aarch64.so.1 (0x000000556db11000)
        libpcre.so.3 => /lib/aarch64-linux-gnu/libpcre.so.3 (0x0000007f938e4000)
        libffi.so.6 => /usr/lib/aarch64-linux-gnu/libffi.so.6 (0x0000007f938cc000)
        libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f9373c000)
        libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f9371b000)
nvidia@tegra-ubuntu:~$

Another interesting thing is that now when I try running the sample code, the error I get now tells me:

nvidia@tegra-ubuntu:~$ nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt

(nvgstiva-app:6354): GStreamer-WARNING **: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstomx.so': libEGL.so.1: cannot open shared object file: No such file or directory
** ERROR: <create_render_bin:59>: Failed to create 'sink_sub_bin_sink1'
** ERROR: <create_render_bin:162>: create_render_bin failed
** ERROR: <create_sink_bin:410>: create_sink_bin failed
** ERROR: <create_processing_instance:926>: create_processing_instance failed
** ERROR: <create_pipeline:1062>: create_pipeline failed
** ERROR: <main:384>: Failed to create pipeline
Quitting
App run failed

Thanks again for all your help!

There seems to be something wrong with your installation. Was it a clean install of Jetpack 3.2, no errors during installation?

libEGL is also included in the nvidia_drivers tarball, you might want to try to extract it from there. What is concerning is why these libraries are missing from your system in the first place? If there’re other libraries missing as well, you might end up extracting everything from the nvidia_drivers tarball.

root@tx2:~# ldd /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstomx.so | grep libEGL
libEGL.so.1 => /usr/lib/aarch64-linux-gnu/tegra-egl/libEGL.so.1 (0x0000007f8bb26000)
root@tx2:~#

-albertr

I believe it was a clean install of Jetpack 3.2. I tried extracting everything from nvidia_drivers, and when I go to /usr/lib/aarch64-linux-gnu/tegra-egl, I see libEGL.so.1. However, it seems that the linking isn’t properly working. How do I go about fixing this, or would I have to flash everything from the beginning again?

Don’t forget to execute ldconfig after adding new libraries. The /usr/lib/aarch64-linux-gnu/tegra-egl directory is included in the library search path as per aarch64-linux-gnu_EGL.conf file, please see below.

-albertr

root@tx2:/usr/lib/aarch64-linux-gnu/tegra-egl# cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf

root@tx2:/usr/lib/aarch64-linux-gnu/tegra-egl# cd /etc/ld.so.conf.d
root@tx2:/etc/ld.so.conf.d# ls -la
total 44
drwxr-xr-x 2 root root 4096 Jul 13 20:19 .
drwxr-xr-x 152 root root 12288 Jul 16 10:17 …
-rw-r–r-- 1 root root 46 May 5 2017 .cuda-8-0.conf.orig
-rw-r–r-- 1 root root 70 Apr 14 2016 aarch64-linux-gnu.conf
lrwxrwxrwx 1 root root 44 May 3 2016 aarch64-linux-gnu_EGL.conf -> /etc/alternatives/aarch64-linux-gnu_egl_conf
lrwxrwxrwx 1 root root 43 May 3 2016 aarch64-linux-gnu_GL.conf -> /etc/alternatives/aarch64-linux-gnu_gl_conf
-rw-r–r-- 1 root root 46 Feb 27 20:21 cuda-9-0.conf
-rw-rw-r-- 1 root root 39 Nov 24 2014 fakeroot-aarch64-linux-gnu.conf
-rw-r–r-- 1 root root 44 Jan 27 2016 libc.conf
-rw-rw-r-- 1 root root 33 Nov 11 2017 nvidia-tegra.conf
-rw-r–r-- 1 root root 19 Jul 13 15:20 zed.conf
root@tx2:/etc/ld.so.conf.d# pwd
/etc/ld.so.conf.d
root@tx2:/etc/ld.so.conf.d# cat /etc/alternatives/aarch64-linux-gnu_egl_conf
/usr/lib/aarch64-linux-gnu/tegra-egl
root@tx2:/etc/ld.so.conf.d#

If you have nothing to loose on your existing system, I would suggest to re-flash Jetpack again, maybe even reflash with the latest Jetpack 3.2.1 release. Re-flashing was not an option for me, since we had quite a few customizations made and cannot afford to lose them.

-albertr

Hi albertr,

Thanks for your help! We would also prefer to not have to flash our Jetpack…however, after running

sudo ln -s /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so /usr/lib/aarch64-linux-gnu/libnvid_mapper.so

I managed to get the source code compiling.

However, now I get this error when running the demo application:

nvidia@tegra-ubuntu:~/nvgstiva-app_sources/nvgstiva-app$ ./nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt
------------>  -----------------
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0:12,0,0:13,0,0:14,0,0:15,0,0:16,128,1920:17,0,100:18,0,1920:19,0,192                                                0:20,0,0:21,0,0:22,0,0:23,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0:12,0,0:13,0,0:14,0,0:15,0,0:16,1080,1:17,1080,1:                                                18,1080,4:19,1080,4:20,0,0:21,0,0:22,0,0:23,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
** ERROR: <create_tracking_bin:31>: Failed to create 'tracking_tracker'
** ERROR: <create_tracking_bin:50>: create_tracking_bin failed
** ERROR: <create_processing_instance:926>: create_processing_instance failed
** ERROR: <create_pipeline:1062>: create_pipeline failed
** ERROR: <main:384>: Failed to create pipeline
Quitting
App run failed
Segmentation fault (core dumped)

Is there a way for me to run this with some kind of verbose option to see exactly where creating the tracker is failing?

I might be wrong, but maybe it’s not able to find nvtracker gstreamer plugin?
You can try to enable debug output in gstreamer to see what’s going on, i.e.:

GST_DEBUG=“5” ./nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt

or check out command line options of nvgstiva-app, i.e.: ./nvgstiva-app --gst-debug-help

Does the following command returns the same results as below?

root@tx2:/usr/lib/aarch64-linux-gnu/gstreamer-1.0# gst-inspect-1.0 | grep track
nvtracker: nvtracker: NvTracker plugin

-albertr

You’re right - when I run the GST_DEBUG I get “no such element factory “nvtracker”!” message. The second command also returns nothing.

However, I’m sure I have those libraries installed as libgstnvtracker.so is in /usr/lib/aarch64-linux-gnu/gstreamer-1.0. I also followed the installation instructions for DeepStream and didn’t get any errors. Should I edit any of my path variables to include that directory?

You can redefine the directory gstreamer is looking for plugins in via environment variable GST_PLUGIN_PATH, i.e.:

GST_PLUGIN_PATH=/usr/lib/aarch64-linux-gnu/gstreamer-1.0 ./nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt

-albertr

BTW, did you update gstreamer plugins from the ones included in the nvgstapps.tbz2 tarball in Jetpack 3.2.1?

-albertr

Hi albertr, thank you so much for all your help.

It ended up being that we didn’t have the VisionWorks libs (so something was funky with our jetpack flashing after all), so we reflashed the device and got farther. However, we are still running into these issues:

nvidia@tegra-ubuntu:~$ nvgstiva-app -c /home/nvidia/configs/PGIE-FP16-CarType-CarMake-CarColor.txt
------------>  -----------------
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
------------> 0,128,128:1,0,0:2,0,0:3,0,0:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0:12,0,0:13,0,0:14,0,0:15,0,0:16,128,1920:17,0,100:18,0,1920:19,0,1920:20,0,0:21,0,0:22,0,0:23,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080:3,1920,1080:4,0,0:5,0,0:6,0,0:7,0,0:8,0,0:9,0,0:10,0,0:11,0,0:12,0,0:13,0,0:14,0,0:15,0,0:16,1080,1:17,1080,1:18,1080,4:19,1080,4:20,0,0:21,0,0:22,0,0:23,0,0: -----------------
------------>  -----------------
                CLASS_IDS TO OEPRATE ON gie_id(0) : 2:
------------> 0,0:1,0:2,0: -----------------
------------> 0,0:1,0:2,0: -----------------
------------> 0,0,0:1,0,0:2,0,0: -----------------
------------> 0,1920,1080:1,100,1080:2,1920,1080: -----------------
------------> 0,0.200000,0.100000,3,0:1,0.200000,0.100000,3,0:2,0.200000,0.100000,3,0: -----------------
CPU usage measurement requires governor to be set to performance------------> 0,0.0,1.0,0.0:1,0.0,1.0,1.0:2,0.0,0.0,1.0:3,1.0,1.0,0.0 -----------------
Error while initialize EGL display connection
nvbuf_utils: Invalid width=0 height=0 layout=0
NvBufferCreateEx with memtag 0 failed
nvcaffegie: NvBufferCreateEx failed

We found another post with this issue but it just suggested clearing the cache, which didn’t work for us. Upon running gst-inspect-1.0, we get this:

nvidia@tegra-ubuntu:/usr/lib/aarch64-linux-gnu/gstreamer-1.0$ gst-inspect-1.0

ERROR: Caught a segmentation fault while loading plugin file:
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstclutter-3.0.so

Please either:
- remove it and restart.
- run with --gst-disable-segtrap --gst-disable-registry-fork and debug.

ERROR: Caught a segmentation fault while loading plugin file:
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstclutter-3.0.so

Please either:
- remove it and restart.
- run with --gst-disable-segtrap --gst-disable-registry-fork and debug.
goom:  goom: GOOM: what a GOOM!
(and then more lines with all the plugins)

Is something still wrong with our installation?

Thanks

What was the kernel stack trace (it should be logged in the kernel log) when it was segfault’ing? Anyways, since you have re-flashed your system with JetPack, it should have brought it to a well known state where you should not see these errors… you might want to review your flashing procedures. I would suggest to follow onscreen instructions during flashing and to make sure there’re no errors are being thrown during the process.

-albertr

I haven’t re-flashed JetPack for awhile and don’t remember how component selection screen looks like and what is the default selection, but make sure you don’t deselect any of the required components. Maybe better yet - select everything to be sure.

-albertr

Hi, st585

I have the same environment with yours. After clear cache, I get the error messages different with you:

nvidia@tegra-ubuntu:~/nvgstiva-app_sources/nvgstiva-app$ sudo rm -rf ${HOME}/.cache/*
nvidia@tegra-ubuntu:~/nvgstiva-app_sources/nvgstiva-app$ nvgstiva-app -c ${HOME}/configs/PGIE-FP16-CarType-CarMake-CarColor.txt

(gst-plugin-scanner:9680): GStreamer-WARNING **: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcaffegie.so': /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcaffegie.so: undefined symbol: NvBufferCreateEx

(gst-plugin-scanner:9680): GStreamer-WARNING **: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvtracker.so': /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvtracker.so: undefined symbol: NvBufferCreateEx

(gst-plugin-scanner:9680): GStreamer-WARNING **: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvclrdetector.so': /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvclrdetector.so: undefined symbol: NvBufferCreateEx
------------>  -----------------
** ERROR: <create_secondary_gie:105>: Failed to create 'secondary_gie_0'
** ERROR: <create_secondary_gie:235>: create_secondary_gie failed
** ERROR: <create_secondary_gie_bin:364>: create_secondary_gie_bin failed
** ERROR: <create_processing_instance:926>: create_processing_instance failed
** ERROR: <create_pipeline:1062>: create_pipeline failed
** ERROR: <main:384>: Failed to create pipeline
Quitting
App run failed

Could you kindly share your opinion about this DeepStream install question? Thank you very much!

feng.qi,

If you can re-flash your Jetson, I would suggest to do so while making sure you select ALL components during flash procedure. If you cannot re-flash, you still can extract all libraries from Jetpack and copy them over manually. Re-run ldconfig after copying them over.

-albertr