Graph Composer inside Deepstream Container breaks down when trying to launch

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)

GPU

• DeepStream Version

6.0

• JetPack Version (valid for Jetson only)

• TensorRT Version
8.2.0-1

• NVIDIA GPU Driver Version (valid for GPU only)

495.29.05

• Issue Type( questions, new requirements, bugs)

I’m testing graph composer feature. When testing natively, the composer command works fine.

However, we are more interested in having this working inside docker container. I have followed instructions here: Application Workflow — DeepStream 6.1.1 Release documentation

But unlike native version, the composer app does not open and fails, with a log of errors like below:


reparing files...
Done.
./kit/kit ./composer/composer-standalone.kit --ext-path ./composer/omni.exp.graph.composer/  --/plugins/carb.scripting-python.plugin/pythonPath=./registry/ --allow-root
[Info] [carb] Logging to file: /root/.nvidia-omniverse/logs/Kit/composer-standalone/1.0/kit_20211116_013414.log
root@philip1993-Z390-UD:/opt/nvidia/deepstream/deepstream-6.0# [0.225s] [ext: omni.kit.pipapi-0.0.0] startup
[0.247s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.268s] [ext: omni.usd.config-1.0.0] startup
[0.270s] [ext: omni.usd.libs-1.0.0] startup
[0.442s] [ext: omni.stats-0.0.0] startup
[0.444s] [ext: omni.gpu_foundation-0.0.0] startup
2021-11-16 01:34:14 [407ms] [Warning] [omni.ext.plugin] [ext: omni.gpu_foundation-0.0.0] Failed to load plugins, folder doesn't exist: '/tmp/.tmpNvXgRaphComposer/run/kit/plugins/rtx'.
2021-11-16 01:34:14 [407ms] [Warning] [omni.ext.plugin] [ext: omni.gpu_foundation-0.0.0] '/tmp/.tmpNvXgRaphComposer/run/kit/plugins/rtx/rtx.xrtools.plugin' in '[[native.plugin]]' was not found.
[0.449s] [ext: omni.kit.async_engine-0.0.0] startup
[0.461s] [ext: omni.kit.loop-default-0.1.0] startup
[0.462s] [ext: omni.assets.plugins-0.0.0] startup
[0.463s] [ext: omni.appwindow-1.0.0] startup
No protocol specified
2021-11-16 01:34:14 [426ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:14 [426ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
No protocol specified
2021-11-16 01:34:14 [427ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:14 [427ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
2021-11-16 01:34:14 [427ms] [Error] [omni.appwindow.plugin] Failed to acquire IWindowing interface
[0.468s] [ext: omni.client-0.1.0] startup
[0.476s] [ext: omni.kit.renderer.core-0.0.0] startup
2021-11-16 01:34:14 [441ms] [Warning] [omni.ext.plugin] [ext: omni.kit.renderer.core-0.0.0] Failed to load plugins, folder doesn't exist: '/tmp/.tmpNvXgRaphComposer/run/kit/plugins/rtx'.
2021-11-16 01:34:14 [441ms] [Warning] [omni.ext.plugin] [ext: omni.kit.renderer.core-0.0.0] '/tmp/.tmpNvXgRaphComposer/run/kit/plugins/rtx/rtx.xrtools.plugin' in '[[native.plugin]]' was not found.
No protocol specified
Failed to establish dbus connectionNo protocol specified
2021-11-16 01:34:14 [521ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:14 [521ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
No protocol specified
No protocol specified


|---------------------------------------------------------------------------------------------|
| Driver Version: 495.29      | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
|     |                                  |        |     |            | Device-ID | UUID       |
|---------------------------------------------------------------------------------------------|
| 0   | NVIDIA TITAN RTX                 | Yes: 0 |     | 24822   MB | 10de      | 0          |
|     |                                  |        |     |            | 1e02      | b6782f48.. |
|=============================================================================================|
| OS: Linux philip1993-Z390-UD, Version: 5.11.0-40-generic
| Processor: Intel(R) Core(TM) i5-9600K CPU @ 3.70GHz | Cores: Unknown | Logical: 6
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 15939 | Free Memory: 530
| Total Page/Swap (MB): 2047 | Free Page/Swap: 2047
|---------------------------------------------------------------------------------------------|
2021-11-16 01:34:15 [950ms] [Warning] [carb.cudainterop.plugin] NVML could not be loaded. GPU Utilization stats will be unavailable. Error: libnvidia-ml.so: cannot open shared object file: No such file or directory
No protocol specified
2021-11-16 01:34:15 [955ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:15 [955ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
No protocol specified
2021-11-16 01:34:15 [1,212ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:15 [1,212ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
No protocol specified
2021-11-16 01:34:15 [1,212ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2021-11-16 01:34:15 [1,212ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.2]) (impl: carb.windowing-glfw.plugin)
2021-11-16 01:34:15 [1,312ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.events.plugin.so!carbOnPluginStartup
2021-11-16 01:34:15 [1,313ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::Scheduler::*)(unsigned int, int, carb::cpp20::latch*), carb::tasking::Scheduler*, unsigned int, int, carb::cpp20::latch*> > >::_M_run()
2021-11-16 01:34:15 [1,314ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!funlockfile
2021-11-16 01:34:15 [1,314ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.appwindow.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)
2021-11-16 01:34:15 [1,315ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!void std::vector<unsigned long, std::allocator<unsigned long> >::_M_realloc_insert<unsigned long const&>(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long const&)
2021-11-16 01:34:15 [1,315ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx
2021-11-16 01:34:15 [1,315ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx
2021-11-16 01:34:15 [1,316ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx
2021-11-16 01:34:15 [1,316ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.ext.plugin.so!carbOnPluginPostShutdown
2021-11-16 01:34:15 [1,317ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!void std::vector<std::string, std::allocator<std::string> >::_M_realloc_insert<std::string&>(__gnu_cxx::__normal_iterator<std::string*, std::vector<std::string, std::allocator<std::string> > >, std::string&)
2021-11-16 01:34:15 [1,318ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!carbOnPluginShutdown
2021-11-16 01:34:15 [1,319ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!char* std::string::_S_construct<char*>(char*, char*, std::allocator<char> const&, std::forward_iterator_tag)
2021-11-16 01:34:15 [1,320ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2021-11-16 01:34:15 [1,321ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2021-11-16 01:34:15 [1,322ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2021-11-16 01:34:15 [1,323ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::operator=(std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> > const&)
2021-11-16 01:34:15 [1,325ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)
2021-11-16 01:34:15 [1,326ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!_init
2021-11-16 01:34:15 [1,326ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
2021-11-16 01:34:15 [1,327ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!__libc_start_main
2021-11-16 01:34:15 [1,327ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init

For additional details, I have CUDA11.5, CUDNN 8.3.0.98

Docker version 20.10.10, build b485636

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Can be reproduced by running:

docker pull nvcr.io/nvidia/deepstream:6.0-devel
xhost +
docker run -it --entrypoint /bin/bash --gpus all --rm --network=host -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix --privileged -v /var/run/docker.sock:/var/run/docker.sock nvcr.io/nvidia/deepstream:6.0-devel
composer

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Please follow the installation instruction even you are using docker container. System Requirements — DeepStream 6.1.1 Release documentation

The only thing I have not done there is installing the driver, because my current driver is more recent. Are you suggesting me to downgrade the driver? As I said, running composer natively works, but it does not work inside Docker.

Can Failed initialize GLFW - DRIVE PX2 / DRIVE - Linux - NVIDIA Developer Forums help?

I understand that those may be a related issue, but it’s a really old conversation and there are no real solutions suggested in that thread. If there is any package additionally required, why are they not inside the prebuilt docker image for nvcr.io/nvidia/deepstream:6.0-devel in the first place? I can try building the docker image myself, but at least I have to know exactly what’s causing the issue.

I understand that deepstream 6.0 is still at an early stage, but the documentations are quite rudimentary. And by the way, have you tried replicating the issue as I have suggested? If you have, could you provide more concreate solution instead of forwarding to another article?

And If you are not a dev, can I get a formal reply from a developer who may know better about this?

nvcr.io/nvidia/deepstream:6.0-devel is not for Graph Composer

I know it’s not, but using devel image is the only way to call composer within docker according to your documentation. Is this not true?

Any further updates on this?

I’m really surprised by your lack of technical expertise on this, but I am more surprised by the absence of willingness to help. I have no doubt that Deepstream 6.0 is a great new product, but your support definitely do not match the quality of the product. This is my first time asking any questions in your forum, but I guess I had my expectations set up too high.

1 Like

Sorry for late response! It needs some time to identify the problem. Will get back soon.

@philip_onecup This error can be reproduced with some machine. It may related to the docker run command settings. Maybe your DISPLAY port is not “:0”. Please check your display port with “nvidia-settings” command.

I’ve tried the following docker command:

docker run --gpus all -it --net=host --ipc=host --rm -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/$user/:/home/$user/ -e DISPLAY=$DISPLAY --privileged -v /var/run/docker.sock:/var/run/docker.sock -w /opt/nvidia/deepstream/deepstream-6.0 nvcr.io/nvidia/deepstream:6.0-devel

1 Like

Will try those out. Thanks.

Hello,
Any news about this ?
I have a similar problem

Unfortunately it’s still the same. Above solutions do not work, and still throws same error.

Can you run “nvidia-settings --eglinfo | head -5” and tell us the result in your device?

Hi I succeeded to run graph composer by this comment.

but when i try to load reference graph, composer load the graph as screenshot below

with the log
Loading model: /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test1/deepstream-test1.yaml
Component type invalid: nvidia::deepstream::NvDsSingleSrcInput
Component type invalid: nvidia::deepstream::NvDsDynamicOutput
Component type invalid: nvidia::deepstream::NvDsDynamicOutput
Component type invalid: nvidia::deepstream::NvDsStreamMux
Component type invalid: nvidia::deepstream::NvDsOnRequestInput
Component type invalid: nvidia::deepstream::NvDsStaticOutput
Component type invalid: nvidia::deepstream::NvDsConnection
Component type invalid: nvidia::deepstream::NvDsInferVideo
Component type invalid: nvidia::deepstream::NvDsStaticInput
Component type invalid: nvidia::deepstream::NvDsStaticOutput
Component type invalid: nvidia::deepstream::NvDsResnet10_4ClassDetectorModel
Component type invalid: nvidia::deepstream::NvDsConnection
Component type invalid: nvidia::deepstream::NvDsOSD
Component type invalid: nvidia::deepstream::NvDsStaticInput
Component type invalid: nvidia::deepstream::NvDsStaticOutput
Component type invalid: nvidia::deepstream::NvDsConnection
Component type invalid: nvidia::deepstream::NvDsVideoRenderer
Component type invalid: nvidia::deepstream::NvDsStaticInput
Component type invalid: nvidia::deepstream::NvDsConnection
Component type invalid: nvidia::deepstream::NvDsPerClassObjectCounting
Component type invalid: nvidia::deepstream::NvDsProbeConnector
Component type invalid: nvidia::deepstream::NvDsProbe
Component type invalid: nvidia::deepstream::NvDsScheduler

Your extensions are not registered correctly. Can you run “registry extn list” to check your registered extensions?

https://docs.nvidia.com/metropolis/deepstream/dev-guide/graphtools-docs/docs/text/GraphComposer_app_Workflow.html

1 Like

Good day @Fiona.Chen , may I know what model of NVIDIA GPU is needed for graph composer? is it RTX or T4?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.