• Hardware Platform (Jetson / GPU): RTX 2060
• DeepStream Version: 6.1.1 (developer docker image)
**• TensorRT Version: 8.4.1 (docker) **
• NVIDIA GPU Driver Version: 515.65.01
• CUDA Version: 11.7
• Issue Type: Bug
Hi, I’m having trouble launching Composer for Deepstream from the nvcr.io/nvidia/deepstream:6.1.1-devel the latest docker image on a NUC with a RTX 2060 which leads to a loop of error messages.
I run the container using: docker run --gpus all -it --rm --net=host --privileged -v /home/training/inference/:/opt/nvidia/deepstream/deepstream-6.1/host -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.1 nvcr.io/nvidia/deepstream:6.1.1-devel and then I’ll use the composer command to launch the tool. Then I’ll get the following output as a loop until I interrupt it:
root@training-NUC11PHi7:/opt/nvidia/deepstream/deepstream-6.1# composer
Preparing files...
Done.
./kit/kit ./composer/composer-standalone.kit --ext-folder composer --/plugins/carb.scripting-python.plugin/pythonPath=./registry/ --/exts/omni.ui/Window/enableDetach=false --allow-root
[Info] [carb] Logging to file: /root/.nvidia-omniverse/logs/Kit/composer-standalone/2.0/kit_20220913_090307.log
root@training-NUC11PHi7:/opt/nvidia/deepstream/deepstream-6.1# 2022-09-13 09:03:08 [182ms] [Warning] [omni.ext.plugin] [ext: omni_kit] Extensions config 'extension.toml' doesn't exist '/tmp/.tmpNvXgRaphComposer/run/composer/omni_kit' or '/tmp/.tmpNvXgRaphComposer/run/composer/omni_kit/config'
[0.217s] [ext: omni.stats-0.0.0] startup
[0.222s] [ext: omni.gpu_foundation-0.0.0] startup
2022-09-13 09:03:08 [216ms] [Warning] [carb] FrameworkImpl::setDefaultPlugin(client: omni.gpu_foundation_factory.plugin, desc : [carb::graphics::Graphics v2.4], plugin : carb.graphics-vulkan.plugin) failed. Plugin selection is locked, because the interface was previously acquired by:
[0.228s] [ext: carb.windowing.plugins-1.0.0] startup
No protocol specified
2022-09-13 09:03:08 [219ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [219ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
[0.229s] [ext: omni.assets.plugins-0.0.0] startup
[0.230s] [ext: omni.kit.renderer.init-0.0.0] startup
No protocol specified
No protocol specified
2022-09-13 09:03:08 [263ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [263ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
No protocol specified
No protocol specified
|---------------------------------------------------------------------------------------------|
| Driver Version: 515.65.1 | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 2060 | Yes: 0 | | 6390 MB | 10de | 0 |
| | | | | | 1f15 | 698d657a.. |
|=============================================================================================|
| OS: Linux training-NUC11PHi7, Version: 5.15.0-46-generic
| Processor: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz | Cores: Unknown | Logical: 8
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 15666 | Free Memory: 11497
| Total Page/Swap (MB): 975 | Free Page/Swap: 975
|---------------------------------------------------------------------------------------------|
2022-09-13 09:03:08 [325ms] [Error] [gpu.foundation.plugin] Failed opening shader cache version file: /root/.cache/ov/Kit/103.1/66b6689c/shadercache/release/version
2022-09-13 09:03:08 [325ms] [Error] [gpu.foundation.plugin] Failed to process shader cache version file
[0.335s] [ext: omni.kit.pipapi-0.0.0] startup
[0.342s] [ext: omni.usd.config-1.0.0] startup
[0.344s] [ext: omni.usd.libs-1.0.0] startup
[0.459s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.477s] [ext: omni.kit.loop-default-0.1.0] startup
[0.477s] [ext: omni.kit.async_engine-0.0.0] startup
[0.479s] [ext: omni.appwindow-1.0.0] startup
No protocol specified
2022-09-13 09:03:08 [473ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [473ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
No protocol specified
2022-09-13 09:03:08 [473ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [473ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
2022-09-13 09:03:08 [473ms] [Error] [omni.appwindow.plugin] Failed to acquire IWindowing interface
[0.485s] [ext: omni.client-0.1.0] startup
[0.493s] [ext: omni.kit.test-0.0.0] startup
[0.618s] [ext: omni.kit.renderer.core-0.0.0] startup
No protocol specified
2022-09-13 09:03:08 [610ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [610ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
No protocol specified
2022-09-13 09:03:08 [610ms] [Error] [carb.windowing-glfw.plugin] GLFW initialization failed.
2022-09-13 09:03:08 [610ms] [Error] [carb] Failed to startup plugin carb.windowing-glfw.plugin (interfaces: [carb::windowing::IGLContext v1.0],[carb::windowing::IWindowing v1.3]) (impl: carb.windowing-glfw.plugin)
2022-09-13 09:03:08 [623ms] [Error] [carb.crashreporter-breakpad.plugin] crash detected
2022-09-13 09:03:08 [623ms] [Warning] [carb.crashreporter-breakpad.plugin] A crash has occurred. If a debugger should be attached, please set the '/crashreporter/debuggerAttachTimeoutMs' setting to a timeout in milliseconds. This can be used to allow the crash reporter to wait for up to that long for a debugger to attach before processing or sending the crash report.
2022-09-13 09:03:08 [623ms] [Error] [carb.crashreporter-breakpad.plugin] failed to write .dmp to: '/root/.local/share/ov/data/Kit/composer-standalone/2.0/9d7e5a53-8f57-4c58-79762498-3648be04.dmp'
2022-09-13 09:03:08 [624ms] [Fatal] [carb.crashreporter-breakpad.plugin] Crash detected in pid 12 thread 12
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] Crash metadata:
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] DumpId = ''
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] ProductName = 'OmniverseKit'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] StartupTime = '1663059787'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] UptimeSeconds = ''
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] Version = '103.1+release.7031.66b6689c.tc'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] appName = 'composer-standalone'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] appState = 'startup'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] appVersion = '2.0.0'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] autoloadExts = ''
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildBranch = 'release'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildCi = 'tc'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildCiNumber = '9839284'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildHash = '66b6689c'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildMajor = '103'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildMinor = '1'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildMr = '0'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildNumber = '7031'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildPatch = '0'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] buildVersion = '103.1.0'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] environmentName = 'default'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] kitRendererDriverVersion = '515.65'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] memoryStats = ''
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] portableMode = '0'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] systemInfo = '
|---------------------------------------------------------------------------------------------|
| Driver Version: 515.65.1 | Graphics API: Vulkan
|=============================================================================================|
| GPU | Name | Active | LDA | GPU Memory | Vendor-ID | LUID |
| | | | | | Device-ID | UUID |
|---------------------------------------------------------------------------------------------|
| 0 | NVIDIA GeForce RTX 2060 | Yes: 0 | | 6390 MB | 10de | 0 |
| | | | | | 1f15 | 698d657a.. |
|=============================================================================================|
| OS: Linux training-NUC11PHi7, Version: 5.15.0-46-generic
| Processor: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz | Cores: Unknown | Logical: 8
|---------------------------------------------------------------------------------------------|
| Total Memory (MB): 15666 | Free Memory: 11497
| Total Page/Swap (MB): 975 | Free Page/Swap: 975
|---------------------------------------------------------------------------------------------|
'
2022-09-13 09:03:08 [624ms] [Warning] [carb.crashreporter-breakpad.plugin] telemetrySessionId = ''
2022-09-13 09:03:08 [624ms] [Fatal] [carb.crashreporter-breakpad.plugin] Thread 12 backtrace follows:
2022-09-13 09:03:08 [630ms] [Fatal] [carb.crashreporter-breakpad.plugin] 000: libcarb.events.plugin.so!std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()+0xd64 (??:?)
2022-09-13 09:03:08 [635ms] [Fatal] [carb.crashreporter-breakpad.plugin] 001: libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::Scheduler::*)(unsigned int, int, carb::cpp20::latch*), carb::tasking::Scheduler*, unsigned int, int, carb::cpp20::latch*> > >::_M_run()+0x5f4 (??:?)
2022-09-13 09:03:08 [639ms] [Fatal] [carb.crashreporter-breakpad.plugin] 002: libcarb.assets.plugin.so!std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned long, unsigned long)+0x8e84 (??:?)
2022-09-13 09:03:08 [643ms] [Fatal] [carb.crashreporter-breakpad.plugin] 003: libcarb.datasource-file.plugin.so!std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()+0x424 (??:?)
2022-09-13 09:03:08 [647ms] [Fatal] [carb.crashreporter-breakpad.plugin] 004: libgpu.foundation.plugin.so!std::fill(std::_Bit_iterator, std::_Bit_iterator, bool const&)+0x374 (??:?)
2022-09-13 09:03:08 [651ms] [Fatal] [carb.crashreporter-breakpad.plugin] 005: libcarb.datasource-omniclient.plugin.so!std::_V2::condition_variable_any::~condition_variable_any()+0x1074 (??:?)
2022-09-13 09:03:08 [654ms] [Fatal] [carb.crashreporter-breakpad.plugin] 006: libomni.kit.renderer.plugin.so!std::__future_base::_Result<carb::datasource::Connection*>::_M_destroy()+0x3234 (??:?)
2022-09-13 09:03:08 [658ms] [Fatal] [carb.crashreporter-breakpad.plugin] 007: libpthread.so.0!funlockfile+0x60 (??:?)
2022-09-13 09:03:08 [661ms] [Fatal] [carb.crashreporter-breakpad.plugin] 008: libomni.appwindow.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)+0x4772 (??:?)
2022-09-13 09:03:08 [664ms] [Fatal] [carb.crashreporter-breakpad.plugin] 009: libomni.kit.renderer.plugin.so!void std::vector<unsigned long, std::allocator<unsigned long> >::_M_realloc_insert<unsigned long const&>(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, unsigned long const&)+0x4b7e (??:?)
2022-09-13 09:03:08 [667ms] [Fatal] [carb.crashreporter-breakpad.plugin] 010: libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx+0xb29 (??:?)
2022-09-13 09:03:08 [671ms] [Fatal] [carb.crashreporter-breakpad.plugin] 011: libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx+0x2950 (??:?)
2022-09-13 09:03:08 [674ms] [Fatal] [carb.crashreporter-breakpad.plugin] 012: libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx+0x449e (??:?)
2022-09-13 09:03:08 [678ms] [Fatal] [carb.crashreporter-breakpad.plugin] 013: libomni.kit.renderer.ext.plugin.so!carbOnPluginPostShutdown+0x431 (??:?)
2022-09-13 09:03:08 [683ms] [Fatal] [carb.crashreporter-breakpad.plugin] 014: libomni.ext.plugin.so!void std::vector<std::string, std::allocator<std::string> >::_M_realloc_insert<std::string&>(__gnu_cxx::__normal_iterator<std::string*, std::vector<std::string, std::allocator<std::string> > >, std::string&)+0x3d87 (??:?)
2022-09-13 09:03:08 [688ms] [Fatal] [carb.crashreporter-breakpad.plugin] 015: libomni.ext.plugin.so!std::pair<std::__detail::_Node_iterator<std::string, true, true>, bool> std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::_M_insert<std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > >(std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > const&, std::integral_constant<bool, true>)+0x4604 (??:?)
2022-09-13 09:03:08 [693ms] [Fatal] [carb.crashreporter-breakpad.plugin] 016: libomni.ext.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)+0x163a6 (??:?)
2022-09-13 09:03:08 [697ms] [Fatal] [carb.crashreporter-breakpad.plugin] 017: libomni.ext.plugin.so!carbOnPluginShutdown+0x5928 (??:?)
2022-09-13 09:03:08 [701ms] [Fatal] [carb.crashreporter-breakpad.plugin] 018: libomni.ext.plugin.so!carbOnPluginShutdown+0xb60a (??:?)
2022-09-13 09:03:08 [706ms] [Fatal] [carb.crashreporter-breakpad.plugin] 019: libomni.ext.plugin.so!carbOnPluginShutdown+0xce56 (??:?)
2022-09-13 09:03:08 [710ms] [Fatal] [carb.crashreporter-breakpad.plugin] 020: libomni.kit.app.plugin.so!void std::vector<std::string, std::allocator<std::string> >::_M_realloc_insert<std::string>(__gnu_cxx::__normal_iterator<std::string*, std::vector<std::string, std::allocator<std::string> > >, std::string&&)+0xec7b (??:?)
2022-09-13 09:03:08 [716ms] [Fatal] [carb.crashreporter-breakpad.plugin] 021: libomni.kit.app.plugin.so!_init+0x477e (??:0)
2022-09-13 09:03:08 [721ms] [Fatal] [carb.crashreporter-breakpad.plugin] 022: libomni.kit.app.plugin.so!carbOnPluginPostShutdown+0x6d6 (??:?)
2022-09-13 09:03:08 [725ms] [Fatal] [carb.crashreporter-breakpad.plugin] 023: kit!_init+0x5e2 (??:0)
2022-09-13 09:03:08 [736ms] [Fatal] [carb.crashreporter-breakpad.plugin] 024: libc.so.6!__libc_start_main+0xf3 (??:?)
2022-09-13 09:03:08 [740ms] [Fatal] [carb.crashreporter-breakpad.plugin] 025: kit!_init+0x93b (??:0)
2022-09-13 09:03:08 [740ms] [Fatal] [carb.crashreporter-breakpad.plugin] Warning: Breakpad failed to save crash minidump. This can be caused by a failure in the child process cloned to debug this process, or a failure to attach to this process (i.e. already being debugged).
2022-09-13 09:03:08 [744ms] [Error] [carb.crashreporter-breakpad.plugin] crash detected
I was able to reproduce this behavior with the DS 6.1 docker image as well as on a totally different computer (RTX 2070 Super).
What can be done to fix this and make composer work?
I’ happy to provide further information if needed. Thank’s in advance.
Florian