Hello @jlafleche
Thanks for your reply
I follow your solutions
-
Apply the dataset with 23 usd files with default resolution value 224: Run out of memory, and kaolin is shutted down
-
Apply the single usd file one by one with low resolution from the default value 224 to 50: Same situation as the former
Unfortunately, it may not work. QAQ
Until now, the way I could use in manually is
- Open kaolin → Asset Conversion into point cloud format (Default Resolution) → Close kaolin
- Repeat this operation until I convert all usd assets in the dataset.
I also posted another article in this forum.
It would work that using data generator let all dataset (22 usd files) convert into RGB. But when I would like to apply whole dataset (22 usd files) for the conversion of “Point Cloud”, it would happen memory leak. Even if I convert it one by one, the first operation does work. However, the second one failed. The Kaolin App would be automatically shutted down.
When executing the “Point Colud” conversion for the second usd file, both of my GPU is out of memory as shown in the figure.
2022-07-07 06:45:37 [136,816ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2022-07-07 06:45:37 [136,816ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed for flags: 0.
2022-07-07 06:45:37 [136,816ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2022-07-07 06:45:37 [136,816ms] [Error] [gpu.foundation.plugin] Failed to update params for RenderOp 25
2022-07-07 06:45:37 [136,816ms] [Error] [gpu.foundation.plugin] Failed to update params for RenderOp Synthetic Image Data RT. Will not execute this or subsequent RenderGraph operations. Aborting RenderGraph execution
2022-07-07 06:45:37 [136,816ms] [Error] [carb.scenerenderer-rtx.plugin] Failed to execute RenderGraph on device 0. Error Code: 2
2022-07-07 06:45:37 [136,819ms] [Error] [carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_DEVICE_MEMORY
2022-07-07 06:45:37 [136,819ms] [Error] [carb.graphics-vulkan.plugin] vkAllocateMemory failed for flags: 0.
2022-07-07 06:45:37 [136,819ms] [Error] [gpu.foundation.plugin] Texture creation failed for the device: 0.
2022-07-07 06:45:37 [136,821ms] [Error] [omni.usd] SceneRenderer failed to render view. Aborting this frame for this view.
2022-07-07 06:45:37 [136,823ms] [Error] [gpu.foundation.plugin] No RpResource was allocated, getResourceBuffer failed
Fatal Python error: Segmentation fault
Current thread 0x00007fa7bb774740 (most recent call first):
2022-07-07 06:45:38 [137,446ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.events.plugin.so!carbOnPluginStartup
2022-07-07 06:45:38 [137,451ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::Scheduler::)(unsigned int, int, carb::cpp20::latch), carb::tasking::Scheduler*, unsigned int, int, carb::cpp20::latch*> > >::_M_run()
2022-07-07 06:45:38 [137,455ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!funlockfile
2022-07-07 06:45:38 [137,459ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!raise
2022-07-07 06:45:38 [137,463ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!funlockfile
2022-07-07 06:45:38 [137,466ms] [Fatal] [carb.crashreporter-breakpad.plugin] libgpu.foundation.so!void std::this_thread::sleep_for<long, std::ratio<1l, 1000000l> >(std::chrono::duration<long, std::ratio<1l, 1000000l> > const&)
2022-07-07 06:45:38 [137,467ms] [Fatal] [carb.crashreporter-breakpad.plugin] librtx.scenedb.plugin.so!void std::vector<unsigned long*, std::allocator<unsigned long*> >::_M_realloc_insert<unsigned long*>(__gnu_cxx::__normal_iterator<unsigned long**, std::vector<unsigned long*, std::allocator<unsigned long*> > >, unsigned long*&&)
2022-07-07 06:45:38 [137,470ms] [Fatal] [carb.crashreporter-breakpad.plugin] libgpu.foundation.so!std::vector<unsigned int, std::allocator >::_M_default_append(unsigned long)
2022-07-07 06:45:38 [137,471ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.scenerenderer-rtx.plugin.so!_init
2022-07-07 06:45:38 [137,472ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.scenerenderer-rtx.plugin.so!_init
2022-07-07 06:45:38 [137,473ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.scenerenderer-rtx.plugin.so!_init
2022-07-07 06:45:38 [137,476ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.scenerenderer-mux.plugin.so!carbOnPluginShutdown
2022-07-07 06:45:38 [137,479ms] [Fatal] [carb.crashreporter-breakpad.plugin] librtx.hydra.so!std::__detail::_Map_base<unsigned short, std::pair<unsigned short const, std::string>, std::allocator<std::pair<unsigned short const, std::string> >, std::__detail::_Select1st, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>, true>::operator(unsigned short const&)
2022-07-07 06:45:38 [137,482ms] [Fatal] [carb.crashreporter-breakpad.plugin] librtx.hydra.so!std::__detail::_Map_base<unsigned short, std::pair<unsigned short const, std::string>, std::allocator<std::pair<unsigned short const, std::string> >, std::__detail::_Select1st, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true>, true>::operator(unsigned short const&)
2022-07-07 06:45:38 [137,485ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.usd.so!std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<omni::usd::UsdContext::Impl::closeAndOpenStage(std::string const&, bool, bool)::{lambda()#1}> >, void> >::_M_invoke(std::_Any_data const&)
2022-07-07 06:45:38 [137,488ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.usd.so!std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<omni::usd::UsdContext::Impl::closeAndOpenStage(std::string const&, bool, bool)::{lambda()#1}> >, void> >::_M_invoke(std::_Any_data const&)
2022-07-07 06:45:38 [137,491ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.usd.so!omni::usd::UsdManager::drawViewports(float)
2022-07-07 06:45:38 [137,492ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext-viewport_window.plugin.so!_init
2022-07-07 06:45:38 [137,496ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.events.plugin.so!void std::this_thread::sleep_for<long, std::ratio<1l, 1000000l> >(std::chrono::duration<long, std::ratio<1l, 1000000l> > const&)
2022-07-07 06:45:38 [137,498ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!void std::vector<unsigned long, std::allocator >::_M_realloc_insert<unsigned long const&>(__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator > >, unsigned long const&)
2022-07-07 06:45:38 [137,501ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!carbOnPluginRegisterEx
2022-07-07 06:45:38 [137,504ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.renderer.plugin.so!std::__future_base::_State_baseV2::_M_is_deferred_future() const
2022-07-07 06:45:38 [137,508ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.events.plugin.so!void std::this_thread::sleep_for<long, std::ratio<1l, 1000000l> >(std::chrono::duration<long, std::ratio<1l, 1000000l> > const&)
2022-07-07 06:45:38 [137,511ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.loop-default.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tupleomni::kit::RunLoopThread::run():{lambda()#1} > >::~_State_impl()
2022-07-07 06:45:38 [137,514ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!_init
2022-07-07 06:45:38 [137,518ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!carbOnPluginPostShutdown
2022-07-07 06:45:38 [137,518ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
2022-07-07 06:45:38 [137,522ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!__libc_start_main
2022-07-07 06:45:38 [137,522ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
Segmentation fault (core dumped)
-
My Questions
- Is it normal that converting into point cloud consumes both of my GPU memory approximately 48 GB?
- Or is it the technical bug in Kaolin APP?