Isaac sim fails to start if there are no inotify watches available

When starting Isaac sim without enough inotify watches available the launch will fail hard.

Increasing the number of available watches will make it start, but it is not very clear from the error log.

To increase the number of inotify watches, edit /etc/systctl.conf and add (or replace)

fs.inotify.max_user_watches=16777216
fs.inotify.max_user_instances=1024

I run with some large code bases.

../../../source/framework/FileSystemWatcher.cpp(394): carb::filesystem::FileSystemWatcherLinux::FileSystemWatcherLinux()(): Assertion (m_inotifyFd != -1) failed: inotify_init() failed: 24

2022-04-22 09:19:08 [79ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.crashreporter-breakpad.plugin.so!_init
2022-04-22 09:19:08 [79ms] [Fatal] [carb.crashreporter-breakpad.plugin] libstdc++.so.6!std::rethrow_exception(std::__exception_ptr::exception_ptr)
2022-04-22 09:19:08 [80ms] [Fatal] [carb.crashreporter-breakpad.plugin] libstdc++.so.6!std::terminate()
2022-04-22 09:19:08 [80ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:08 [80ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:08 [80ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:08 [81ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::pair<std::__detail::_Node_iterator<std::string, true, true>, bool> std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::_M_insert<std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > >(std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > const&, std::integral_constant<bool, true>)
2022-04-22 09:19:08 [81ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::pair<std::__detail::_Node_iterator<std::string, true, true>, bool> std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::_M_insert<std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > >(std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > const&, std::integral_constant<bool, true>)
2022-04-22 09:19:08 [81ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2022-04-22 09:19:08 [81ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2022-04-22 09:19:08 [81ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)
2022-04-22 09:19:08 [82ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!_init
2022-04-22 09:19:08 [82ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
2022-04-22 09:19:08 [82ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!__libc_start_main
2022-04-22 09:19:08 [82ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
terminate called without an active exception
2022-04-22 09:19:09 [83ms] [Error] [carb.crashreporter-breakpad.plugin] failed to write .dmp to: '/home/jesper/.local/share/ov/data/Kit/Isaac-Sim/2021.2/4d5bbf91-9e13-4a8e-65913db8-e3ffa13a.dmp'
2022-04-22 09:19:09 [83ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!funlockfile
2022-04-22 09:19:09 [84ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!gsignal
2022-04-22 09:19:09 [84ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!abort
2022-04-22 09:19:09 [84ms] [Fatal] [carb.crashreporter-breakpad.plugin] libstdc++.so.6!__cxa_throw_bad_array_new_length
2022-04-22 09:19:09 [84ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.crashreporter-breakpad.plugin.so!_init
2022-04-22 09:19:09 [85ms] [Fatal] [carb.crashreporter-breakpad.plugin] libstdc++.so.6!std::rethrow_exception(std::__exception_ptr::exception_ptr)
2022-04-22 09:19:09 [85ms] [Fatal] [carb.crashreporter-breakpad.plugin] libstdc++.so.6!std::terminate()
2022-04-22 09:19:09 [85ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:09 [85ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:09 [86ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.so!_init
2022-04-22 09:19:09 [86ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::pair<std::__detail::_Node_iterator<std::string, true, true>, bool> std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::_M_insert<std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > >(std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > const&, std::integral_constant<bool, true>)
2022-04-22 09:19:09 [86ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::pair<std::__detail::_Node_iterator<std::string, true, true>, bool> std::_Hashtable<std::string, std::string, std::allocator<std::string>, std::__detail::_Identity, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, true, true> >::_M_insert<std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > >(std::string const&, std::__detail::_AllocNode<std::allocator<std::__detail::_Hash_node<std::string, true> > > const&, std::integral_constant<bool, true>)
2022-04-22 09:19:09 [86ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2022-04-22 09:19:09 [86ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.ext.plugin.so!std::_Rb_tree<std::string, std::string, std::_Identity<std::string>, std::less<std::string>, std::allocator<std::string> >::_M_erase(std::_Rb_tree_node<std::string>*)
2022-04-22 09:19:09 [87ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!std::string __gnu_cxx::__to_xstring<std::string, char>(int (*)(char*, unsigned long, char const*, __va_list_tag*), unsigned long, char const*, ...)
2022-04-22 09:19:09 [87ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.kit.app.plugin.so!_init
2022-04-22 09:19:09 [87ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
2022-04-22 09:19:09 [87ms] [Fatal] [carb.crashreporter-breakpad.plugin] libc.so.6!__libc_start_main
2022-04-22 09:19:09 [87ms] [Fatal] [carb.crashreporter-breakpad.plugin] kit!_init
2022-04-22 09:19:09 [6,415ms] [Warning] [gpu.foundation.plugin] Texture:'/tmp/carb.2Z2HjF/copy.39x39.png' is not released correctly and cause leak.
2022-04-22 09:19:09 [6,416ms] [Warning] [gpu.foundation.plugin] Texture:'/home/jesper/.local/share/ov/pkg/isaac_sim-2021.2.1/exts/omni.isaac.app.launcher/icons/omni.isaac.sim.headless.websocket.png' is not released correctly and cause leak.
2022-04-22 09:19:09 [6,416ms] [Warning] [gpu.foundation.plugin] Texture:'/home/jesper/.local/share/ov/pkg/isaac_sim-2021.2.1/exts/omni.isaac.app.launcher/icons/omni.isaac.sim.headless.kitremote.png' is not released correctly and cause leak.
2022-04-22 09:19:09 [6,416ms] [Warning] [gpu.foundation.plugin] Texture:'/home/jesper/.local/share/ov/pkg/isaac_sim-2021.2.1/exts/omni.isaac.app.launcher/icons/omni.isaac.sim.png' is not released correctly and cause leak.
2022-04-22 09:19:09 [6,416ms] [Warning] [gpu.foundation.plugin] Release above leaking textures on exit.