Issues running Farm Queue

Omniverse General Discussion
1

I’m attempting to run the farm extensions for the first time. Watching the tutorial videos it appears that some additional services should get installed, but on my system they do not. For example, omni.services.farm.management.ui and omni.services.farm.management.tasks do not seem to install.

Here’s the text when I run farm queue:

[Info] [carb] Logging to file: C:/Users/VRLab/.nvidia-omniverse/logs/Kit/farm.queue-0.2.0/100.3/kit_20210331_104236.log
[Info] [carb] Initializing plugin: carb.events.plugin (interfaces: [carb::events::IEvents v0.2]) (impl: carb.events.plugin)
[0.149s] Loading user config located at: ‘C:/Users/VRLab/AppData/Local/ov/data/Kit/farm.queue-0.2.0/100.3/user.config.json’
[0.413s] [ext: omni.kit.async_engine-0.0.0] startup
[0.422s] [ext: omni.kit.loop-default-0.1.0] startup
[0.424s] [ext: omni.kit.test-0.0.0] startup
[0.440s] [ext: omni.kit.pipapi-0.0.0] startup
[0.453s] [ext: omni.kit.pip_archive-0.0.0] startup
[0.461s] [ext: omni.services.transport.client.base-1.0.0] startup
[0.468s] [ext: omni.services.client-0.2.0] startup
[0.792s] [ext: omni.services.transport.client.https_async-1.0.0] startup
[1.035s] [ext: omni.assets-0.0.0] startup
[1.040s] [ext: omni.timeline-0.0.0] startup
[1.047s] [ext: omni.kit.commands-0.0.0] startup
[1.070s] [ext: carb.audio-0.1.0] startup
[1.078s] [ext: omni.usd-0.1.8] startup
[1.341s] [ext: omni.client-0.1.0] startup
[1.362s] [ext: omni.services.transport.client.http_async-1.0.0] startup
[1.548s] [ext: omni.kit.registry.nucleus-0.0.0] startup
2021-03-31 17:42:40 [3,582ms] [Warning] [omni.client.plugin] Tick: authentication: Could not resolve host name “kit-extensions.ov.nvidia.com”
2021-03-31 17:42:40 [3,609ms] [Error] [carb.python] [omni.kit.registry.nucleus]: syncing index failed. Result: Result.ERROR_CONNECTION
2021-03-31 17:42:40 [3,610ms] [Error] [omni.ext.plugin] Failed to resolve extension dependencies.
Solver input:
(name: ‘farm.queue’, tag: ‘’, version: ‘0.2.0’, exact: ‘true’, optional: ‘false’)
Failure hints:

[3.715s] app started

I’ve tried on multiple networks, in case it was a firewall blocking access to kit-extensions.ov.nvidia.com.

I can access localhost:8011/docs, but there is no section for UI or Tasks. Also, there is no response from localhost:8011/farm/management/ui/status (I assume because the service didn’t run or install).

Here’s an image that may have valuable information:

FarmQueue
FarmQueue1920×1160 309 KB

Thanks!

2

Hi Keith,

I’ll have a look at what might be going on and will get back to you today.

Thanks,
Jozef

3

Hi Keith,

Having had a look, it looks like the version that is available on the registry has some reference to our internal extension registry rather than the public one.
I am working on re-releasing the correct version that should sort this out.
I will let you know as soon as it is available.

Thanks,
Jozef

4

Thanks, Jozef.

-Keith

5

Hi Keith,

Just a quick update.
Things have been a bit busy with GTC the last few weeks.
I should have an updated, and improved, version live by early next week.
Apologies for the delay.

Thanks,
Jozef

6

Thank you, Jozef. No hurry. I know there’s a lot to get done for GTC.

-Keith

7

Hello Jozef,

it seems that this problem persists in the current docker image.
When I try to open the extensions window, ISAAC Sim crashes.

Here are the logs:

2021-06-16 12:02:02 [232,834ms] [Info] [carb.python] [omni.kit.registry.nucleus]: (syncing index (async)) copying: omniverse://kit-extensions.ov.nvidia.com/exts/kit/default/2/index.toml -> /tmp/tmp91glxx3d/index.toml
2021-06-16 12:02:02 [232,836ms] [Info] [omni.client.plugin]  Tick: provider_nucleus: Received username/password from application for kit-extensions.ov.nvidia.com:3009
2021-06-16 12:02:02 [232,836ms] [Info] [omni.client.plugin]  Tick: provider_nucleus: Connecting to kit-extensions.ov.nvidia.com:3009 as svc-kit-extension-downloader using new authentication method
2021-06-16 12:02:02 [232,837ms] [Info] [omni.client.plugin]  Tick: authentication: Discovery(kit-extensions.ov.nvidia.com:3009): Searching for OmniAuth.idl.ts : Credentials at wss://kit-extensions.ov.nvidia.com/omni/discovery
2021-06-16 12:02:02 [232,838ms] [Info] [omni.client.plugin]  Tick: authentication: Discovery(wss://kit-extensions.ov.nvidia.com/omni/discovery): Create transport failed: Unable to connect to kit-extensions.ov.nvidia.com:443
2021-06-16 12:02:02 [232,838ms] [Warning] [omni.client.plugin]  Tick: authentication: Could not resolve host name "kit-extensions.ov.nvidia.com"
2021-06-16 12:02:02 [232,838ms] [Info] [omni.client.plugin]  Tick: authentication: Discovery(kit-extensions.ov.nvidia.com:3009): Searching for OmniAuth.idl.ts : Credentials at ws://kit-extensions.ov.nvidia.com:3333
2021-06-16 12:02:02 [232,839ms] [Info] [omni.client.plugin]  Tick: authentication: Discovery(ws://kit-extensions.ov.nvidia.com:3333): Create transport failed: Unable to connect to kit-extensions.ov.nvidia.com:3333
2021-06-16 12:02:02 [232,839ms] [Warning] [omni.client.plugin]  Tick: authentication: Could not resolve host name "kit-extensions.ov.nvidia.com"
2021-06-16 12:02:02 [232,839ms] [Info] [omni.client.plugin]  Tick: provider_nucleus: Connecting to kit-extensions.ov.nvidia.com:3009 as svc-kit-extension-downloader using old authentication method
2021-06-16 12:02:02 [232,841ms] [Info] [omni.client.plugin]  Tick: provider_nucleus: Connecting to kit-extensions.ov.nvidia.com:3009 failed: Connection refused
2021-06-16 12:02:02 [233,303ms] [Info] [carb.python] [omni.kit.registry.nucleus]: result: Result.ERROR_CONNECTION
2021-06-16 12:02:02 [233,303ms] [Error] [carb.python] [omni.kit.registry.nucleus]: syncing index failed. Result: Result.ERROR_CONNECTION
2021-06-16 12:02:03 [233,455ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.events.plugin.so!carbOnPluginStartup
2021-06-16 12:02:02 [233,303ms] [Info] [carb.python] [omni.kit.registry.nucleus]: [kit/default] refresh_index() -> failure.
2021-06-16 12:02:03 [233,462ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::Scheduler::*)(unsigned int, int, carb::cpp20::latch*), carb::tasking::Scheduler*, unsigned int, int, carb::cpp20::latch*> > >::_M_run()
2021-06-16 12:02:03 [233,469ms] [Fatal] [carb.crashreporter-breakpad.plugin] libomni.usd.so!std::_Function_base::_Base_manager<omni::usd::OmniUsdMessenger::Impl::Impl(omni::usd::UsdContext*)::{lambda(carb::events::IEvent*)#1}>::_M_manager(std::_Any_data&, std::_Function_base::_Base_manager<omni::usd::OmniUsdMessenger::Impl::Impl(omni::usd::UsdContext*)::{lambda(carb::events::IEvent*)#1}> const&, std::_Manager_operation)
2021-06-16 12:02:03 [233,477ms] [Fatal] [carb.crashreporter-breakpad.plugin] libpthread.so.0!funlockfile
2021-06-16 12:02:03 [233,483ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.imaging.plugin.so!_init
2021-06-16 12:02:03 [233,489ms] [Fatal] [carb.crashreporter-breakpad.plugin] libgpu.foundation.so!void std::this_thread::sleep_for<long, std::ratio<1l, 1000000l> >(std::chrono::duration<long, std::ratio<1l, 1000000l> > const&)
2021-06-16 12:02:03 [233,496ms] [Fatal] [carb.crashreporter-breakpad.plugin] libgpu.foundation.so!void std::this_thread::sleep_for<long, std::ratio<1l, 1000000l> >(std::chrono::duration<long, std::ratio<1l, 1000000l> > const&)
2021-06-16 12:02:03 [233,503ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.assets.plugin.so!carbOnPluginShutdown
2021-06-16 12:02:03 [233,510ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.assets.plugin.so!carbOnPluginShutdown
2021-06-16 12:02:03 [233,516ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.assets.plugin.so!std::__shared_count<(__gnu_cxx::_Lock_policy)2>::operator=(std::__shared_count<(__gnu_cxx::_Lock_policy)2> const&)
2021-06-16 12:02:03 [233,523ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::ThreadPool::*)(), carb::tasking::ThreadPool*> > >::~_State_impl()
2021-06-16 12:02:03 [233,530ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::ThreadPool::*)(), carb::tasking::ThreadPool*> > >::~_State_impl()
2021-06-16 12:02:03 [233,536ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (carb::tasking::ThreadPool::*)(), carb::tasking::ThreadPool*> > >::~_State_impl()
2021-06-16 12:02:03 [233,543ms] [Fatal] [carb.crashreporter-breakpad.plugin] libcarb.tasking.plugin.so!make_fcontext
Segmentation fault (core dumped)
8

the same issue as yours. Have you fixed this problem? I can’t open the extensions window right now…

9

Hi Maximilian and Seyoatda,

Apologies for the delay in getting back to you.
Is this specific to the Farm Queue or in general issues with running headless?

If it is Farm Queue related we have a few things coming shortly that should resolve all of those issues.

Thanks,
Jozef