Waiting for a request that has taken longer than 0ms to complete

Omniverse Platform Nucleus
1

Hi,

I am running Kit headlessly in a container like this:

#!/usr/bin/env bash                                                           
set -e                                                                        
set -u                                                                        
                                                                              
LD_LIBRARY_PATH=/opt/nvidia/omniverse/kit-sdk-launcher/plugins/carb_gfx       
exec "/opt/nvidia/omniverse/kit-sdk-launcher/kit" \                           
    "--exec" "hello_world.py" \                                               
    "--enable" "dbs.ipm.geojson" \                                            
    "--enable" "dbs.ipm.trackgenerator" \                                     
    "--no-window" \                                                           
    "--allow-root"

As you can see, I am using two custom extensions. These are located on an external Nucleus instance. While the code seems to work, I am getting a lot of warnings:

$ ./startup_ipm.sh
[Info] [carb] Logging to file: /home/ipm/.nvidia-omniverse/logs/Kit/kit/103.5/kit_20230606_143141.log
[0.281s] [ext: omni.usd.config-1.0.0] startup
[0.285s] [ext: omni.usd.libs-1.0.0] startup
[0.387s] [ext: omni.kit.async_engine-0.0.0] startup
[0.389s] [ext: omni.assets.plugins-0.0.0] startup
[0.390s] [ext: omni.client-0.1.0] startup
[0.401s] [ext: omni.kit.registry.nucleus-0.0.0] startup
2023-06-06 14:31:41 [506ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 457, in sync_index
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 893, in _ensure_synced
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 219, in __init__
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 283, in _run
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 81, in _client_exists
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 92, in stat
2023-06-06 14:31:41 [590ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 457, in sync_index
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 893, in _ensure_synced
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 219, in __init__
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 284, in _run
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 98, in _client_copy
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 188, in copy
2023-06-06 14:31:41 [606ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 457, in sync_index
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 893, in _ensure_synced
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 219, in __init__
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 283, in _run
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 81, in _client_exists
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 92, in stat
2023-06-06 14:31:41 [608ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 457, in sync_index
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 893, in _ensure_synced
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nu[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: downloading: 'omniverse://nucleus.dsdlab.db.de/ipm-extensions-stage-dev/3/archives/dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37.zip' -> '/tmp/tmp3ss2i8tb/dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37.zip'...
[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: unpacking: /tmp/tmp3ss2i8tb/dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37.zip -> /home/ipm/.cache/ov/exts/3
Unzipping: dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37.zip (80.3 KiB)
No continuous progress report because this is not a proper terminal. Be patient ...
100% (speed 31.8 MiB/s)
Total of 0.00 seconds
[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: creating a link: /home/ipm/.local/share/ov/data/Kit/kit/103.5/exts/3/dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37 -> /home/ipm/.cache/ov/exts/3/dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a+cp37
[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: downloading: 'omniverse://nucleus.dsdlab.db.de/ipm-extensions-stage-dev/3/archives/dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37.zip' -> '/tmp/tmpf8juju2u/dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37.zip'...
[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: unpacking: /tmp/tmpf8juju2u/dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37.zip -> /home/ipm/.cache/ov/exts/3
Unzipping: dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37.zip (4.71 MiB)
No continuous progress report because this is not a proper terminal. Be patient ...
100% (speed 170 MiB/s)
Total of 0.03 seconds
[omni.kit.registry.nucleus]: [omni.kit.registry.nucleus]: creating a link: /home/ipm/.local/share/ov/data/Kit/kit/103.5/exts/3/dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37 -> /home/ipm/.cache/ov/exts/3/dbs.ipm.geojson-0.1.30-20230606133840-64d2558+cp37
cleus/omni/kit/registry/nucleus/registry.py", line 219, in __init__
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 284, in _run
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 98, in _client_copy
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 188, in copy
[0.621s] Pulling extension: `dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a` from the registry.
2023-06-06 14:31:41 [614ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 778, in pull_extension
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 810, in _pull_extension_async
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 708, in _find_archive
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 81, in _client_exists
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 92, in stat
2023-06-06 14:31:41 [618ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 778, in pull_extension
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 820, in _pull_extension_async
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 98, in _client_copy
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 188, in copy
[0.647s] Extension: `dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a` was installed.
[0.647s] Pulling extension: `dbs.ipm.geojson-0.1.30-20230606133840-64d2558` from the registry.
2023-06-06 14:31:41 [641ms] [Warning] [omni.client.python] Waiting for a request that has taken longer than 0ms to complete: 
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 778, in pull_extension
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 820, in _pull_extension_async
  File "/opt/nvidia/omniverse/kit-sdk-launcher/python/lib/python3.7/contextlib.py", line 74, in inner
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.kit.registry.nucleus/omni/kit/registry/nucleus/registry.py", line 98, in _client_copy
  File "/opt/nvidia/omniverse/kit-sdk-launcher/extscore/omni.client/omni/client/__init__.py", line 188, in copy
[0.709s] Extension: `dbs.ipm.geojson-0.1.30-20230606133840-64d2558` was installed.
[0.711s] [ext: dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a] startup
[0.713s] [ext: dbs.ipm.geojson-0.1.30-20230606133840-64d2558] startup
[0.801s] [ext: omni.kit.registry.nucleus-0.0.0] shutdown
[0.804s] [ext: omni.client-0.1.0] shutdown
[0.811s] [ext: omni.assets.plugins-0.0.0] shutdown
[0.811s] [ext: omni.kit.async_engine-0.0.0] shutdown
[0.813s] [ext: omni.usd.libs-1.0.0] shutdown
[0.816s] [ext: omni.usd.config-1.0.0] shutdown
[0.817s] [ext: dbs.ipm.geojson-0.1.30-20230606133840-64d2558] shutdown
[0.819s] [ext: dbs.ipm.trackgenerator-0.0.65-20230606140343-119c65a] shutdown

Any advice appreciated
Bruno

2

Hi Bruno,
I believe these to be related to connecting to the Nucleus Registry to pull and install the extensions, but it doesn’t seem to be taking too long or have any practical implications correct? It is basically trying to install the extension locally, similar way as it would if you would use the Extension Manager UI.

A couple of questions:

  • Do similar warnings happen when you are not running this in a container? Just launching Kit from a terminal?
  • Same, if you run with the GUI, do same warnings appear when searching and installing the extensions from Nucleus in the Extension Manager UI?

Once you have a more stable extension development, and if you are enabling all the time these extensions you can also add those by default to your container. But agreed, that having them in a Nucleus server registry is still helpful to install, push and pull updates though, and it should work

3

Thanks Teresa,

it seems to happen only when running in a container, not if I run from a terminal or when running the UI.

Best,
Bruno