Python Environment with `omni` package

[syntheticDataArchitect]

Hello,

How do I access the python environment in an omniverse container? In particular, I need to run replicator code in nvcr.io/nvidia/omniverse-replicator:1.5.3-r2. I cannot import the omni.replicator module and have access to the carbonite libcarb.so.

As described in the doc, scripts can be run via /startup.sh, which uses /opt/nvidia/omniverse/code-launcher/kit/kit, but for my project, I need to use the python runtime to import omni.replicator. I expect to be able to set up the environment by running the commands in /opt/nvidia/omniverse/kit-sdk-launcher/python.sh

To get started set MY_DIR, as noted in kit-sdk-launcher/python.sh, “The executable path will be where python is located. So we set Carbonite app path through env var:”

MY_DIR=/opt/nvidia/omniverse/kit-sdk-launcher
export CARB_APP_PATH=${MY_DIR}

Next, the PYTHONPATH is set. I have to use the alternative here as explained below:

 ### source ${MY_DIR}/setup_python_env.sh       ### NO REPLICATOR
source /opt/nvidia/omniverse/code-launcher/setup_python_env.sh

There are several differences between these twp setup scripts. Importantly, there is no replicator dependency in kit-sdk-launcher/setup_python_env.sh. So I am using the one from the code-launcher.

Now attempting to import omni.replicator.core from the kit based python, missing carbonite:

${MY_DIR}/python/bin/python3.7
Python 3.7.13 (default, Mar 28 2022, 19:19:54) 
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import omni
>>> omni.__dir__()
['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__file__', '__path__']
>>> import omni.replicator.core
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/nvidia/omniverse/code-launcher/extscache/omni.replicator.core-1.5.3+lx64.r.cp37/omni/replicator/core/__init__.py", line 1, in <module>
    from .scripts import create, distribution, example, get, modify, orchestrator, physics, randomizer, settings, trigger
  File "/opt/nvidia/omniverse/code-launcher/extscache/omni.replicator.core-1.5.3+lx64.r.cp37/omni/replicator/core/scripts/create.py", line 15, in <module>
    import carb.settings
  File "/opt/nvidia/omniverse/code-launcher/kit/kernel/py/carb/__init__.py", line 29, in <module>
    from ._carb import *
ImportError: libcarb.so: cannot open shared object file: No such file or directory
>>> 

Incidentally, there is another python environment in MY_DIR=/opt/nvidia/omniverse/code-launcher/kit, but the setup_python_env.sh is identical to that of kit-sdk-launcher (so no replicator module).

Do I need to run packman? (kit-sdk-launcher/dev/tools/packman/python.sh OR code-launcher/dev/packman/python.sh)

[syntheticDataArchitect]

Working on a python runtime of the code-launcher. Why do I get this missing interface error when importing omni.replicator.core

RuntimeError: Failed to acquire interface: omni::kit::IApp (pluginName: nullptr)

As indicated above, replicator is added to the python path of code-launcher (not kit-sdk-launcher), while carbonite is associated with kit-sdk-launcher. I can get Python to find carbonite within the code-launcher environment by setting up as follows within the omniverse-replicator container…

#Location of libcarb.so
export CARB_APP_PATH=/opt/nvidia/omniverse/kit-sdk-launcher

MY_DIR=/opt/nvidia/omniverse/code-launcher/kit

#Set PYTHONPATH
SCRIPT_DIR=/opt/nvidia/omniverse/code-launcher
export PYTHONPATH=$PYTHONPATH:$SCRIPT_DIR/extscache/omni.replicator.core-1.5.3+lx64.r.cp37:$SCRIPT_DIR/extscache/omni.warp-0.5.1+cp37

#Install missing dependencies 
sudo chmod a+w -R /opt/nvidia/omniverse/code-launcher/kit/python/lib/python3.7/site-packages
${MY_DIR}/python/bin/python3.7 -m pip install boto3

#Run Python
${MY_DIR}/python.sh

Now, when I attempt to import replicator I get the missing app interface RuntimeError shown in the screenshot.

[mati-nvidia]

Hi @syntheticDataArchitect. Is there a reason why you want to run Replicator from another Python environment instead of just shelling out running Code with a script if needed?

e.g.
omni.code.bat --exec "my_script.py"

[syntheticDataArchitect]

Yes, I am working on an app that wraps replicator so our users can select settings and the app will configure the staged graph for runs. I can do this by having our nodes create lines of python and write them all to a file at the end of the logic, but I am looking for a maintainable integration with omniverse.

[mati-nvidia]

Ok. This approach is still labeled as experimental in our docs, but I’ll check with the dev team for guidance.

[mati-nvidia]

One other idea that I’ll offer up if is if you’ve considered creating a Kit extension and/or microservice to have your app communicate with a running instance of Code?

[syntheticDataArchitect]

Thank you @mati-nvidia. Looking forward to getting python running in the container - this would make a valuable improvement to my project. For a simple example, a camera node, rather than generating text to add the camera to the layer, it would actually call omni.replicator.core.create.camera(…). More importantly will be for object generators and randomization modifiers.

[nworkerxz9q8]

Just double checking, but are you importing import omni.replicator.core and not just omni.replicator?

[syntheticDataArchitect]

Yes, this test and goal is to import omni.replicator.core.

[nworkerxz9q8]

To run anything from omniverse in the container, you need to launch with startup.sh. This will launch a headless version of replicator and you pass it a python file, like here: Omniverse Replicator | NVIDIA NGC

Startup.sh will create an omniverse python environment with omni libraries. This won’t work from the python environment alone.

[syntheticDataArchitect]

Okay, I’ll use the omniverse-replicator container with static files. In parallel, I’ll work on a design for creating a custom kit app. I don’t know much about carbonite yet, and need a headless app. Can you a good place to start reading? Also it might help if you could share the Dockerfile used to create the omniverse-replicator image.

Thank you!