Use multiple threads in Omniverse

[OlgKN_El]

Hi,

I’m using Omniverse to create a complicated scene. Currently, I partition the whole scene into different areas, and they are not relevant to each other.

Unfortunately, it’s quite slow to generate the areas one by one. And ideally, No data race will occur. So, I’m wondering if is it possible to use multiple threads in Omniverse so that I can boost the performance/speed of generation?

I check the topic Multi-threading in Omniverse extensions - Omniverse / Developer - NVIDIA Developer Forums
and the Pixar document Universal Scene Description: Threading Model and Performance Considerations (openusd.org).

If I understand correctly, I cannot write to the same stage from multiple threads. And the feasible way : “it is safe for different threads to write simultaneously to different stages”.

With this information, I tried to write a small ugly demo.

This is the code snippet without threads:

import omni
from pxr import Usd, UsdGeom, Sdf
import random
from datetime import datetime

now = datetime.now()
dt_str = now.strftime("_%Y_%m_%d__%H_%M_%S")

global_stage = omni.usd.get_context().get_stage()


def create_sphere(stage):
    geom_xform = UsdGeom.Xform.Define(stage, '/Geom')
    geom_prim = stage.GetPrimAtPath('/Geom')
    sphere = UsdGeom.Sphere.Define(stage, '/Geom/sphere')
    sphere_prim = stage.GetPrimAtPath('/Geom/sphere')

    stage.SetDefaultPrim(geom_prim)
    UsdGeom.XformCommonAPI(sphere_prim).SetTranslate((
        random.randint(0, 10),
        random.randint(0, 10),
        random.randint(0, 10)
    ))
    stage.GetRootLayer().Save()
    return stage


stage_num = 10


def create_stage_with_sphere(index):
    stage_path = f"./cache/Substage_{dt_str}_{index}.usd"
    sub_stage = Usd.Stage.CreateNew(stage_path)

    create_sphere(sub_stage)


##########################################
# parallelize this
for i in range(stage_num):
    create_stage_with_sphere(i)
#########################################

for i in range(stage_num):
    prim_path = Sdf.Path(
        "/World").AppendChild("Substage_{}".format(i))
    sub_prim = global_stage.DefinePrim(prim_path, "Xform")

    sub_prim.GetReferences().AddReference(
        "./cache/Substage_{}_{}.usd".format(dt_str, i))

It works good.

Then, I use multi-threading to have a try:

import omni
from pxr import Usd, UsdGeom, Sdf
import random
from datetime import datetime
import threading

now = datetime.now()
dt_str = now.strftime("_%Y_%m_%d__%H_%M_%S")

global_stage = omni.usd.get_context().get_stage()


def create_sphere(stage):
    geom_xform = UsdGeom.Xform.Define(stage, '/Geom')
    geom_prim = stage.GetPrimAtPath('/Geom')
    sphere = UsdGeom.Sphere.Define(stage, '/Geom/sphere')
    sphere_prim = stage.GetPrimAtPath('/Geom/sphere')

    stage.SetDefaultPrim(geom_prim)
    UsdGeom.XformCommonAPI(sphere_prim).SetTranslate((
        random.randint(0, 10),
        random.randint(0, 10),
        random.randint(0, 10)
    ))
    stage.GetRootLayer().Save()
    return stage


stage_num = 10


def create_stage_with_sphere(index):
    stage_path = f"./cache/Substage_{dt_str}_{index}.usd"
    sub_stage = Usd.Stage.CreateNew(stage_path)

    create_sphere(sub_stage)


threads = []
for i in range(stage_num):
    t = threading.Thread(target=create_stage_with_sphere, args=(i,))
    t.start()
    threads.append(t)

for t in threads:
    t.join()


for i in stage_num:
    prim_path = Sdf.Path(
        "/World").AppendChild("Substage_{}".format(i))
    sub_prim = global_stage.DefinePrim(prim_path, "Xform")

    sub_prim.GetReferences().AddReference(
        "./cache/Substage_{}_{}.usd".format(dt_str, i))

When I execute the version with multi threads, Omniverse would be stuck, and I cannot get any responses like log info or error messages.

If multi-threading is doable in Omniverse, could you help me to solve this problem?

Thank you so much!

[anthony.yaghi]

To my understanding, Omniverse uses async programming (coroutines) to keep the UI interactive, I suggest you take a look at the code I posted in my question here why does multiprocessing only works with fork it could work for your use case.
Another thing to keep in mind is, in Python threads don’t really execute in parallel, rather there is only one thread running at a time but Python switch between threads giving the illusion of running things in parallel. This means multi-threading doesn’t always offer any increase in performance, you probably want to look at multi-processing instead (like the example code above).

[mati-nvidia]

Hi @OlgKN_El. I would avoid threads like Anthony mentioned because of the Python GIL limitation.

The slowdown you’re probably hitting has to do with change notifications that USD fires. You have a couple of options in Omniverse:

1. Some code can be accelerated using USDRT/Fabric: USDRT Project Overview — usdrt 7.0.5 documentation. It’s fairly new, but it’s our long-term solution.
2. Using Sdf.ChangeBlock. While the docs say, “DANGER DANGER DANGER” we have lots of examples in Omniverse where it’s used responsibly. Here’s an example: Using Sdf.ChangeBlock to batch changes by mati-nvidia · Pull Request #1 · True-VFX/kit-ext-cube_array · GitHub. For the best results, you’d want to use the Sdf API directly instead of the USD API, so this solution takes a considerable amount of work.

[OlgKN_El]

Thank you so much! I will have a try!

[OlgKN_El]

Thank you very much!