The documentation for writing extensions seems incomplete

Omniverse General Discussion
,
1

Hello, I’d like to automate a workflow in Audio2Gesture but I’m having a hard time.

Obviously I have read the initial tutorials and videos here and here. I’ve also checked the API documentation here, here, here, etc. but I still don’t know how to actually do anything.

Yes, I can add a new default extension (the one with the counter). I can hook it up to VS code and debug. I can use the snippets in the Script Editor window as a guide for some things. But as for accomplishing my goal of automating an Audio2Gesture workflow there doesn’t seem to be any direction anywhere.

What I tried doing was copying some of the code I found in this Audio2Gesture test file

C:\users\me\appdata\local\ov\pkg\machinima-2022.3.0\extscache\omni.audio2gesture.core-0.2.19+104.1.wx64\omni\audio2gesture\core\tests\test_a2g_core.py

and I had some success (was able to get an offline instance half-way created in the UI) but it would constantly crash and never really work and even if it did I’m not quite sure what I’m doing. This can’t be the right way to do it, right?

Just as an example here is the kind of code I ended up with in my failed attempt at writing an extension:

    async def doa2g(self):
        self._a2g_core = omni.audio2gesture.core.get_ext()
        self._stage = omni.usd.get_context().get_stage()
        
        name = "automatic_a2g"
        self._a2g_core.offline_manager().create_instance(name)
        await asyncio.sleep(0.2)

        DEMO_STAGE_FPATH = os.path.normpath(
            os.path.join(omni.audio2gesture.core_deps.get_deps_root(), "audio2gesture-data/data/precomputed/A2G.usda")
        )
        TEST_TARGET_SKEL_NOTAGS = "C:/Users/me/AppData/Local/ov/pkg/machinima-2022.3.0/extscache/omni.audio2gesture.core-0.2.19+104.1.wx64/omni/audio2gesture/core/tests/tests_data/test_target_skeleton_notags_no_ovcontent.usda"

        self._stage = Usd.Stage.Open(DEMO_STAGE_FPATH)
        Sdf.CopySpec(self._stage.GetRootLayer(), "/A2G_root", self._stage.GetRootLayer(), "/A2G_BaseSkeleton")
        source_skel_path = "/A2G_BaseSkeleton/A2GSkeleton"

        prim = self._stage.OverridePrim("/Test")
        prim.GetReferences().AddReference(TEST_TARGET_SKEL_NOTAGS)
        target_skel_path = "/Test/Character/Rig/Root"

mostly I was just following the test code. But the problem is even if that worked for what I’m doing I have no idea what to do next or if I’m going about things in the wrong way.

My goal is to programmatically run Audio2Gesture. e.g. I give it a model, and audio file, and it exports a .usd with the animation. Even better would be if I could get it to render the output to some kind of video stream (perhaps even in the form of a png for each frame) or use a connector to stream the animation to blender but that is way advanced relative to where I am now which is I can’t even get it to create a offline or streaming pipeline programmatically.