Audio 2 Face - Extension

I’m developing a extension for A2F that connects to a LLM.
Once i recieve the response from the LLM in text form i want to use NVIDIA Riva to synthesize speech. I already have a non-extension test environment where i have this working. But i would like to have a extension built so that i can enable this directly in Audio 2 Face.

However i am getting this error:

2024-04-02 13:18:56  [Error] [omni.ui.python] NameError: name 'riva' is not defined
2024-04-02 13:18:56  [Error] [omni.ui.python] 
2024-04-02 13:18:56  [Error] [omni.ui.python] At:
2024-04-02 13:18:56  [Error] [omni.ui.python]   d:/nvidia/myextensions/curiousxr-exts-avatar/exts/curiousxr.avatar.extension/curiousxr/avatar/extension/ ui_wavefile
2024-04-02 13:18:56  [Error] [omni.ui.python] 

  • I have tried to include riva or riva.client in the toml dependencies but it always fail to install. My thought here is that i’m not using the correct wording.
2024-04-02 13:43:21  [Warning] [omni.kit.pipapi.pipapi] 'riva.client' failed to install.
  • Tried: nvidia-riva-client based on this reference. It also fails even if i replace the - with .
2024-04-02 13:57:28  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.
  • Whenever i add riva.client it breaks the ui. I’m assuming this is because it fails to install at this point

Any help would be greatly appriciated. I have everything working in A2F except this.

Hi @W1r3d Did you add riva to the extension.toml? How did you add riva?

One (not super clean) way of doing it is to run pip install inside the extension code.

import omni.kit.pipapi
    import riva.client
except ImportError:
    import riva.client
1 Like

This video shows how to do this in the extension.toml file:
PIP API — Omniverse Extensions latest documentation (

1 Like

Hello Ehsan.HM

Thank you for your quick reply! :)

I am using my extension in both Omnvierse Code(2023.1.1) and A2F(2023.2.0).
Just before you wrote i tried this in the main file which did not work.

import riva.client

In the .toml i tried to add riva in this section. Both as nvidia-riva-client, riva.client, nvidia.riva.client

requirements = ["names", "random", "requests", "", "subprocess", "json", "os", "numpy", "IPython", "nvidia.riva.client"]
use_online_index = true

implemented your code in the file and when i run it i get these errors in the console

2024-04-02 19:56:21  [Warning] [omni.kit.helper.file_utils.extension] Failed to deserialize file event uri: bad query field: ''
2024-04-02 19:56:21  [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2024-04-02 19:56:21  [Warning] [omni.ext._impl._internal] There is already a module 's3transfer' in 'd:\nvidia\librarypath\code-2023.1.1\extscache\omni.pip.compute-1.1.0+wx64\pip_prebundle\s3transfer\'. Can't import another module with the same name in public space from 'd:\nvidia\librarypath\code-2023.1.1\extscache\omni.replicator.core-1.9.8+105.0.wx64.r.cp310\pip_prebundle'.
2024-04-02 19:56:21  [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
2024-04-02 19:57:06  [Warning] [omni.kit.pipapi.pipapi] 'nvidia.riva.client' failed to install.
2024-04-02 19:57:49  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.
2024-04-02 19:57:50  [Warning] [omni.kit.viewport.menubar.core.menu_item.viewport_menu_item] ViewportMenuItem order argument is deprecated, use order_setting_path
2024-04-02 19:57:54  [Warning] [carb.tasking.plugin] carb.tasking:pinToCurrentThread(): Task 326417514660 pinned to current thread. Thread pinning is inefficient and should be avoided!
2024-04-02 19:57:54  [Warning] [omni.fabric.plugin] getType called on non-existent attribute /Render/RenderProduct_omni_kit_widget_viewport_ViewportTexture_0 camera
2024-04-02 19:57:54  [Warning] [omni.fabric.plugin] 
2024-04-02 20:01:13  [Warning] [omni.kit.pipapi.pipapi] 'nvidia.riva.client' failed to install.
2024-04-02 20:01:59  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.

I get this when i try to run the particular section that needs riva.client

2024-04-02 20:02:35  [Error] [omni.ui.python] NameError: name 'riva' is not defined
2024-04-02 20:02:35  [Error] [omni.ui.python] 
2024-04-02 20:02:35  [Error] [omni.ui.python] At:
2024-04-02 20:02:35  [Error] [omni.ui.python]   d:/nvidia/myextensions/curiousxr-exts-avatar/exts/curiousxr.avatar.extension/curiousxr/avatar/extension/ ui_wavefile
2024-04-02 20:02:35  [Error] [omni.ui.python] 

That tutorial is great. I’ve been following that way for most of my plugins for my extension.
I’ve managed to try out names, random, and verify that other plugins of my script are working. Like for instance grpc.

1 Like

Maybe i can try to uninstall Isaac Sim / Replicator?
Based on this:

2024-04-02 19:56:21  [Warning] [omni.ext._impl._internal] There is already a module 's3transfer' in 'd:\nvidia\librarypath\code-2023.1.1\extscache\omni.pip.compute-1.1.0+wx64\pip_prebundle\s3transfer\'. Can't import another module with the same name in public space from 'd:\nvidia\librarypath\code-2023.1.1\extscache\omni.replicator.core-1.9.8+105.0.wx64.r.cp310\pip_prebundle'

Testing to deactivate the replicator plugin and installing from the Code Script editor
code in the script editor

import omni.kit.pipapi
use_online_index = True

Error msg

2024-04-03 16:16:18  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.

Script editor

I’m wondering if we can use the example from Code somehow

# omni.kit.pipapi extension is required
import omni.kit.pipapi

# It wraps `pip install` calls and reroutes package installation into user specified environment folder.
# That folder is added to sys.path.
# Note: This call is blocking and slow. It is meant to be used for debugging, development. For final product packages
# should be installed at build-time and packaged inside extensions.
    module="semver", # sometimes module is different from package name, module is used for import check

# use
import semver
ver = semver.VersionInfo.parse('1.2.3-pre.2+build.4')

I added this to the extensions.toml file:

requirements = ["nvidia-riva-client"]
use_online_index = true

And this in the file:

import riva.client

I did the above while the app was closed and it seems to work. Is there anything else you’re trying to install that doesn’t work?

I’ve tried this for two separate extensions now. Using Code 2023.1.1. Both fail to install riva.

  • I am using both VS Code and Omniverse + Omniverse Code as Administrator

Here is the whole log from the console when starting Code 2023.1.1. Here i have just added the “nvidia-riva-client” to the .toml requirements. I have not added the import part.

2024-04-03 17:18:17  [Warning] [omni.kit.helper.file_utils.extension] Failed to deserialize file event uri: bad query field: ''
2024-04-03 17:18:17  [Warning] [omni.stageupdate.plugin] Deprecated: direct use of IStageUpdate callbacks is deprecated. Use IStageUpdate::getStageUpdate instead.
2024-04-03 17:18:17  [Warning] [omni.replicator.core.scripts.annotators] Annotator PostProcessDispatch is already registered, overwriting annotator template
2024-04-03 17:18:55  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.
2024-04-03 17:19:36  [Warning] [omni.kit.pipapi.pipapi] 'nvidia-riva-client' failed to install.
2024-04-03 17:19:37  [Warning] [omni.kit.viewport.menubar.core.menu_item.viewport_menu_item] ViewportMenuItem order argument is deprecated, use order_setting_path
2024-04-03 17:19:40  [Warning] [carb.tasking.plugin] carb.tasking:pinToCurrentThread(): Task 764504178704 pinned to current thread. Thread pinning is inefficient and should be avoided!
2024-04-03 17:19:40  [Warning] [omni.fabric.plugin] getType called on non-existent attribute /Render/RenderProduct_omni_kit_widget_viewport_ViewportTexture_0 camera
2024-04-03 17:19:40  [Warning] [omni.fabric.plugin] 

Can’t think of anything else that i have trouble installing.
Do you have an example of something i could try that is similar to nvidia-riva-client?

I could be wrong, but it looks like the extension.toml is not formatted correctly. Is it possible to share it?

Here’s mine. (had to add .txt to upload)

extension.toml.txt (1.7 KB)

Of course.
I hope it’s ok if i omitt some things from the .toml that is related to the extension naming.

Added mine
extension.toml.txt (1.8 KB)

# Semantic Versioning is used:
version = "1.0.0"

# Lists people or organizations that are considered the "authors" of the package.
authors = ["NVIDIA"]

# The title and description fields are primarily for displaying extension info in UI
description="A simple python extension example to use as a starting point for your extensions."

# Path (relative to the root) or content of readme markdown file for UI.
readme  = "docs/"

# URL of the extension source repository.
repository = ""

# One of categories for UI.
category = "Example"

# Keywords for the extension
keywords = ["kit", "example"]

# Location of change log file in target (final) folder of extension, relative to the root.
# More info on writing changelog:

# Preview image and icon. Folder named "data" automatically goes in git lfs (see .gitattributes file).
# Preview image is shown in "Overview" of Extensions window. Screenshot of an extension might be a good preview image.
preview_image = "data/preview.png"

# Icon is shown in Extensions window, it is recommended to be square, of size 256x256.
icon = "data/icon.png"

# Use omni.ui to build simple UI
"omni.kit.uiapp" = {}

# Main python module this extension provides, it will be publicly available as "import MY EXTENSION NAME".

requirements = ["names", "random", "requests", "", "subprocess", "json", "os", "numpy", "IPython"]
use_online_index = true

# Extra dependencies only to be used during test run
dependencies = [
    "omni.kit.ui_test" # UI testing extension

I think the main problem lies on this line:


The name cannot have white spaces. It must match the folder structure of your extension. e.g. ""

ah no it actually has i just swapped out the name.


name = "my.awesome.extension"

Currently re-installing code 2023.1.1 to see if that makes a difference.

Ah, ok, does it also match the folder structure of the extension?

1 Like

Yes it matches. I can confirm in my explorer view

Re-install complete of Omniverse Code 2023.1.1
I only have nvidia-riva-client in the .toml at this point.
I only have my extension active.
Re-activation of extension gives this error:

2024-04-03 18:20:02  [Error] [] [py stderr]: d:\nvidia\librarypath\code-2022.3.3\extscache\omni.isaac.core_archive-2.0.1+104.1.wx64\pip_prebundle\requests\ RequestsDependencyWarning: urllib3 (1.26.12) or chardet (None)/charset_normalizer (3.0.1) doesn't match a supported version!
2024-04-03 18:20:02  [Error] []   RequestsDependencyWarning)
2024-04-03 18:20:02  [Error] [] 
2024-04-03 18:22:40  [Error] [omni.kit.pipapi.pipapi] Failed to import python module nvidia-riva-client. Error: No module named 'nvidia-riva-client'
2024-04-03 18:22:40  [Warning] [omni.kit.pipapi.pipapi] [omni.kit.pipapi] 'nvidia-riva-client' failed to install.

Are you able to share your extension (could be in private message) so I take a deeper look?

1 Like

Yes, plz check DM

It seems that some files that i had in my extension folder were causing havoc. Cleaning my folder up i am now able to get it to work in Omniverse Code.

Thank you Ehsan.HM for you assistance and patience. It has been invaluable!

1 Like

Superb, if you get more errors regarding protobuf, it’s because the protobuf version that Audio2Face uses is older. The workaround is to load your extension in a newer app e.g. USD Composer, then copy the generated protobuf folder into audio2face folder. In my case it’s like this:


and now it works

1 Like

I have yet to try this. I got sidetracked by debugging other stuff. Will test this today.
Thank you for your insights and quick reply. :)

I have tried this now without any success. I’m thinking i’m not installing protobuf correctly.

I have this in my A2F folder: ov\nucleus\Kit\Audio2Face\2023.2\pip3-envs\default



this is how it looks inside the protobuf folders(some folders do not have the.txt files):

this is how the inside of my system protobuf 4.25.3 folder looks like

i just want to emphasize that i am able to run riva services on my system and through docker without any complications(protobuf or grpc errors). I also have a working python package that is working with this exact code.

Another question that i have is why is it working in Code and not any other application?


I don’t know what i did differently. But i re-installed A2F and restarted my computer a couple of times.

Copied the protobuf folder from Code to A2F when both apps were shut down.

Started A2F up again and now it works except for Audio 2 Face…

I think the question here is rather how do i get my extension to downgrade Riva to Protobuf 3.17.3 which is the same as A2F. → Tried this. Didn’t work.

I did do this. Check my other thread for what happednd: Audio 2 Face Extension - PiP install - #9 by W1r3d

1 Like