How to get servicemaker and use riva-build to combine .riva files into a .rmir?

Hardware - GPU RTX 3060Ti
Hardware - CPU 12th Gen Intel(R) Core™ i5-12600K
Operating System - Windows 11 Pro 23H2 64bit + WSL
Riva Version - 13.1

I have downloaded the ngc CLI, and I used the quickstart script to use riva. I did not find any comprehensive guide about servicemaker; the documentation is too ambigious.

I’m extremely new to Riva, and I have a custom model I want to use with it. I have 2 files, the fastpitch and hifigan under riva_quickstart_arm64_v2.13.1. I’m not sure where to put them, I know somewhere in riva_quickstart_arm64_v2.13.1/model_repository. I guess edit the config.sh file to look for the models from there.

Sorry in advance if there is another post similar to my issue, I may have passed by it.
And thanks in advance for helping!

follow the info here for examples on the riva-build commands Custom Models — NVIDIA Riva

once you have a .rmir file, it should be placed at $riva_model_loc/rmir
You can see where riva_model_loc is set to in the config.sh file, or change it if needed. you can also see more detailed instructions in the config.sh file (excerpt bellow)

# Locations to use for storing models artifacts
#
# If an absolute path is specified, the data will be written to that location
# Otherwise, a Docker volume will be used (default).
#
# riva_init.sh will create a `rmir` and `models` directory in the volume or
# path specified.
#
# RMIR ($riva_model_loc/rmir)
# Riva uses an intermediate representation (RMIR) for models
# that are ready to deploy but not yet fully optimized for deployment. Pretrained
# versions can be obtained from NGC (by specifying NGC models below) and will be
# downloaded to $riva_model_loc/rmir by `riva_init.sh`
#
# Custom models produced by NeMo or TLT and prepared using riva-build
# may also be copied manually to this location $(riva_model_loc/rmir).
#
# Models ($riva_model_loc/models)
# During the riva_init process, the RMIR files in $riva_model_loc/rmir
# are inspected and optimized for deployment. The optimized versions are
# stored in $riva_model_loc/models. The riva server exclusively uses these
# optimized versions.

riva_model_loc="riva-model-repo"

I hope you are able to deploy your models. Personally I am having difficulties during the next step of deployment with riva_start.sh where the model fails to deploy due to Unsupported model IR version: 9, max supported IR version: 8

Got the solution. Pull the servicemaker image

docker pull nvcr.io/nvidia/riva/riva-speech:2.13.1-servicemaker

And then run it using this command

docker run --gpus all -it --rm \ #Running inside the container
    -v ~/RIVA/artifacts:/servicemaker-dev \
    -v ~/RIVA/riva_repo:/data \
    --entrypoint="/bin/bash" \
    nvcr.io/nvidia/riva/riva-speech:2.13.1-servicemaker

riva-build speech_synthesis \ #Merging the models
    /servicemaker-dev/voice.rmir:tlt_encode \
    /servicemaker-dev/voice_fastpitch.riva:tlt_encode \
    /servicemaker-dev/voice_hifigan.riva:tlt_encode \
    --voice_name=VoiceNameHere \
    --sample_rate 22050

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.