Please provide the following information when requesting support.
Riva Version riva_quickstart:2.8.1
Hi! I have downloaded TTS Es Multispeaker FastPitch HiFiGAN | NVIDIA NGC model but I am not able to make riva and tritonserver work.
This is what I have done for now:
- Create riva with Nemo docker (using the last nemo docker version, 22.09.)
docker run --gpus all -it --rm \
-v $(pwd):/NeMo \
--shm-size=8g \
-p 8888:8888 \
-p 6006:6006 \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
--device=/dev/snd \
nvcr.io/nvidia/nemo:22.09
Then installed nemo2riva python package from riva_quickstart_v2.8.2.
pip3 install "riva_quickstart_v2.8.1/nemo2riva-2.8.1-py3-none-any.whl"
And generate riva files:
nemo2riva --key tlt_encode --out tts_es_hifigan_ft_fastpitch_multispeaker.riva tts_es_hifigan_ft_fastpitch_multispeaker.nemo
Which works well, and
nemo2riva --key tlt_encode --out tts_es_fastpitch_multispeaker.riva tts_es_fastpitch_multispeaker.nemo
Where I started to have problems and updated versions of the next packages.
pip install hydra-core==1.2.0
pip install omegaconf==2.2.3
pip install pytorch-lightning==1.8.6
pip install nemo-toolkit==1.14.0
Updating these packages solve all the problems I had with nem2riva.
- I used
servicemakerdocker forriva-buildandriva-deploy.
docker run --init -it --rm --gpus '"'"device=0"'"' \
-v $(pwd)/riva_model_loc:/data \
-e "MODEL_DEPLOY_KEY=tlt_encode" \
--name riva-service-maker \
nvcr.io/nvidia/riva/riva-speech:2.8.1-servicemaker
riva-build speech_synthesis \
tts_es_hifigan_ft_fastpitch_multispeaker.rmir:tlt_encode \
tts_es_fastpitch_multispeaker.riva:tlt_encode \
tts_es_hifigan_ft_fastpitch_multispeaker.riva:tlt_encode \
--language_code es_US \
--num_speakers=174 \
--phone_set=ipa \
--sample_rate 44100 \
--voice_name Latin-American-Spanish \
--subvoices es-AR-Female:0,es-AR-Male:32,es-CL-Female:44,es-CL-Male:57,es-CO-Female:75,es-CO-Male:91,es-PE-Female:108,es-PE-Male:126,es-VE-Female:151,es-VE-Male:162 \
--wfst_tokenizer_model=tokenize_and_classify.far \
--wfst_verbalizer_model=verbalize.far
I have downloaded the tokenizer and verbalizer from Riva ASR Spanish Inverse Normalization Grammar | NVIDIA NGC.
This works, I have created the rmir file and now I deploy.
riva-deploy -f tts_es_hifigan_ft_fastpitch_multispeaker.rmir:tlt_encode /data/models
Models have been created, being tts_es_fastpitch_multispeaker.riva in ONNX format because it cannot convert it to TensorRT.
- I have launched
rivawith these created models.
docker run --init --rm -it \
--gpus '"'"device=0"'"' \
-p 50051:50051 \
-v $(pwd)/riva_model_loc:/data \
--name riva-speech \
-p 8000:8000 -p 8001:8001 -p 8002:8002 \
nvcr.io/nvidia/riva/riva-speech:2.8.1
start-riva --riva-uri=0.0.0.0:50052 --nlp_service=false --asr_service=true --tts_service=true
But the server stops and doesn’t work. This is the error that I am seeing:
I0112 11:02:09.698625 12536 server.cc:633]
+---------------------------------------------+---------+---------------------------------------------+
| Model | Version | Status |
+---------------------------------------------+---------+---------------------------------------------+
| riva-onnx-fastpitch_encoder-Latin-American- | 1 | UNAVAILABLE: Internal: onnx runtime error 1 |
| Spanish | | 0: Load model from /data/models/riva-onnx-f |
| | | astpitch_encoder-Latin-American-Spanish/1/m |
| | | odel.onnx failed:This is an invalid model. |
| | | In Node, ("Identity_0", Identity, "Identity |
| | | _0", -1) : ("onnx::MatMul_3779": tensor(flo |
| | | at16),) -> ("onnx::MatMul_3978",) , Error N |
| | | o opset import for domain 'Identity_0' |
| | | |
| riva-trt-hifigan-Latin-American-Spanish | 1 | READY |
| spectrogram_chunker-Latin-American-Spanish | 1 | READY |
| tts_postprocessor-Latin-American-Spanish | 1 | READY |
| tts_preprocessor-Latin-American-Spanish | 1 | READY |
+---------------------------------------------+---------+---------------------------------------------+
Once converted Nemo model to ONNX the ONNX model uses an Identity_0 operation that tritonserver doesn’t like. How can I remove this opset operator when converting the model?
Can you give me a hand to see how to approach this?
Thank you very much!