Tacotron 2 TRTIS: Is it possible support non-English language like Vietnamese in trtis_cpp?

ttscolab · May 13, 2021, 1:42am

I am following the Text-To-Speech Services deploy guide - “Getting a Real Time Factor Over 60 for Text-To-Speech Services Using NVIDIA Jarvis” (https://developer.nvidia.com/blog/getting-real-time-factor-over-60-for-text-to-speech-using-jarvis/).

I have successfully deployed and inference with Tacotron 2 and Waveglow in English, however when switching to Vietnamese model, I got an error when inference when client sent text with Vietnamese language characters. However the inference succeeded with English characters. I changed /models/tacotron2waveglow/config.pbtxt, /models/tacotron2waveglow/mapping.txt with Vietnamese characters, /engines and engines were converted from the Vietnamese Tacotron 2 model.

The error I got on the trtserver side: “Error Exception in CustomContext :: execute (): Could not find ‘(195)’ in the alphabet.”
Thanks for help!

SunilJB · May 17, 2021, 4:26am

Hi @ttscolab,
I think by default it takes by default english_cleaner

github.com

NVIDIA/DeepLearningExamples/blob/bd257e1494adce448a685da2c898c0a05d40cde8/PyTorch/SpeechSynthesis/Tacotron2/preprocess_audio2mel.py#L17


def parse_args(parser):
    """
    Parse commandline arguments.
    """
    parser.add_argument('-d', '--dataset-path', type=str,
                        default='./', help='Path to dataset')
    parser.add_argument('--wav-files', required=True,
                        type=str, help='Path to filelist with audio paths and text')
    parser.add_argument('--mel-files', required=True,
                        type=str, help='Path to filelist with mel paths and text')
    parser.add_argument('--text-cleaners', nargs='*',
                        default=['english_cleaners'], type=str,
                        help='Type of text cleaners for input text')
    parser.add_argument('--max-wav-value', default=32768.0, type=float,
                        help='Maximum audiowave value')
    parser.add_argument('--sampling-rate', default=22050, type=int,
                        help='Sampling rate')
    parser.add_argument('--filter-length', default=1024, type=int,
                        help='Filter length')
    parser.add_argument('--hop-length', default=256, type=int,
                        help='Hop (stride) length')

You might have to use transliteration_cleaners instead: DeepLearningExamples/cleaners.py at bd257e1494adce448a685da2c898c0a05d40cde8 · NVIDIA/DeepLearningExamples · GitHub

Thanks

ttscolab · May 21, 2021, 2:36am

Thanks @SunilJB , but i used basic_cleaners when I inference with Pytorch model. No Error occurred.

SunilJB · May 21, 2021, 5:43am

Hi @ttscolab

basic_cleaners is just a “Basic pipeline that lowercases and collapses whitespace without transliteration”
transliteration_cleaners is “Pipeline for non-English text that transliterates to ASCII.”

It’s recommended to use transliteration_cleaners for non-English text, but based on the use case you can experiment with basic_cleaners as well.

Thanks