Hello everyone,
I’m a software engineer building an AI-powered application and would really appreciate guidance from the NVIDIA NeMo or Speech AI team.
Our goal is not text-to-speech, voice cloning, dubbing, or translation.
Instead, we are trying to build an AI pipeline that preserves the speaker completely while improving only the pronunciation.
The intended workflow is:
- A user uploads a prerecorded video.
- Our application extracts the original audio.
- If necessary, we also provide an accurate transcript to help the AI correctly interpret words that may be difficult to recognize because of a strong foreign accent.
- The AI processes the speech while preserving:
- the speaker’s original voice identity,
- emotions,
- speaking style,
- pacing,
- pauses,
- timing,
- and overall delivery.
- The only desired change is improving pronunciation and reducing the foreign accent.
- The processed audio is merged back into the original video.
The output should sound like the same person, with the same timing and emotions, but with a significantly more natural pronunciation.
During the past few weeks, I’ve evaluated several commercial speech AI solutions (Krisp, Sanas, Deepgram, Speechmatics, Rime, and others), but I haven’t found a solution that clearly supports this exact workflow.
My questions are:
- Is this currently possible using NVIDIA NeMo, Riva, NIM, or another NVIDIA speech technology?
- Would this require combining multiple models, or is there an existing speech-to-speech pipeline that could achieve this?
- Are there any NVIDIA reference architectures or research projects addressing this use case?
- If not currently supported, is this an area that NVIDIA is actively researching?
Any technical guidance or recommendations would be greatly appreciated.
Thank you very much!