I encountered several issues while running the assessment.ipynb
file in the NVIDIA DLI course “Building Conversational AI Applications”, specifically during the setup and execution of Riva Speech Skills. Below are the details of the problems:
- Model Loading Failure:
- The Triton Inference Server fails to load models and terminates unexpectedly during the initialization phase.
- Error messages from
docker logs
include:
error: creating server: Internal - failed to load all models
> Triton server died before reaching ready state. Terminating Riva startup.
- Additional errors indicate missing configuration files (
config.pbtxt
) for certain models:
Poll failed for model directory '1': failed to open text file for read /data/models/1/config.pbtxt: No such file or directory
- Excessive Number of Models:
- The
/data/models
directory contains an excessive number of models, including ASR, TTS, and NLP-related models. - This appears to prolong the model loading process, leading to timeout issues.
- Timeout Issues:
- Riva Speech Skills waits for the models to load but fails due to a timeout:
Timeout 29: Found 4 live models and 0 in-flight non-inference requests
- Despite increasing the timeout value, the server still fails to initialize all models successfully.
- Docker Environment Configuration:
- Potential misconfigurations in Docker container resource allocation (e.g., memory, GPU usage) could also be contributing to the problem.
Steps Taken:
- Verified the contents of the
/data/models
directory usingdocker exec
and confirmed that some models are missing critical files likeconfig.pbtxt
. - Attempted to reduce the number of models by only keeping those relevant to ASR, but the server still fails to start.
- Edited the
riva_start.sh
script to extend the timeout period but encountered the same issue.
Request for Assistance:
- What is the recommended way to handle the excessive number of models? Is there a list of essential models required for basic ASR functionality?
- How can I ensure all necessary model files (e.g.,
config.pbtxt
) are present and properly configured? - Are there additional changes needed in the
riva_start.sh
script or Docker configuration to resolve this issue? - Could there be compatibility issues between the Triton Inference Server and Riva Speech Skills, given the current setup?
Any guidance or suggestions to resolve these issues would be greatly appreciated. Thank you!