Health ready check failed

AI & Data Science Deep Learning (Training & Inference) Riva
1

Hardware - GPU (A100/A30/T4/V100): NVIDIA RTX 3080
Hardware - CPU: Intel(R) Xeon(R) W-2225 CPU @ 4.10GHz
Operating System: Ubuntu 20.04.4 LTS
Riva Version: 2.8.0
TLT Version (if relevant)
How to reproduce the issue ?

When running the command
bash riva_start.sh
I get this error:
Health ready check failed.

Have attached the detailed logs in the text file.
riva_log.txt (146.7 KB)

Thank you so much for your help!

3

Hi @lohwenye

Thanks for your interest in Riva

Thanks for sharing the riva-speech docker logs

I find that one of the model is not present

E1208 07:12:27.101070 106 model_repository_manager.cc:996] Poll failed for model directory 'riva-trt-conformer-en-US-asr-streaming-am-streaming': failed to open text file for read /data/models/riva-trt-conformer-en-US-asr-streaming-am-streaming/config.pbtxt: No such file or directory

If you are using the default (found in riva_model_loc variable inside config.sh)

Please run
docker volume rm riva-model-repo
and
bash riva_clean.sh

And retry the setup again and let us know if you still face the issue

Thanks

4

Hi Rvinobha,

Thanks for the advice!

I have done the steps as suggested, but still face the same issue with Health ready check failed.

Have attached the latest docker logs.
riva_log2.txt (78.0 KB)

Thank you so much for your help!

5

HI @lohwenye

Apologies that did’nt help
Will check with the internal team on this

Thanks for sharing the docker logs riva-speech,
Request to also share the

  1. config.sh
  2. Complete log of riva_init.sh

Once shared I will check with the internal team

Thanks

6

I’m also experiencing the same “ Health Ready check failed” when running riva_start.sh. When checking the docker logs I see that it can’t connect to any of the remote hosts. I’m following the embedded QuickStart guide on Xavier NX