Unable to start riva

AI & Data Science Deep Learning (Training & Inference) Riva
1

Please provide the following information when requesting support.

Hardware - NVIDIA GeForce RTX 3060 Laptop GPU
Hardware - 11th Gen Intel(R) Core™ i7-11800H @ 2.30GHz
Operating System - Ubuntu 20.04.3 LTS
Riva Version - 1.9.0-beta

After following the quick start guide (both of them) and executing succesfully
$ bash riva_init.sh
…
Riva initialization complete. Run ./riva_start.sh to launch services.

trying to start the following command:
$ bash riva_start.sh
Starting Riva Speech Services. This may take several minutes depending on the number of models deployed.
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
Waiting for Riva server to load all models…retrying in 10 seconds
…

2

here is the output of $ docker logs riva-speech

riva-speech.log.zip (56.3 KB)

3

Same issue on a different system with ubuntu 18.04

Hardware - NVIDIA GeForce RTX 2060 SUPER
Hardware - AMD Ryzen 9 3900XT 12-Core Processor
Operating System - Ubuntu 18.04.6 LTS
Riva Version - 1.9.0-beta

riva-speech-18.04.log.zip (48.8 KB)

6

Hi @javier7 ,
Thanks for your post and your interest in Riva, Apologies for the delay
Thanks for sharing the logs for both the machines that you have tried upon,
Analyzing the logs i can quickly find in both the machines
Cuda Error in copyToDevice: 2 (out of memory)
Please find the Server Hardware requirement link below
https://docs.nvidia.com/deeplearning/riva/user-guide/docs/support-matrix.html#server-hardware
As the link suggests,
Care must be taken to not exceed the memory available when selecting models to deploy. 16+GB VRAM is recommended.

As Both RTX 2060 and RTX 3060 Laptop GPU will have VRAM less than 16GB,
I would recommend to comment out the models that will not be used in the config.sh file, details on where config.sh is located is available in the quick start guide
https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html

I am attaching a screenshot below for reference, For example i want to use the TTS model (text to speech), then i can comment out all the models that I dont need by adding # to the start of those lines (Highlighted by Yellow Box) and uncommenting the model that i need to use (Here in below image highlighted by green box) and vice versa
You can do the same as per your requirement/usecase in the config.sh file,

image
image1820×762 49.2 KB

Alternatively we can also disable service as a whole if not needed in config.sh, lines below

Enable or Disable Riva Services

service_enabled_asr=true
service_enabled_nlp=true
service_enabled_tts=true

set to true or false depending upon your requirement/usecase,

So the Overall flow, here I would recommend you to try

  1. Navigate inside the quickstart directory cd riva_quickstart_v1.9.0-beta
  2. Run bash riva_clean.sh
  3. Modify the config.sh file to select only the model/service for your usecase/requirement as instructed above
  4. Run bash riva_init.sh (If possbile please share the console output ; Command command | tee ~/outputfile.txt in our case bash riva_init.sh | tee riva_init_log.txt, Please share the riva_init_logs.txt)
  5. Run bash riva_start.sh

Please try the above steps and let me know whether it works, also please do share the Nvidia Driver Version used, command nvidia-smi

2 Likes
7

Amazing, I understand, thanks for the answer, is it possible to run the components that do not fit the memory on one gpu on the other gpu that is in the other system in the same lan??

8

Hi @javier7 ,
Thanks you for your interest in Riva,
Unfortunately, Currently at the moment, we do not have support for running/inferencing over Multiple Machines with GPU over the same LAN Network,
You have to run two Riva Instances over the two machines separately

2 Likes
9

I understand, Thank you very much