Trying jarvis ont the local docker

jason.hung1 · March 12, 2021, 9:37am

Description

Hi
I’m trying to start jarvis server with NeMo model STT Zh Quartznet15x5 STT Zh Quartznet15x5 | NVIDIA NGC
I successfully completed the Build process according to https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/custom-model-deployment.html but an error occurred in Deploy process Option 2 start jarvis-server
docker logs shows

== Jarvis Speech Skills ==

NVIDIA Release 20.11 (build 20304504)

Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying
project or file.

NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.

Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:19:46.629267 62 metrics.cc:219] Collecting metrics for GPU 0: Tesla V100-DGXS-32GB
I0312 09:19:46.838313 62 pinned_memory_manager.cc:199] Pinned memory pool is created at ‘0x7f2bb4000000’ with size 268435456
I0312 09:19:46.841249 62 cuda_memory_manager.cc:99] CUDA memory pool is created on device 0 with size 1000000000
I0312 09:19:46.849471 62 model_repository_manager.cc:810] loading: jarvis-asr-feature-extractor-streaming:1
I0312 09:19:46.849702 62 model_repository_manager.cc:810] loading: jarvis-asr-ctc-decoder-cpu-streaming:1
I0312 09:19:46.849819 62 model_repository_manager.cc:810] loading: jarvis-asr-voice-activity-detector-ctc-streaming:1
I0312 09:19:46.849942 62 model_repository_manager.cc:810] loading: jarvis-trt-jarvis-asr-am-streaming:1
I0312 09:19:46.854303 62 custom_backend.cc:198] Creating instance jarvis-asr-ctc-decoder-cpu-streaming_0_0_cpu on CPU using livis_asr_decoder_cpu.so
I0312 09:19:46.854560 62 custom_backend.cc:198] Creating instance jarvis-asr-voice-activity-detector-ctc-streaming_0_0_cpu on ibtriton_jarvis_asr_vad.so
I0312 09:19:46.854862 62 custom_backend.cc:201] Creating instance jarvis-asr-feature-extractor-streaming_0_0_gpu0 on GPU 0 (7.btriton_jarvis_asr_features.so
I0312 09:19:47.258561 62 model_repository_manager.cc:983] successfully loaded 'jarvis-asr-voice-activity-detector-ctc-streamin1
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:19:47.686355 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr-ctc-decoder-cpu-streaming’ version 1
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:20:03.326595 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr-feature-extractor-streaming’ version
I0312 09:20:03.766920 62 plan_backend.cc:333] Creating instance jarvis-trt-jarvis-asr-am-streaming_0_0_gpu0 on GPU 0 (7.0) usian
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:20:04.323157 62 plan_backend.cc:666] Created instance jarvis-trt-jarvis-asr-am-streaming_0_0_gpu0 on GPU 0 with strea0 and optimization profile default[0];
I0312 09:20:04.325880 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-trt-jarvis-asr-am-streaming’ version 1
I0312 09:20:04.326704 62 model_repository_manager.cc:810] loading: jarvis-asr:1
I0312 09:20:04.327123 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr’ version 1
I0312 09:20:04.327241 62 server.cc:141]
±--------±-------±-----+
| Backend | Config | Path |
±--------±-------±-----+
±--------±-------±-----+

I0312 09:20:04.327551 62 tritonserver.cc:1620]
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.5.0 |
| server_extensions | classification sequence model_repository schedule_policy model_configuration system_shareda_shared_memory binary_tensor_data statistics |
| model_repository_path[0] | /data/models |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 1000000000 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+

I0312 09:20:04.329248 62 grpc_server.cc:3979] Started GRPCInferenceService at 0.0.0.0:8001
I0312 09:20:04.329748 62 http_server.cc:2717] Started HTTPService at 0.0.0.0:8000
I0312 09:20:04.371290 62 http_server.cc:2736] Started Metrics Service at 0.0.0.0:8002

Triton server is ready…
I0312 09:20:04.941128 152 grpc_health.cc:42] JarvisHealthService initialized with server: localhost:8001
I0312 09:20:04.941220 152 grpc_jarvis_asr.cc:130] Setting uri for ASRServiceImpl
I0312 09:20:04.941236 152 grpc_jarvis_asr.cc:131] Initializing different models
I0312 09:20:04.942224 152 model_registry.cc:52] JarvisModelRegistry initialized with server: localhost:8001
I0312 09:20:04.944448 152 model_registry.cc:81] Server Name: triton, Server version: 2.5.0
I0312 09:20:04.944850 152 model_registry.cc:102] Our model repository has a total of: 5 models
I0312 09:20:04.944864 152 model_registry.cc:107] Model names: jarvis-asr, Model version: 1
I0312 09:20:04.951378 152 model_registry.cc:120] ‘Successfully registering jarvis-asr’
I0312 09:20:04.951535 152 model_registry.cc:107] Model names: jarvis-asr-ctc-decoder-cpu-streaming, Model version: 1
I0312 09:20:04.956008 152 model_registry.cc:107] Model names: jarvis-asr-feature-extractor-streaming, Model version: 1
I0312 09:20:04.959628 152 model_registry.cc:107] Model names: jarvis-asr-voice-activity-detector-ctc-streaming, Model versio
I0312 09:20:04.962378 152 model_registry.cc:107] Model names: jarvis-trt-jarvis-asr-am-streaming, Model version: 1
I0312 09:20:04.963699 152 model_registry.cc:125] Successfully registered: 1 models.
I0312 09:20:04.963742 152 client.cc:54] JarvisNLPClient initialized with server: localhost:8001
I0312 09:20:04.964113 152 client.cc:70] Our model repository has: 5 models.
W0312 09:20:04.966213 152 client.cc:94] Registration of ‘jarvis-asr’ failed with unknown service type
I0312 09:20:04.975970 152 grpc_jarvis_asr.cc:153] Seeding RNG used for correlation id with time: 1615540804
I0312 09:20:04.976390 152 jarvis_server.cc:70] ASR Service connected to Triton at localhost:8001
I0312 09:20:04.976408 152 jarvis_server.cc:73] Jarvis Conversational AI Server listening on 0.0.0.0:50051

Any help would be appreciated.
Thanks

SunilJB · March 12, 2021, 5:01pm

Hi @jason.hung1
I don’t see any error in the log text attached in the forum. Could you please attach the log file and elaborate more on the issue you are getting?
Also could you please share the setup/platform info on which your are trying to run jarvis skill?
Please refer to support matrix for all the supported GPUs. https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/support-matrix.html

Thanks

jason.hung1 · March 15, 2021, 4:08am

Hi @SunilJB
I’m sorry that my question is not as clear as the description.
I use the NeMo model of stt zh quartnet15X5,jarvis build a .jmis file, and then
bash jarvis_init.sh
bash jarvis_start.sh
bash jarvis_start_client.sh
jarvis_asr_client --audio_file=/work/wav/sample.wav
I expect jarvis_asr_client --audio_file=/work/wav/sample.wav to be like https: //docs.nvidia.com/deeplearning/jarvis/user-guide/docs/quick-start-guide.html gave me ASR result but he appeared RPC failed: Error: Model is not available on server
Below is my log and environment. Thanks
config.sh (5.5 KB)
jarvis_client.log (234 Bytes)
jarvis_server.log (11.1 KB)

Environment

GPU Model: Tesla V100 DGXS 32GB
Nvidia Driver Version: 440.59
CUDA Version: 10.2
Operating System + Version: Linux dgx 4.15.0-101-generic X86_64
Container:nvcr.io/nvidia/jarvis/jarvis-speech:1.0.0-b.2-server

SunilJB · March 15, 2021, 8:43am

Thank @jason.hung1
Just wanted to check if you tried below steps to deploy the NeMo Model
https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/model-overview.html#nemo-to-jarvis

Thanks

jason.hung1 · March 15, 2021, 8:59am

@SunilJB
I have tried https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/model-overview.html#nemo-to-jarvis. After I tried that, I tried https://docs.nvidia.com/deeplearning/jarvis/user-guide/docs/model-overview.html#jarvis-build
with jarvis-build speech_recognition /servicemaker-dev/jmir_chinese.jmir
/servicemaker-dev/quartznet_asr.enemo.
I think those two files are too big, I can’t upload them here

Thanks

SunilJB · March 18, 2021, 5:05am

@jason.hung1

Let me try to reproduce the issue at my end. Will update you.

Thanks

SunilJB · March 18, 2021, 6:10pm

Hi @jason.hung1
I think you have deployed the model in streaming mode. In that case you need to use the streaming client.
Try running jarvis_streaming_asr_client instead of jarvis_asr_client

Thanks

jason.hung1 · March 19, 2021, 1:58am

I have rebuilt an offline model and he succeeded, thank you very much for your help.

pete.hanlon · May 9, 2021, 3:38pm

Hi @jason.hung1

What did you change to rebuild your model in offline mode?

Thanks

pete.hanlon · May 9, 2021, 4:49pm

I tried adding --offline to my jarvis-build command and re-deployed. I’ve now got jarvis-asr-feature-extractor-streaming-offline, jarvis-asr-ctc-decoder-cpu-streaming-offline, jarvis-asr-voice-activity-detector-ctc-streaming-offline, jarvis-trt-jarvis-asr-am-streaming-offline in my models directory but I still get an error when I try to run transcribe_file_offline.py

jason.hung1 · May 10, 2021, 7:15am

Hi @pete.hanlon

I just follow Riva — NVIDIA Riva to build the offline mode, but I didn’t try the transcribe_file_offline.py.
I use jarvis_start_client.sh to start the jarvis client and then use the jarvis_asr_client command to check the offline mode.
Maybe you can post the error code you encountered or open a new issue.

pete.hanlon · May 10, 2021, 8:14am

Thanks for your help Jason, I will raise a new ticket.