Description
Hi
I’m trying to start jarvis server with NeMo model STT Zh Quartznet15x5 NVIDIA NGC
I successfully completed the Build process according to Deploying Your Custom Model into Jarvis — NVIDIA Jarvis Speech Skills v1.0.0-b.2 documentation but an error occurred in Deploy process Option 2 start jarvis-server
docker logs shows
== Jarvis Speech Skills ==
NVIDIA Release 20.11 (build 20304504)
Copyright © 2018-2020, NVIDIA CORPORATION. All rights reserved.
Various files include modifications © NVIDIA CORPORATION. All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying
project or file.
NOTE: Legacy NVIDIA Driver detected. Compatibility mode ENABLED.
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:19:46.629267 62 metrics.cc:219] Collecting metrics for GPU 0: Tesla V100-DGXS-32GB
I0312 09:19:46.838313 62 pinned_memory_manager.cc:199] Pinned memory pool is created at ‘0x7f2bb4000000’ with size 268435456
I0312 09:19:46.841249 62 cuda_memory_manager.cc:99] CUDA memory pool is created on device 0 with size 1000000000
I0312 09:19:46.849471 62 model_repository_manager.cc:810] loading: jarvis-asr-feature-extractor-streaming:1
I0312 09:19:46.849702 62 model_repository_manager.cc:810] loading: jarvis-asr-ctc-decoder-cpu-streaming:1
I0312 09:19:46.849819 62 model_repository_manager.cc:810] loading: jarvis-asr-voice-activity-detector-ctc-streaming:1
I0312 09:19:46.849942 62 model_repository_manager.cc:810] loading: jarvis-trt-jarvis-asr-am-streaming:1
I0312 09:19:46.854303 62 custom_backend.cc:198] Creating instance jarvis-asr-ctc-decoder-cpu-streaming_0_0_cpu on CPU using livis_asr_decoder_cpu.so
I0312 09:19:46.854560 62 custom_backend.cc:198] Creating instance jarvis-asr-voice-activity-detector-ctc-streaming_0_0_cpu on ibtriton_jarvis_asr_vad.so
I0312 09:19:46.854862 62 custom_backend.cc:201] Creating instance jarvis-asr-feature-extractor-streaming_0_0_gpu0 on GPU 0 (7.btriton_jarvis_asr_features.so
I0312 09:19:47.258561 62 model_repository_manager.cc:983] successfully loaded 'jarvis-asr-voice-activity-detector-ctc-streamin1
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:19:47.686355 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr-ctc-decoder-cpu-streaming’ version 1
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:20:03.326595 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr-feature-extractor-streaming’ version
I0312 09:20:03.766920 62 plan_backend.cc:333] Creating instance jarvis-trt-jarvis-asr-am-streaming_0_0_gpu0 on GPU 0 (7.0) usian
Jarvis waiting for Triton server to load all models…retrying in 1 second
I0312 09:20:04.323157 62 plan_backend.cc:666] Created instance jarvis-trt-jarvis-asr-am-streaming_0_0_gpu0 on GPU 0 with strea0 and optimization profile default[0];
I0312 09:20:04.325880 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-trt-jarvis-asr-am-streaming’ version 1
I0312 09:20:04.326704 62 model_repository_manager.cc:810] loading: jarvis-asr:1
I0312 09:20:04.327123 62 model_repository_manager.cc:983] successfully loaded ‘jarvis-asr’ version 1
I0312 09:20:04.327241 62 server.cc:141]
±--------±-------±-----+
| Backend | Config | Path |
±--------±-------±-----+
±--------±-------±-----+
I0312 09:20:04.327355 62 server.cc:184]
±-------------------------------------------------±--------±-------+
| Model | Version | Status |
±-------------------------------------------------±--------±-------+
| jarvis-asr | 1 | READY |
| jarvis-asr-ctc-decoder-cpu-streaming | 1 | READY |
| jarvis-asr-feature-extractor-streaming | 1 | READY |
| jarvis-asr-voice-activity-detector-ctc-streaming | 1 | READY |
| jarvis-trt-jarvis-asr-am-streaming | 1 | READY |
±-------------------------------------------------±--------±-------+
I0312 09:20:04.327551 62 tritonserver.cc:1620]
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.5.0 |
| server_extensions | classification sequence model_repository schedule_policy model_configuration system_shareda_shared_memory binary_tensor_data statistics |
| model_repository_path[0] | /data/models |
| model_control_mode | MODE_NONE |
| strict_model_config | 1 |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 1000000000 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
±---------------------------------±----------------------------------------------------------------------------------------------------------------------------------------+
I0312 09:20:04.329248 62 grpc_server.cc:3979] Started GRPCInferenceService at 0.0.0.0:8001
I0312 09:20:04.329748 62 http_server.cc:2717] Started HTTPService at 0.0.0.0:8000
I0312 09:20:04.371290 62 http_server.cc:2736] Started Metrics Service at 0.0.0.0:8002
Triton server is ready…
I0312 09:20:04.941128 152 grpc_health.cc:42] JarvisHealthService initialized with server: localhost:8001
I0312 09:20:04.941220 152 grpc_jarvis_asr.cc:130] Setting uri for ASRServiceImpl
I0312 09:20:04.941236 152 grpc_jarvis_asr.cc:131] Initializing different models
I0312 09:20:04.942224 152 model_registry.cc:52] JarvisModelRegistry initialized with server: localhost:8001
I0312 09:20:04.944448 152 model_registry.cc:81] Server Name: triton, Server version: 2.5.0
I0312 09:20:04.944850 152 model_registry.cc:102] Our model repository has a total of: 5 models
I0312 09:20:04.944864 152 model_registry.cc:107] Model names: jarvis-asr, Model version: 1
I0312 09:20:04.951378 152 model_registry.cc:120] ‘Successfully registering jarvis-asr’
I0312 09:20:04.951535 152 model_registry.cc:107] Model names: jarvis-asr-ctc-decoder-cpu-streaming, Model version: 1
I0312 09:20:04.956008 152 model_registry.cc:107] Model names: jarvis-asr-feature-extractor-streaming, Model version: 1
I0312 09:20:04.959628 152 model_registry.cc:107] Model names: jarvis-asr-voice-activity-detector-ctc-streaming, Model versio
I0312 09:20:04.962378 152 model_registry.cc:107] Model names: jarvis-trt-jarvis-asr-am-streaming, Model version: 1
I0312 09:20:04.963699 152 model_registry.cc:125] Successfully registered: 1 models.
I0312 09:20:04.963742 152 client.cc:54] JarvisNLPClient initialized with server: localhost:8001
I0312 09:20:04.964113 152 client.cc:70] Our model repository has: 5 models.
W0312 09:20:04.966213 152 client.cc:94] Registration of ‘jarvis-asr’ failed with unknown service type
I0312 09:20:04.975970 152 grpc_jarvis_asr.cc:153] Seeding RNG used for correlation id with time: 1615540804
I0312 09:20:04.976390 152 jarvis_server.cc:70] ASR Service connected to Triton at localhost:8001
I0312 09:20:04.976408 152 jarvis_server.cc:73] Jarvis Conversational AI Server listening on 0.0.0.0:50051
Any help would be appreciated.
Thanks