Hi all,
I could not get the bash jarvis_start.sh` executed successfully. I completed executing ‘bash jarvis_init.sh’ with the necessary models being downloaded and extracted. I am using Ubuntu 20.04 with RTX 3060, CPU AMD Ryzen 5800X.
The error I got is
Starting Jarvis Speech Services. This may take several minutes depending on the number of models deployed.
Waiting for Jarvis server to load all models...retrying in 10 seconds
This is the output of docker logs jarvis-speech.
==========================
== Jarvis Speech Skills ==
==========================
NVIDIA Release 20.11 (build 19933361)
Copyright (c) 2018-2020, NVIDIA CORPORATION. All rights reserved.
Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying
project or file.
NOTE: The SHMEM allocation limit is set to the default of 64MB. This may be
insufficient for the inference server. NVIDIA recommends the use of the following flags:
nvidia-docker run --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 ...
> Jarvis waiting for Triton server to load all models...retrying in 1 second
I0422 02:00:23.852090 74 metrics.cc:219] Collecting metrics for GPU 0: NVIDIA GeForce RTX 3060
I0422 02:00:23.969278 74 pinned_memory_manager.cc:199] Pinned memory pool is created at '0x7f7cc0000000' with size 268435456
I0422 02:00:23.969574 74 cuda_memory_manager.cc:99] CUDA memory pool is created on device 0 with size 1000000000
I0422 02:00:23.980896 74 model_repository_manager.cc:810] loading: jarvis-trt-jarvis_ner-nn-bert-base-uncased:1
I0422 02:00:23.980974 74 model_repository_manager.cc:810] loading: jarvis-trt-jarvis_intent_weather-nn-bert-base-uncased:1
I0422 02:00:23.981026 74 model_repository_manager.cc:810] loading: jarvis-trt-jarvis_punctuation-nn-bert-base-uncased:1
I0422 02:00:23.981072 74 model_repository_manager.cc:810] loading: jarvis-trt-jarvis_qa-nn-bert-base-uncased:1
I0422 02:00:23.981126 74 model_repository_manager.cc:810] loading: jarvis-trt-jarvis_text_classification_domain-nn-bert-base-uncased:1
I0422 02:00:23.981187 74 model_repository_manager.cc:810] loading: jarvis-trt-jasper:1
I0422 02:00:23.981268 74 model_repository_manager.cc:810] loading: jarvis-trt-tacotron2_encoder:1
I0422 02:00:23.981347 74 model_repository_manager.cc:810] loading: jarvis-trt-waveglow:1
I0422 02:00:23.981495 74 model_repository_manager.cc:810] loading: jarvis_detokenize:1
I0422 02:00:23.981597 74 model_repository_manager.cc:810] loading: jarvis_label_tokens_weather:1
I0422 02:00:23.981720 74 model_repository_manager.cc:810] loading: jarvis_ner_label_tokens:1
I0422 02:00:23.981801 74 custom_backend.cc:198] Creating instance jarvis_detokenize_0_0_cpu on CPU using libtriton_jarvis_nlp_detokenizer.so
I0422 02:00:23.981809 74 model_repository_manager.cc:810] loading: jarvis_punctuation_gen_output:1
I0422 02:00:23.981882 74 model_repository_manager.cc:810] loading: jarvis_punctuation_label_tokens_cap:1
I0422 02:00:23.981962 74 model_repository_manager.cc:810] loading: jarvis_punctuation_label_tokens_punct:1
I0422 02:00:23.982048 74 model_repository_manager.cc:810] loading: jarvis_punctuation_merge_labels:1
I0422 02:00:23.982145 74 model_repository_manager.cc:810] loading: jarvis_qa_postprocessor:1
I0422 02:00:23.982194 74 custom_backend.cc:198] Creating instance jarvis_ner_label_tokens_0_0_cpu on CPU using libtriton_jarvis_nlp_seqlabel.so
I0422 02:00:23.982236 74 model_repository_manager.cc:810] loading: jarvis_qa_preprocessor:1
I0422 02:00:23.982322 74 model_repository_manager.cc:810] loading: jarvis_tokenizer:1
I0422 02:00:23.982414 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-ctc-decoder-cpu-streaming:1
I0422 02:00:23.982512 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-feature-extractor-streaming:1
I0422 02:00:23.982618 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-offline-ctc-decoder-cpu-streaming-offline:1
I0422 02:00:23.982722 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-offline-feature-extractor-streaming-offline:1
I0422 02:00:23.982818 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-offline-voice-activity-detector-ctc-streaming-offline:1
I0422 02:00:23.982901 74 model_repository_manager.cc:810] loading: jasper-asr-trt-ensemble-vad-streaming-voice-activity-detector-ctc-streaming:1
I0422 02:00:23.982987 74 model_repository_manager.cc:810] loading: tacotron2_decoder_postnet:1
I0422 02:00:23.982993 74 custom_backend.cc:198] Creating instance jarvis_punctuation_gen_output_0_0_cpu on CPU using libtriton_jarvis_nlp_punctuation.so
I0422 02:00:23.983081 74 model_repository_manager.cc:810] loading: tts_preprocessor:1
I0422 02:00:23.983177 74 model_repository_manager.cc:810] loading: waveglow_denoiser:1
I0422 02:00:23.983547 74 custom_backend.cc:198] Creating instance jarvis_punctuation_label_tokens_cap_0_0_cpu on CPU using libtriton_jarvis_nlp_seqlabel.so
I0422 02:00:23.983716 74 custom_backend.cc:198] Creating instance jarvis_punctuation_label_tokens_punct_0_0_cpu on CPU using libtriton_jarvis_nlp_seqlabel.so
I0422 02:00:23.984161 74 custom_backend.cc:198] Creating instance jarvis_punctuation_merge_labels_0_0_cpu on CPU using libtriton_jarvis_nlp_labels.so
I0422 02:00:23.984475 74 custom_backend.cc:198] Creating instance jarvis_qa_postprocessor_0_0_cpu on CPU using libtriton_jarvis_nlp_qa.so
I0422 02:00:23.984746 74 custom_backend.cc:198] Creating instance jarvis_qa_preprocessor_0_0_cpu on CPU using libtriton_jarvis_nlp_tokenizer.so
I0422 02:00:23.984953 74 custom_backend.cc:198] Creating instance jarvis_tokenizer_0_0_cpu on CPU using libtriton_jarvis_nlp_tokenizer.so
I0422 02:00:23.985166 74 custom_backend.cc:198] Creating instance jasper-asr-trt-ensemble-vad-streaming-ctc-decoder-cpu-streaming_0_0_cpu on CPU using libtriton_jarvis_asr_decoder_cpu.so
I0422 02:00:23.985463 74 custom_backend.cc:198] Creating instance jasper-asr-trt-ensemble-vad-streaming-offline-ctc-decoder-cpu-streaming-offline_0_0_cpu on CPU using libtriton_jarvis_asr_decoder_cpu.so
I0422 02:00:23.985704 74 custom_backend.cc:198] Creating instance jasper-asr-trt-ensemble-vad-streaming-offline-voice-activity-detector-ctc-streaming-offline_0_0_cpu on CPU using libtriton_jarvis_asr_vad.so
I0422 02:00:23.985819 74 custom_backend.cc:198] Creating instance jasper-asr-trt-ensemble-vad-streaming-voice-activity-detector-ctc-streaming_0_0_cpu on CPU using libtriton_jarvis_asr_vad.so
I0422 02:00:23.986227 74 custom_backend.cc:198] Creating instance jarvis_label_tokens_weather_0_0_cpu on CPU using libtriton_jarvis_nlp_seqlabel.so
I0422 02:00:23.986778 74 custom_backend.cc:201] Creating instance jasper-asr-trt-ensemble-vad-streaming-feature-extractor-streaming_0_0_gpu0 on GPU 0 (8.6) using libtriton_jarvis_asr_features.so
I0422 02:00:23.986917 74 custom_backend.cc:201] Creating instance jasper-asr-trt-ensemble-vad-streaming-offline-feature-extractor-streaming-offline_0_0_gpu0 on GPU 0 (8.6) using libtriton_jarvis_asr_features.so
I0422 02:00:23.987047 74 custom_backend.cc:201] Creating instance tts_preprocessor_0_0_gpu0 on GPU 0 (8.6) using libtriton_jarvis_tts_preprocessor.so
I0422 02:00:23.987074 74 custom_backend.cc:201] Creating instance waveglow_denoiser_0_0_gpu0 on GPU 0 (8.6) using libtriton_jarvis_tts_denoiser.so
I0422 02:00:24.102499 74 model_repository_manager.cc:983] successfully loaded 'tts_preprocessor' version 1
I0422 02:00:24.118252 74 model_repository_manager.cc:983] successfully loaded 'jarvis_punctuation_gen_output' version 1
I0422 02:00:24.118572 74 model_repository_manager.cc:983] successfully loaded 'jarvis_detokenize' version 1
I0422 02:00:24.130729 74 tacotron-decoder-postnet.cc:870] TRITONBACKEND_ModelInitialize: tacotron2_decoder_postnet (version 1)
I0422 02:00:24.130920 74 model_repository_manager.cc:983] successfully loaded 'jarvis_punctuation_merge_labels' version 1
I0422 02:00:24.130924 74 model_repository_manager.cc:983] successfully loaded 'jarvis_punctuation_label_tokens_cap' version 1
I0422 02:00:24.130937 74 model_repository_manager.cc:983] successfully loaded 'jarvis_punctuation_label_tokens_punct' version 1
I0422 02:00:24.131062 74 model_repository_manager.cc:983] successfully loaded 'jarvis_label_tokens_weather' version 1
I0422 02:00:24.131101 74 model_repository_manager.cc:983] successfully loaded 'jarvis_ner_label_tokens' version 1
I0422 02:00:24.131395 74 model_repository_manager.cc:983] successfully loaded 'jarvis_qa_postprocessor' version 1
I0422 02:00:24.152859 74 tacotron-decoder-postnet.cc:764] model configuration:
{
"name": "tacotron2_decoder_postnet",
"platform": "",
"backend": "jarvis_tts_taco_postnet",
"version_policy": {
"latest": {
"num_versions": 1
}
},
"max_batch_size": 8,
"input": [
{
"name": "input_decoder",
"data_type": "TYPE_FP32",
"format": "FORMAT_NONE",
"dims": [
1,
400,
512
],
"is_shape_tensor": false,
"allow_ragged_batch": false
},
{
"name": "input_processed_decoder",
"data_type": "TYPE_FP32",
"format": "FORMAT_NONE",
"dims": [
400,
128,
1,
1
],
"is_shape_tensor": false,
"allow_ragged_batch": false
},
{
"name": "input_num_characters",
"data_type": "TYPE_INT32",
"format": "FORMAT_NONE",
"dims": [
1
],
"is_shape_tensor": false,
"allow_ragged_batch": false
}
],
"output": [
{
"name": "spectrogram_chunk",
"data_type": "TYPE_FP32",
"dims": [
1,
80,
80
],
"label_filename": "",
"is_shape_tensor": false
},
{
"name": "z",
"data_type": "TYPE_FP32",
"dims": [
8,
2656,
1
],
"label_filename": "",
"is_shape_tensor": false
},
{
"name": "num_valid_samples",
"data_type": "TYPE_INT32",
"dims": [
1
],
"label_filename": "",
"is_shape_tensor": false
},
{
"name": "end_flag",
"data_type": "TYPE_INT32",
"dims": [
1
],
"label_filename": "",
"is_shape_tensor": false
}
],
"batch_input": [],
"batch_output": [],
"optimization": {
"priority": "PRIORITY_DEFAULT",
"input_pinned_memory": {
"enable": true
},
"output_pinned_memory": {
"enable": true
}
},
"sequence_batching": {
"oldest": {
"max_candidate_sequences": 8,
"preferred_batch_size": [
8
],
"max_queue_delay_microseconds": 100
},
"max_sequence_idle_microseconds": 60000000,
"control_input": [
{
"name": "START",
"control": [
{
"kind": "CONTROL_SEQUENCE_START",
"int32_false_true": [
0,
1
],
"fp32_false_true": [],
"data_type": "TYPE_INVALID"
}
]
},
{
"name": "READY",
"control": [
{
"kind": "CONTROL_SEQUENCE_READY",
"int32_false_true": [
0,
1
],
"fp32_false_true": [],
"data_type": "TYPE_INVALID"
}
]
},
{
"name": "END",
"control": [
{
"kind": "CONTROL_SEQUENCE_END",
"int32_false_true": [
0,
1
],
"fp32_false_true": [],
"data_type": "TYPE_INVALID"
}
]
},
{
"name": "CORRID",
"control": [
{
"kind": "CONTROL_SEQUENCE_CORRID",
"int32_false_true": [],
"fp32_false_true": [],
"data_type": "TYPE_UINT64"
}
]
}
]
},
"instance_group": [
{
"name": "tacotron2_decoder_postnet_0",
"kind": "KIND_GPU",
"count": 1,
"gpus": [
0
],
"profile": []
}
],
"default_model_filename": "",
"cc_model_filenames": {},
"metric_tags": {},
"parameters": {
"num_samples_per_frame": {
"string_value": "256"
},
"z_dim0": {
"string_value": "8"
},
"z_dim1": {
"string_value": "2656"
},
"tacotron_decoder_engine": {
"string_value": "/data/models/tacotron2_decoder_postnet/1/model.plan"
},
"num_mels": {
"string_value": "80"
},
"encoding_dimension": {
"string_value": "512"
},
"max_execution_batch_size": {
"string_value": "8"
},
"chunk_length": {
"string_value": "80"
},
"max_input_length": {
"string_value": "400"
},
"attention_dimension": {
"string_value": "128"
}
},
"model_warmup": [],
"model_transaction_policy": {
"decoupled": true
}
}
I0422 02:00:24.153068 74 tacotron-decoder-postnet.cc:927] TRITONBACKEND_ModelInstanceInitialize: tacotron2_decoder_postnet_0 (device 0)
I0422 02:00:24.161061 74 model_repository_manager.cc:983] successfully loaded 'jasper-asr-trt-ensemble-vad-streaming-voice-activity-detector-ctc-streaming' version 1
I0422 02:00:24.176291 74 model_repository_manager.cc:983] successfully loaded 'jasper-asr-trt-ensemble-vad-streaming-offline-voice-activity-detector-ctc-streaming-offline' version 1
I0422 02:00:24.192692 74 model_repository_manager.cc:983] successfully loaded 'jarvis_tokenizer' version 1
I0422 02:00:24.209160 74 model_repository_manager.cc:983] successfully loaded 'jarvis_qa_preprocessor' version 1
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
I0422 02:00:29.718572 74 model_repository_manager.cc:983] successfully loaded 'jasper-asr-trt-ensemble-vad-streaming-offline-ctc-decoder-cpu-streaming-offline' version 1
I0422 02:00:29.739433 74 model_repository_manager.cc:983] successfully loaded 'jasper-asr-trt-ensemble-vad-streaming-ctc-decoder-cpu-streaming' version 1
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
> Jarvis waiting for Triton server to load all models...retrying in 1 second
[E] [TRT] INVALID_CONFIG: The engine plan file is not compatible with this version of TensorRT, expecting library version 7.2.1 got 7.2.2, please rebuild.
[E] [TRT] engine.cpp (1646) - Serialization Error in deserialize: 0 (Core engine deserialization failure)
[E] [TRT] INVALID_STATE: std::exception
[E] [TRT] INVALID_CONFIG: Deserialize the cuda engine failed.
WARNING: Failed to load denoiser: Failed to deserialize engine.
E0422 02:00:35.670627 74 dynamic_batch_scheduler.cc:248] Initialization failed for dynamic-batch scheduler thread 0: initialize error for 'waveglow_denoiser': (23) unable to load denoiser model
E0422 02:00:35.670744 74 sequence_batch_scheduler.cc:1286] failed creating dynamic sequence batcher for OldestFirst 0: Initialization failed for all dynamic-batch scheduler threads
E0422 02:00:35.670995 74 model_repository_manager.cc:986] failed to load 'waveglow_denoiser' version 1: Internal: Initialization failed for all sequence-batch scheduler threads
[03/22/2021-02:00:35] [03/22/2021-02:00:35] [03/22/2021-02:00:35] [03/22/2021-02:00:35] > Jarvis waiting for Triton server to load all models...retrying in 1 second
/opt/jarvis/bin/start-jarvis: line 4: 74 Segmentation fault (core dumped) tritonserver --log-verbose=0 --strict-model-config=true $model_repos --cuda-memory-pool-byte-size=0:1000000000
> Triton server died before reaching ready state. Terminating Jarvis startup.
Check Triton logs with: docker logs
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]