Hello,
im trying to deploy a .nemo Quartznet Model.
To reproduce, im listing the steps i have done sofar:
- I transformed my
.nemofile into an.rivafile by using the nemo2riva package.
→german_quartznet.riva - Riva-Build as follows:
docker run --gpus all -it --rm -v /home/ws/untow/riva2:/servicemaker-dev -v /home/ws/untow/riva2/models:/data --entrypoint="/bin/bash" nvcr.io/nvidia/riva/riva-speech:1.5.0-beta-servicemaker
riva-build speech_recognition /servicemaker-dev/german_quartznet_greedy.rmir /servicemaker-dev/german_quartznet.riva --offline --decoder_type=greedy
- ./riva_init.sh output:
Logging into NGC docker registry if necessary...
Pulling required docker images if necessary...
Note: This may take some time, depending on the speed of your Internet connection.
> Pulling Riva Speech Server images.
> Image nvcr.io/nvidia/riva/riva-speech:1.5.0-beta-server exists. Skipping.
> Image nvcr.io/nvidia/riva/riva-speech-client:1.5.0-beta exists. Skipping.
> Image nvcr.io/nvidia/riva/riva-speech:1.5.0-beta-servicemaker exists. Skipping.
Converting RMIRs at /home/ws/untow/riva2/riva_model_loc/rmir to Riva Model repository.
+ docker run --init -it --rm --gpus '"device=0"' -v /home/ws/untow/riva2/riva_model_loc:/data -e MODEL_DEPLOY_KEY=tlt_encode --name riva-service-maker nvcr.io/nvidia/riva/riva-speech:1.5.0-beta-servicemaker deploy_all_models /data/rmir /data/models
==========================
=== Riva Speech Skills ===
==========================
NVIDIA Release devel (build 22382700)
Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.
Various files include modifications (c) NVIDIA CORPORATION. All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying
project or file.
NOTE: The SHMEM allocation limit is set to the default of 64MB. This may be
insufficient for the inference server. NVIDIA recommends the use of the following flags:
nvidia-docker run --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 ...
2021-09-07 12:01:16,375 [INFO] Writing Riva model repository to '/data/models'...
2021-09-07 12:01:16,375 [INFO] The riva model repo target directory is /data/models
2021-09-07 12:01:16,743 [INFO] Extract_binaries for featurizer -> /data/models/riva-asr-feature-extractor-streaming-offline/1
2021-09-07 12:01:16,744 [INFO] Extract_binaries for nn -> /data/models/riva-trt-riva-asr-am-streaming-offline/1
2021-09-07 12:01:16,776 [INFO] Printing copied artifacts:
2021-09-07 12:01:16,776 [INFO] {'onnx': '/data/models/riva-trt-riva-asr-am-streaming-offline/1/model_graph.onnx'}
2021-09-07 12:01:16,776 [INFO] Building TRT engine from ONNX file
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:227: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] ERROR: safeContext.cpp (124) - Cudnn Error in initializeCommonContext: 4 (Could not initialize cudnn, please check cudnn installation.)
[TensorRT] ERROR: safeContext.cpp (124) - Cudnn Error in initializeCommonContext: 4 (Could not initialize cudnn, please check cudnn installation.)
2021-09-07 12:01:23,164 [ERROR] Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/servicemaker/cli/deploy.py", line 86, in deploy_from_rmir
generator.serialize_to_disk(
File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 356, in serialize_to_disk
module.serialize_to_disk(repo_dir, rmir, config_only, verbose, overwrite)
File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 244, in serialize_to_disk
self.update_binary(version_dir, rmir, verbose)
File "/opt/conda/lib/python3.8/site-packages/servicemaker/triton/triton.py", line 603, in update_binary
with self.build_trt_engine_from_onnx(model_weights) as engine, open(
AttributeError: __enter__
+ echo
+ echo 'Riva initialization complete. Run ./riva_start.sh to launch services.'
Riva initialization complete. Run ./riva_start.sh to launch services.
- ./riva_start.sh
→health ready check failed
The weird thing is that i was able to initialize without an error yesterday. I already started from scratch 4 or 5 times now but the Error is consistent.
Thanks for your help in advance!