Please provide the following information when requesting support.
Hardware - GPU (RTX5090)
Hardware - CPU
Operating System
Riva Version:2.19.0
TLT Version (if relevant)
025-10-26 01:28:20,680 [INFO] [TRT-LLM] [I] Set weight_only_quant_matmul_plugin to float16.
[10/26/2025-01:28:20] [TRT-LLM] [I] Set nccl_plugin to None.
2025-10-26 01:28:20,680 [INFO] [TRT-LLM] [I] Set nccl_plugin to None.
2025-10-26 01:28:20,723 [ERROR] Traceback (most recent call last):
File “/usr/local/lib/python3.12/dist-packages/servicemaker/cli/deploy.py”, line 103, in deploy_from_rmir
generator.serialize_to_disk(
File “/usr/local/lib/python3.12/dist-packages/servicemaker/triton/triton.py”, line 497, in serialize_to_disk
module.serialize_to_disk(repo_dir, rmir, config_only, verbose, overwrite, gpus)
File “/usr/local/lib/python3.12/dist-packages/servicemaker/triton/triton.py”, line 342, in serialize_to_disk
self.update_binary(version_dir, rmir, verbose)
File “/usr/local/lib/python3.12/dist-packages/servicemaker/triton/asr.py”, line 186, in update_binary
whisper_build(
File “/usr/local/lib/python3.12/dist-packages/servicemaker/triton/trtllm_whisper.py”, line 670, in run_build
generate_engine(build_args)
File “/usr/local/lib/python3.12/dist-packages/servicemaker/triton/trtllm_whisper.py”, line 314, in generate_engine
parallel_build(model_config, ckpt_dir, build_config, args.output_dir, workers, args.log_level, model_cls, **kwargs)
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/build.py”, line 419, in parallel_build
passed = build_and_save(rank, rank % workers, ckpt_dir,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/build.py”, line 384, in build_and_save
engine = build_model(build_config,
^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/commands/build.py”, line 377, in build_model
return build(model, build_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/builder.py”, line 1260, in build
model(**inputs)
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/module.py”, line 52, in call
output = self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/enc_dec/model.py”, line 1985, in forward
hidden_states = encoder_layer(hidden_states,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/module.py”, line 52, in call
output = self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/enc_dec/model.py”, line 246, in forward
attention_output = self.attention(hidden_states,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/module.py”, line 52, in call
output = self.forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/local/lib/python3.12/dist-packages/tensorrt_llm/layers/attention.py”, line 1658, in forward
assert get_sm_version(
^^^^^^^^^^^^^^^
AssertionError: bert_attention_plugin does not support SM >= 100
- ‘[’ 1 -ne 0 ‘]’
- echo ‘Error in deploying RMIR models.’
Error in deploying RMIR models. - exit 1