TensorRT‑LLM fails on RTX PRO 2000 Blackwell with CUDA 13.2 (engine build + Python API + Triton backend)

AI & Data Science TensorRT for RTX
,
1

Environment Overview

  • GPU: NVIDIA RTX PRO 2000 Blackwell

  • CUDA: 13.2 (required by driver 596.36)

  • cuDNN: 9.x

  • TensorRT: 10.x

  • TensorRT‑LLM: latest public release

  • OS: Windows 11 + WSL2 Ubuntu 22.04

  • Python: 3.10 / 3.11

  • Triton Inference Server: open-source build

We attempted multiple TensorRT‑LLM installation and execution paths (Python API, C++ build, engine generation, Triton backend). All attempts failed.

1. Python API Import Errors

  • ModuleNotFoundError: No module named 'tensorrt_llm.bindings'

  • ImportError: libnvinfer.so: cannot open shared object file

  • ImportError: cannot import name 'Mapping' from 'collections'

Indicates missing or incompatible TensorRT‑LLM bindings and ABI mismatch.

2. Engine Builder Failures

  • FP8/BF16 kernel generation crashes

  • Unsupported kernel errors for Blackwell

  • RuntimeError: Failed to build TensorRT engine

Suggests incomplete kernel support for Blackwell architecture.

3. CUDA / cuDNN / TensorRT Compatibility Issues

  • CUDA 13.2 not recognized by TensorRT‑LLM

  • cuDNN 9.x kernel mismatch

  • TensorRT 10.x shared library conflicts

Likely version matrix misalignment for Blackwell GPUs.

4. Triton Backend Failures

  • Backend unable to load TensorRT‑LLM engine

  • Model repository validation failures

  • Runtime crashes during warmup

Indicates backend instability with TensorRT‑LLM on RTX PRO 2000.

Request to NVIDIA Engineering

We kindly request clarification on:

  1. Is TensorRT‑LLM currently compatible with CUDA 13.x?

  2. Is TensorRT‑LLM fully supported on RTX PRO 2000 Blackwell?

  3. Which versions of CUDA, cuDNN, TensorRT, and TensorRT‑LLM are required?

  4. Are there known limitations or pending updates for Blackwell support?

  5. Is there an internal build or patch available for TensorRT‑LLM on Blackwell GPUs?

We can provide full logs and reproduction steps if needed.

Thank you,
Vidmantas

2

TRT-LLM is not supported on Windows

Refer to the installation guide: Installing on Linux via pip — TensorRT LLM