RTX 5090 + Tensorrt + SDXL / Image Generation : Impossible ? Comfy-UI (Runpod)

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

For an image generator (SDXL)
I am experiencing issues while building and running a TensorRT engine on a NVIDIA RTX 5090 GPU with CUDA 12.8.
The conversion of models (e.g., PyTorch to ONNX) and subsequent inference execution fails due to compatibility issues between the latest CUDA version and TensorRT.
This causes errors such as model export failures, dynamic shape issues, and performance degradation.
Etc.
GPT-Research / GPT-Pro failed to help, suggesting hacks and fixed I consider as ‘daring’ and ‘wild’, ending up generating ‘broken’ images with tensorrt.

The official Comfy-UI tensorrt nodes are not working either.

Environment

TensorRT Version: cu12_bindings-10.14.1.48.post1-cp312-none-manylinux_2_28_x86_64.whl
GPU Type: NVIDIA RTX 5090
Nvidia Driver Version: 570.195.03
CUDA Version: 12.8
CUDNN Version: cudnn-cu12==9.10.2.21
Operating System + Version: Ubuntu / Linux on Runpod
Python Version: python3.12

PyTorch Version: 2.10.0.dev20250924+cu128
Baremetal or Container (if container which image + tag): (hearmeman/comfyui-wan-template:v10) on Runpod

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Step 1:
Buy RTX5090 / Install Cuda 12.8, + setup Comfy ui (or use runpod hearmeman/comfyui-wan-template:v10)
Step 2:
Install and run official Tensorrt from Comfy-UI Manager.
Step 3:
You will see Error.

4

Hi @rick.coelkusu,

While your current configuration should theoretically work, I recommend trying the following setup which worked for me on RTX Blackwell:

Driver Version: 590.44.xx (or newer)
CUDA Version: 13.1
Kernel Modules: nvidia-open (NOT proprietary)

For Ubuntu:

sudo apt purge '*nvidia*' -y
sudo apt autoremove -y
sudo apt update
sudo apt install nvidia-driver-590-open -y
sudo reboot

For Blackwell GPUs, I recommend using 580+ drivers for better compatibility. While CUDA 12.8 supports Blackwell, the 580+ drivers appear to offer the best compatibility with this architecture.

Let me know if this works for you.

Thank you.

5

Hey man, thanks for the resonse!

I can not use Cuda13.1 in runpod.
Currently the highest is Cuda 12.9.

And after 4 days of trial and error, I start to feel my sanity slipping haha.
I just feel lost / out of options. And it seems like no one is experienced in this. I even offered money / searched in upwork / in different discord servers.

I feel like almost no one has expertise in this area. And it’s combined with the fact, that the official comfy-ui ‘tensorrt’ node simply does not work.

So I’m kinda leaning towards “giving up”.
By any chance, do you have an RTX 5090?
If yes:
Is there any chance I can just send you my sdxl model, and you send me back the .engine?
^^