Which cuda toolkit, cudnn, tensort version?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi guys, i have been searching for days and i still cannot figure out which cuda toolkit should i install for my gtx 980 ti gpu. i want to use the tensorRT but it keeps giving me errors. so i am not sure which cuda toolkit, cudnn, onnxruntime and tensorRT version i should use? how do you find out which gpu supports tensorRT?

TensorRT in google search shows it supports sm 7.5 and above. mine seems to have a smm 22? so should that be able to run tensorRT?

i am using :
windows 10
gtx 980 ti gpu
16gb ram
cuda 11.8
cudnn-windows-x86_64-8.6.0.163_cuda11-archive
ONNX Runtime onnx-1.15.0 onnxruntime-gpu-1.17.1
TensorRT-8.5.3.1

This is the error:
[ WARN:0@129.080] global loadsave.cpp:241 cv::findDecoder imread_(‘D:/next/Rope-development/Rope-development/face\2025-03-23 00_36_00-Scarlett-Johansson-Stills-from-Oscars-2020-Red-Carpet-.jpg (773├ù1159).png’): can’t open/read file: check file path/integrity
Bad file D:/next/Rope-development/Rope-development/face\2025-03-23 00_36_00-Scarlett-Johansson-Stills-from-Oscars-2020-Red-Carpet-.jpg (773×1159).png
[ WARN:0@129.529] global loadsave.cpp:241 cv::findDecoder imread_(‘D:/next/Rope-development/Rope-development/face\Esha_Gupta_snapped_on_sets_of_High_FeverΓǪ_Dance_Ka_Naya_Tevar_(04).jpg’): can’t open/read file: check file path/integrity
Bad file D:/next/Rope-development/Rope-development/face\Esha_Gupta_snapped_on_sets_of_High_Fever…Dance_Ka_Naya_Tevar(04).jpg
Invalid SOS parameters for sequential JPEG
Exception in Tkinter callback
Traceback (most recent call last):
File “C:\Users\Godspeed\AppData\Local\Programs\Python\Python310\lib\tkinter_init_.py”, line 1921, in call
return self.func(*args)
File “C:\Users\Godspeed\AppData\Local\Programs\Python\Python310\lib\tkinter_init_.py”, line 839, in callit
func(*args)
File “D:\next\Rope-development\Rope-development\rope\Coordinator.py”, line 58, in coordinator
vm.get_requested_video_frame(action[0][1], marker=True)
File “D:\next\Rope-development\Rope-development\rope\VideoManager.py”, line 312, in get_requested_video_frame
temp = [self.swap_video(target_image, self.current_frame, marker), self.current_frame] # temp = RGB
File “D:\next\Rope-development\Rope-development\rope\VideoManager.py”, line 948, in swap_video
img = self.func_w_test(“swap_video”, self.swap_core, img, fface[0], fface[1], s_e, fface[2], found_face.get(‘DFLModel’, False), parameters, control)
File “D:\next\Rope-development\Rope-development\rope\VideoManager.py”, line 1038, in func_w_test
result = func(*args, **argsv)
File “D:\next\Rope-development\Rope-development\rope\VideoManager.py”, line 1187, in swap_core
self.models.run_swapper(input_face_disc, latent, swapper_output)
File “D:\next\Rope-development\Rope-development\rope\Models.py”, line 449, in run_swapper
self.swapper_model = onnxruntime.InferenceSession( “./models/inswapper_128.fp16.onnx”, providers=self.providers)
File “D:\next\Rope-development\Rope-development\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py”, line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File “D:\next\Rope-development\Rope-development\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py”, line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : TensorRT EP failed to create engine from network for fused node: TensorrtExecutionProvider_TRTKernel_graph_torch_jit_5965111383520720122_0_0

3

Hi @sivadarshiny2005 ,
Please consider the following pointers

  1. CUDA Toolkit, cuDNN, and TensorRT Compatibility:
  • The GTX 980 Ti has a CUDA compute capability of 6.1. TensorRT requires a GPU with a minimum compute capability of 6.0. Therefore, your GTX 980 Ti is technically compatible with TensorRT.
  • For proper operation, ensure you are using:
    • CUDA Toolkit: 11.2 or higher is generally compatible with TensorRT 8.5.
    • cuDNN: Ensure the cuDNN version matches the CUDA Toolkit version (you are using cuDNN 8.6.0 which should be compatible with CUDA 11.8).
    • ONNX Runtime: Make sure that the ONNX Runtime version is compatible with both CUDA and TensorRT.
  1. TensorRT Compatibility and Error Resolution:
  • The error messages you are encountering are indicative of compatibility issues with the ONNX model and TensorRT. Here are some potential solutions:
    • Using Polygraphy: Run TensorRT’s Polygraphy tool to sanitize and fold constants in your ONNX model:
polygraphy surgeon sanitize model.onnx --fold-constants --output model_folded.onnx

* **Plugin Errors**: For custom operators, ensure that necessary plugins are registered. You may consider implementing custom plugins if TensorRT does not natively support certain ONNX operators.
  1. File Access Issues:
  • It seems you’re also facing issues related to image file access, as indicated by the “can’t open/read file” warnings. Make sure that the specified file paths are valid, the files exist, and there are no permission issues.
    Hope this helps.