On my windows, I can’t make trt inference when import pytorch simultaneously. I made minimal reproducible code as list below.
If i import torch before tensorrt, it will run
print('done'). But if import tensorrt first, it will stuck in the
for name in engine: after print the input & output name and there is no
There is a lot of import tensorrt/torch in py file of the repo, I tried to import torch before every import tensorrt but didn’t help.
Is there a best practice using pytorch and tensorrt in the same time on windows?
What I have tried：
I use psutil to check the loaded dll. If import tensorrt first, there will be on more dll called(C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvrtc64_111_0.dll). Hide it from the PATH didn’t. Maybe the issues is the order of load dll?
My $env:PATH have cudnn and cudatoolkit lib path. And pytorch will also download cudatoolkit and cudnn lib. I noticed the __init__.py of tensorrt uses ctypes.CDLL to load cudnn and some cuda libs. So I tried to remove the cudnn and cudatoolkit path from PATH and add the torch lib path(containg cudnn and cuda libs) to my $env:PATH but didn’t help.
import ctypes import tensorrt as trt import torch # import psutil # import os # p = psutil.Process( os.getpid() ) # for dll in p.memory_maps(): # print(dll.path) # ctypes.CDLL("./build/bin/Release/mmdeploy_tensorrt_ops.dll") with trt.Logger() as logger, trt.Runtime(logger) as runtime: with open("work_dir/trt/model2", mode='rb') as f: engine_bytes = f.read() engine = runtime.deserialize_cuda_engine(engine_bytes) for name in engine: print(name) print('done')
TensorRT Version: 22.214.171.124
GPU Type: GTX2070S
Nvidia Driver Version: 512.59
CUDA Version: 11.1
CUDNN Version: 126.96.36.199
Operating System + Version: windows 10 21H2（19044.1682）
Python Version (if applicable): 3.8.13
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.8.1+cu111
Baremetal or Container (if container which image + tag):
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)