Description
On my windows, I can’t make trt inference when import pytorch simultaneously. I made minimal reproducible code as list below.
If i import torch before tensorrt, it will run print('done'). But if import tensorrt first, it will stuck in the for name in engine: after print the input & output name and there is no print('done').
There is a lot of import tensorrt/torch in py file of the repo, I tried to import torch before every import tensorrt but didn’t help.
Is there a best practice using pytorch and tensorrt in the same time on windows?
What I have tried：
I use psutil to check the loaded dll. If import tensorrt first, there will be on more dll called(C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvrtc64_111_0.dll). Hide it from the PATH didn’t. Maybe the issues is the order of load dll?
My $env:PATH have cudnn and cudatoolkit lib path. And pytorch will also download cudatoolkit and cudnn lib. I noticed the __init__.py of tensorrt uses ctypes.CDLL to load cudnn and some cuda libs. So I tried to remove the cudnn and cudatoolkit path from PATH and add the torch lib path(containg cudnn and cuda libs) to my $env:PATH but didn’t help.
Some relevent issues but no answer :
issue1693
issue480
script
import ctypes
import tensorrt as trt
import torch
# import psutil
# import os
# p = psutil.Process( os.getpid() )
# for dll in p.memory_maps():
# print(dll.path)
# ctypes.CDLL("./build/bin/Release/mmdeploy_tensorrt_ops.dll")
with trt.Logger() as logger, trt.Runtime(logger) as runtime:
with open("work_dir/trt/model2", mode='rb') as f:
engine_bytes = f.read()
engine = runtime.deserialize_cuda_engine(engine_bytes)
for name in engine:
print(name)
print('done')
Environment
TensorRT Version: 8.2.3.0
GPU Type: GTX2070S
Nvidia Driver Version: 512.59
CUDA Version: 11.1
CUDNN Version: 8.2.1.0
Operating System + Version: windows 10 21H2（19044.1682）
Python Version (if applicable): 3.8.13
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.8.1+cu111
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)