Description
the problem shown as the attached file that when we loop our inference for several times, each image per loop, it always shows the result of the first loop.
Like, these two images are different result[ like OK or NG], but the engine will infer the same [ like NG].
And if reload the engine, the result may get the same different result[ like OK].
Environment
TensorRT Version: 8.2.5.1
GPU Type: RTX5000
Nvidia Driver Version: 515.65
CUDA Version: 11.7
CUDNN Version:
Operating System + Version: ubuntu 22.02
Python Version (if applicable): python3.8
Relevant Files
Steps To Reproduce
The env was [download] by this way
docker run --runtime=nvidia --gpus '"'device=3'"' -it --rm --entrypoint "" nvcr.io/nvidia/tensorrt:22.05-py3 /bin/bash
and the result is
[I] The logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value.
Please include:
the trt loger command is
### Load TensorRT engine
trt_logger = trt.Logger(trt.Logger.INFO)
runtime = trt.Runtime(trt_logger)
with open(ENGINE_PATH, "rb") as f:
engine = runtime.deserialize_cuda_engine(f.read())
And infer is
def infer(img):
bindings = [int(devide_in), int(devide_out)]
np.copyto(host_in, img.ravel())
cuda.memcpy_htod_async(devide_in, host_in, stream)
context.execute_async(bindings=bindings, stream_handle=stream.handle)
cuda.memcpy_dtoh_async(host_out, devide_out, stream)
stream.synchronize()
return host_out
I try to use init() and pop() to avoid the existing trt value, but it was uselessful.
cuda.init()
device = cuda.Device(0)
ctx = device.make_context()
ctx.pop()
How to avoid the logger return the existing value? It puzzled me for a long time!