[TensorRT] engine with trt file infer the same result

Description

the problem shown as the attached file that when we loop our inference for several times, each image per loop, it always shows the result of the first loop.

Like, these two images are different result[ like OK or NG], but the engine will infer the same [ like NG].
And if reload the engine, the result may get the same different result[ like OK].

Environment

TensorRT Version: 8.2.5.1
GPU Type: RTX5000
Nvidia Driver Version: 515.65
CUDA Version: 11.7
CUDNN Version:
Operating System + Version: ubuntu 22.02
Python Version (if applicable): python3.8

Relevant Files

Steps To Reproduce

The env was [download] by this way

docker run --runtime=nvidia --gpus '"'device=3'"' -it --rm --entrypoint "" nvcr.io/nvidia/tensorrt:22.05-py3 /bin/bash

and the result is

 [I] The logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. Uses of the global logger, returned by nvinfer1::getLogger(), will return the existing value.

Please include:

the trt loger command is

### Load TensorRT engine

trt_logger = trt.Logger(trt.Logger.INFO)
runtime = trt.Runtime(trt_logger)
with open(ENGINE_PATH, "rb") as f:
    engine = runtime.deserialize_cuda_engine(f.read())

And infer is

def infer(img):
    bindings = [int(devide_in), int(devide_out)]
    np.copyto(host_in, img.ravel())
    cuda.memcpy_htod_async(devide_in, host_in, stream)
    context.execute_async(bindings=bindings, stream_handle=stream.handle)
    cuda.memcpy_dtoh_async(host_out, devide_out, stream)
    stream.synchronize()
    return host_out

I try to use init() and pop() to avoid the existing trt value, but it was uselessful.

cuda.init()
device = cuda.Device(0)
ctx = device.make_context()
ctx.pop()

How to avoid the logger return the existing value? It puzzled me for a long time!

Hi,

Sorry for the delayed response, Are you still facing this issue?

Thank you.

YES, still

Hi,

Could you please share with us the more details, minimal issue repro ONNX model and script to try from our end for better debugging.

Thank you.

Hi,

Sorry, the issue has been resolved. The same result was made by the problem of picture normalization.

We tried a lot of normalized way, finally use this: preprocess_input import from keras.applications.imagenet_utils.

Thanks.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.