Error 3 Cuda initialization while deserializing TensorRT model

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello,

I encountered an error when running these lines of code in my project:

with open(engine_file_path, "rb") as f, trt.Runtime(trt.Logger()) as runtime:
    engine = runtime.deserialize_cuda_engine(f.read())

[TensorRT] ERROR: Cuda initialization failure with error 3.              Please check cuda installation:             http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html.
 ERROR - This exception occurred while initialing detector pipeline. Stopping
Traceback (most recent call last):
  File ".../yolo_tensorrt/yolo_trt_detector.py", line 15, in __init__
    self.engine = self.load_tensorrt_engine(checkpoint_path, self.TRT_LOGGER)
  File ".../yolo_tensorrt/yolo_trt_detector.py", line 23, in load_tensorrt_engine
    with open(engine_file_path, "rb") as f, trt.Runtime(trt.Logger()) as runtime:
TypeError: pybind11::init(): factory function returned nullptr

I am using this image: nvcr.io/nvidia/tensorrt:19.02-py3 ,
Ubuntu 16.04
Python 3.5
Cuda 10.0.130
Cudnn 7.4.2
Tensorrt 5.0.2

First running /opt/tensorrt/python/python_setup.sh, then running the script.

Also, when only running the code snippet above in the python console, I get no errors:

>>> type(engine)
<class 'tensorrt.tensorrt.ICudaEngine'>

Could you please explain the error code (error 3) and the error itself (pybind11::init(): factory function returned nullptr)?

1 Like
2

I have the same problem,any body solve this problem?

3

i got the same error,could anyone help??

4

I have the same problem.

When I run inference on the main thread of my application it’s fine, but it blocks the main thread which doesn’t work for my use case so I need to run in a separate process or thread.

I have tried multiple implementations to run inference via a separate process. My latest attempt was to run inference using a separate worker task via Python Celery (http://www.celeryproject.org/).

I have also tried to use the Python threading API and the multiprocessing API but ran into the same issues.

with open(engine_file_path, "rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
            return runtime.deserialize_cuda_engine(f.read())

TypeError('pybind11::init(): factory function returned nullptr',)
5

Anyone found the solution yet? I am facing the same issue here. Any hints would be very helpful.

6

I had a similar issue, and solve it by adding a line of code on the main process, before start the subprocesses:

multiprocessing.set_start_method('spawn')

Source: https://stackoverflow.com/a/55812288/8664574

1 Like
7

Hi bro, have you solved this problem ？
I have met the same, there is a wrong error like this :[TensorRT] ERROR: INVALID_CONFIG: Deserialize the cuda engine failed
Here are the version I used. I
CUDNN：8.0.0
CUDA :10.2
Python :3.6.9
platform：NX
:
.

image
image1462×451 292 KB