My program has two python process.
Main process stream images in batch and subprocess do inference.
Inside subprocess, trying to load TensorRT engine and have error.
The error happened at
with trt.Runtime(TRT_LOGGER) as runtime:
The whole error is
Traceback (most recent call last): File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap self.run() File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "deployment_saac.py", line 138, in BatchProcessing with trt.Runtime(TRT_LOGGER) as runtime: TypeError: pybind11::init(): factory function returned nullptr
My subprocess code is as follow
def BatchProcessing(self, e, q, batch1_, batch2_): images=np.zeros((BATCHSIZE, CHANNEL*HEIGHT*WIDTH), dtype=np.float32) if(os.path.exists(ENGINE_DIR)):#check engine path exists with open(ENGINE_DIR, 'rb') as f, trt.Runtime(TRT_LOGGER) as runtime:#read engine engine = runtime.deserialize_cuda_engine(f.read()) else: calibration_files = create_calibration_dataset() calibration_cache = "saac_calibration.cache" # Process 5 images at a time for calibration # This batch size can be different from MaxBatchSize (1 in this example) batchstream = calibrator.ImageBatchStream(BATCHSIZE, calibration_files, sub_mean_chw) int8_calibrator = calibrator.PythonEntropyCalibrator(["image"], batchstream, calibration_cache) engine=build_int8_engine(int8_calibrator)
How to solve the error?