My program has two python process.
Main process stream images in batch and subprocess do inference.
Inside subprocess, trying to load TensorRT engine and have error.
The error happened at
with trt.Runtime(TRT_LOGGER) as runtime:
The whole error is
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
self.run()
File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "deployment_saac.py", line 138, in BatchProcessing
with trt.Runtime(TRT_LOGGER) as runtime:
TypeError: pybind11::init(): factory function returned nullptr
My subprocess code is as follow
def BatchProcessing(self, e, q, batch1_, batch2_):
images=np.zeros((BATCHSIZE, CHANNEL*HEIGHT*WIDTH), dtype=np.float32)
if(os.path.exists(ENGINE_DIR)):#check engine path exists
with open(ENGINE_DIR, 'rb') as f, trt.Runtime(TRT_LOGGER) as runtime:#read engine
engine = runtime.deserialize_cuda_engine(f.read())
else:
calibration_files = create_calibration_dataset()
calibration_cache = "saac_calibration.cache"
# Process 5 images at a time for calibration
# This batch size can be different from MaxBatchSize (1 in this example)
batchstream = calibrator.ImageBatchStream(BATCHSIZE, calibration_files, sub_mean_chw)
int8_calibrator = calibrator.PythonEntropyCalibrator(["image"], batchstream, calibration_cache)
engine=build_int8_engine(int8_calibrator)
How to solve the error?