I am having issues running TensorRT python client on one of our systems. More specifically, TensorRT python client runtime API cannot run on multiple processes. TensorRT python client runtime API is working on a single process.
Here are the methods/work arounds/tests that I have tried,
1.Running the TensorRT runtime API outside of the multi processing first, then running the created runtime object into the multi process. The runtime object then deserialises into a cuda engine. This gives me a runtime error ([TensorRT] ERROR: cudaDeviceProfile.cpp (52) - Cuda Error in generateForCurrent: 3 (initialization error))
2.Works fine on the python console
3.Works fine on the TensorRT python sample code
4.Created TensorRT engine and context outside of multiprocessing process and injected the objects into the process. However, during inference, an error occured, “[TensorRT] ERROR: engine.cpp (370) - Cuda Error in ~ExecutionContext: 3 (initialization error)
terminate called after throwing an instance of ‘nvinfer1::CudaError’
Please assist on this.