multithreaded case of yolov3_onnx example

Provide details on the platforms you are using:
OS:Ubuntu 1804
GPU: 1080Ti
Nvidia driver version: 440
CUDA version 10.2
CUDNN version 7.6.5
Python version 2.7.17
TensorRT version

Describe the problem

Hello! I am trying to do object detection using tensorrt. I successfully ran the python sample yolov3_onnx. Now I am building up on the sample code and am trying to do detection on image callback. The engine successfully builds but the issue seems to arise from multithreaded call of detect() function and how I use create_execution context().

Try 1:
Put the create_execution_context() inside detect() function before calling do_inference_v2().
ERROR: …/rtSafe/cuda/caskConvolutionRunner.cpp (290) - Cask Error in checkCaskExecError: 7 (Cask Convolution execution)
Try 2:
Create the execution context only once, after I build the engine.
ERROR: …/rtSafe/safeContext.cpp (133) - Cudnn Error in configure: 7 (CUDNN_STATUS_MAPPING_ERROR)

How should I approach this? Any tips appreciated. Thank you.


Please refer to below link for thread safety related guidelines: