multithreaded case of yolov3_onnx example

Provide details on the platforms you are using:
OS:Ubuntu 1804
GPU: 1080Ti
Nvidia driver version: 440
CUDA version 10.2
CUDNN version 7.6.5
Python version 2.7.17
TensorRT version 7.0.0.11

Describe the problem


Hello! I am trying to do object detection using tensorrt. I successfully ran the python sample yolov3_onnx. Now I am building up on the sample code onnx_to_tensorrt.py and am trying to do detection on image callback. The engine successfully builds but the issue seems to arise from multithreaded call of detect() function and how I use create_execution context().

Try 1:
Put the create_execution_context() inside detect() function before calling do_inference_v2().
ERROR: …/rtSafe/cuda/caskConvolutionRunner.cpp (290) - Cask Error in checkCaskExecError: 7 (Cask Convolution execution)
Try 2:
Create the execution context only once, after I build the engine.
ERROR: …/rtSafe/safeContext.cpp (133) - Cudnn Error in configure: 7 (CUDNN_STATUS_MAPPING_ERROR)

How should I approach this? Any tips appreciated. Thank you.

Hi,

Please refer to below link for thread safety related guidelines:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-700/tensorrt-best-practices/index.html#thread-safety

Thanks