Threading on TX2 using Tensorrt


Hi all,
I wanna to create 4 threadings to use model.engine separately. Such as threading 1 can use model.engine for inference and threading 2 , threading 3, threading 4 can do inference too using python

When I put it in threading 1, the program will shut down.

Traceback (most recent call last):
  File "/usr/lib/python3.6/", line 916, in _bootstrap_inner
  File "", line 124, in run
    h_input, d_input, h_output, d_output = allocate_buffers(engine)
  File "", line 80, in allocate_buffers
    h_input = cuda.pagelocked_empty(trt.volume(engine.get_binding_shape(0)), dtype= trt.nptype(DTYPE))
pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?

If I add these in threading

device = cuda.Device(1)
ctx = device.make_context()

It can do inference .
How can I do this in threading 2, threading 3 and threading4?
Many thanks


TensorRT Version: 7
GPU Type: TX2
CUDA Version: Jetpack4.4
CUDNN Version: Jetpack4.4
Operating System + Version: Ubuntu18
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): tf1.15.0

1 Like

Please refer to below thread safety guidelines: