Running two or more docker GPU using docker containers concurrently

[AastaLLL]

Hi,

Which deep inference library do you use?

For TensorRT, please check a sample below:
There are some CUDA context handling that might be helpful for your issue.

How to use TensorRT by the multi-threading package of python

Hi, The sample is ready. The workflow looks like this: Master: create TensorRT engine and buffer, store the created CUDA context. Thread1: restore master’s CUDA context, run inference, destroy its CUDA context. Master: destroy the stored CUDA context. The sample is created with TensorRT default MNIST model. 1. Prepare data $ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --saveEngine=mnist.trt $ cd /usr/src/tensorrt/data/mnist/ $ sudo pip3 install pillow $ pyt…

Thanks.