# \[TensorRT\] engine happed a error in multithreaded

**URL:** <https://forums.developer.nvidia.com/t/tensorrt-engine-happed-a-error-in-multithreaded/239708>\
**Category:** TensorRT\
**Tags:** tensorrt, cuda\
**Created:** [January 16, 2023, 6:08am UTC](https://forums.developer.nvidia.com/t/tensorrt-engine-happed-a-error-in-multithreaded/239708 "2023-01-16T06:08:45Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![devin.he](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@devin.he](https://forums.developer.nvidia.com/u/devin.he)\
**Post date:** [January 19, 2023, 6:58am UTC](https://forums.developer.nvidia.com/t/tensorrt-engine-happed-a-error-in-multithreaded/239708/3 "2023-01-19T06:58:00Z")

</div>

Tks！

I found the TensorRT is [Thread Safety](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-803/best-practices/index.html#thread-safety)！

```auto
The TensorRT builder may only be used by one thread at a time. If you need to run multiple builds simultaneously, you will need to create multiple builders.
The TensorRT runtime can be used by multiple threads simultaneously, so long as each object uses a different execution context.
Note: Plugins are shared at the engine level, not the execution context level, and thus plugins which may be used simultaneously by multiple threads need to manage their resources in a thread-safe manner. This is however not required for plugins based on IPluginV2Ext and derivative interfaces since we clone these plugins when ExecutionContext is created.
The TensorRT library pointer to the logger is a singleton within the library. If using multiple builder or runtime objects, use the same logger, and ensure that it is thread-safe.

```

So， I change the sever in **single thread** way and it works well.

And refer

> [@How to use TensorRT by the multi-threading package of python](https://forums.developer.nvidia.com/t/how-to-use-tensorrt-by-the-multi-threading-package-of-python/123085/8):
>
> Hi, The sample is ready. The workflow looks like this: Master: create TensorRT engine and buffer, store the created CUDA context. Thread1: restore master’s CUDA context, run inference, destroy its CUDA context. Master: destroy the stored CUDA context. The sample is created with TensorRT default MNIST model. 1. Prepare data $ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --saveEngine=mnist.trt $ cd /usr/src/tensorrt/data/mnist/ $ sudo pip3 install pillow $ pyt…

But when I try to use it in [multithreaded](https://forums.developer.nvidia.com/t/tensorrt-engine-happed-a-error-in-multithreaded/239708) way, it asks App to use the different GPU mem.

It didn’t work and send the same result.

```auto
./receive230116022455/ac_receive.jpg
[01/16/2023-06:04:08] [TRT] [E] 1: [convolutionRunner.cpp::execute::391] Error Code 1: Cask (Cask convolution execution)
[01/16/2023-06:04:08] [TRT] [E] 1: [checkMacros.cpp::catchCudaError::272] Error Code 1: Cuda Runtime (invalid resource handle)
['./receive230116022455/ac_receive.jpg', array([0., 0., 0., 0.], dtype=float32), 'NG1']

```

---

_[View the full topic](https://forums.developer.nvidia.com/t/tensorrt-engine-happed-a-error-in-multithreaded/239708)._