# Tensorrt inference with multi thread and multi stream

**URL:** <https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423>\
**Category:** Jetson AGX Xavier\
**Tags:** tensorrt\
**Created:** [July 21, 2022, 4:53am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423 "2022-07-21T04:53:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wilbur](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/wilbur/32/215527_2.png) [@Wilbur](https://forums.developer.nvidia.com/u/Wilbur)\
**Post date:** [July 21, 2022, 4:53am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423/1 "2022-07-21T04:53:07Z")

</div>

hi, Nvidia’s friends, I use multi-thread to make inferences in parallel; it is mentioned here([Developer Guide :: NVIDIA Deep Learning TensorRT Documentation](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#streaming)) that each thread has an IExecutionContext and a Cuda Stream. Is the IExecutionContext in each thread mentioned here created by the user explicitly or will each thread automatically generate an IExecutionContext?

---

<div class="post-metadata">

**Author:** ![hitbuyi](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hitbuyi](https://forums.developer.nvidia.com/u/hitbuyi)\
**Post date:** [July 22, 2022, 1:44am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423/4 "2022-07-22T01:44:46Z")

</div>

Does Nvidia provide this kind of examples?

---

<div class="post-metadata">

**Author:** ![kayccc](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/kayccc/32/14043_2.png) [@kayccc](https://forums.developer.nvidia.com/u/kayccc)\
**Post date:** [July 25, 2022, 3:54am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423/5 "2022-07-25T03:54:44Z")

</div>

Your topic was posted in the wrong category. I am moving this to the Jetson AGX category for visibility.

Which Jetson platform and JetPack you’re using?

---

<div class="post-metadata">

**Author:** ![AastaLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/aastalll/32/14043_2.png) [@AastaLLL](https://forums.developer.nvidia.com/u/AastaLLL)\
**Post date:** [July 25, 2022, 3:59am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423/7 "2022-07-25T03:59:37Z")

</div>

Hi,

Is our _trtexec_ sample can meet your requirement?

> **[TensorRT/samples/trtexec at main · NVIDIA/TensorRT](https://github.com/NVIDIA/TensorRT/tree/main/samples/trtexec)**
>
> NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. - NVIDIA/TensorRT

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [August 17, 2022, 3:10am UTC](https://forums.developer.nvidia.com/t/tensorrt-inference-with-multi-thread-and-multi-stream/221423/9 "2022-08-17T03:10:14Z")

</div>

Closed on August 17, 2022, 3:10am UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
