Hi all, Purpose: So far I need to put the TensorRT in the second threading. I have read this document but I still have no idea how to exactly do TensorRT part on python. I already have a sample which can successfully run on TRT. Now I just want to run a really simple multi-threading code with T…

Hi, The sample is ready. The workflow looks like this: Master: create TensorRT engine and buffer, store the created CUDA context. Thread1: restore master’s CUDA context, run inference, destroy its CUDA context. Master: destroy the stored CUDA context. The sample is created with TensorRT default …

How to use TensorRT by the multi-threading package of python

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier

Chieh May 15, 2020, 9:25am 7

Hi @AastaLLL,

Got it!
Thank you so much!

Running two models in multiple models increases the FPS

Latency when running TensorRT engine on two GPU

Topic		Replies	Views
Running Multiple TensorRT Engines On Jetson AGX Jetson AGX Xavier tensorrt , yolo	2	2574	August 29, 2021
[TensorRT] engine happed a error in multithreaded TensorRT tensorrt , cuda	2	1638	January 19, 2023
How to implement TensorRT as an inference server? TensorRT	2	1789	October 24, 2019
Multi thread Inference tensorrt python TensorRT	1	691	February 19, 2021
Threading on TX2 using Tensorrt TensorRT	1	540	May 20, 2020
Can multiple CUDA contexts share an inference engine? TensorRT tensorrt , cuda	3	207	January 21, 2025
Using TensorRT in multithreading always generate errors when exiting TensorRT tensorrt	3	3402	October 4, 2021
TensorRT multi stream TensorRT	3	2915	February 29, 2024
TensorRT do_inference error TensorRT	19	8600	November 14, 2022
Why tensorrt got stuck when using ThreadPool in Python TensorRT	1	718	July 28, 2023

How to use TensorRT by the multi-threading package of python

Related topics