Hi all, Purpose: So far I need to put the TensorRT in the second threading. I have read this document but I still have no idea how to exactly do TensorRT part on python. I already have a sample which can successfully run on TRT. Now I just want to run a really simple multi-threading code with T…

Hi, The sample is ready. The workflow looks like this: Master: create TensorRT engine and buffer, store the created CUDA context. Thread1: restore master’s CUDA context, run inference, destroy its CUDA context. Master: destroy the stored CUDA context. The sample is created with TensorRT default …

How to use TensorRT by the multi-threading package of python

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier

Chieh May 15, 2020, 9:25am 7

Hi @AastaLLL,

Got it!
Thank you so much!

Running two models in multiple models increases the FPS

Latency when running TensorRT engine on two GPU

Topic		Replies	Views
Unable to run two TensorRT models in a cascade manner TensorRT tensorrt , python	7	4963	October 12, 2021
how to run trt in multithreading？ Jetson TX2	15	7947	October 18, 2021
Using TensorRT in multithreading always generate errors when exiting TensorRT tensorrt	3	3284	October 4, 2021
Can TensorRT do inference in a child thread ? TensorRT	6	2203	August 11, 2020
TensorRT do_inference error TensorRT	19	8385	November 14, 2022
Running Multiple TensorRT Engines On Jetson AGX Jetson AGX Xavier tensorrt , yolo	2	2447	August 29, 2021
[TensorRT] engine happed a error in multithreaded TensorRT tensorrt , cuda	2	1552	January 19, 2023
TensorRT fails to exit properly TensorRT tensorrt , cuda , pycuda	8	2834	October 14, 2021
TensorRT inference context in ROS callback TensorRT tensorrt , cuda	13	2547	January 8, 2023
How to implement TensorRT as an inference server? TensorRT	2	1729	October 24, 2019

How to use TensorRT by the multi-threading package of python

Related topics