NVIDIA Developer Forums

Capable of running multiple Inference concurrently on the same GPU using TensorRT?

AI & Data Science Deep Learning (Training & Inference) TensorRT

zhangjiamin March 27, 2019, 9:47am 1

Can I run multiple Inference simultaneously on one GPU using TensorRT on the Jetson TX2?

If yes, How to do?

Thanks.

Topic		Replies	Views	Activity
Can I inference two engine simultaneous on jetson using TensorRT? Jetson TX2	4	2251	October 18, 2021
Tensorrt on a machine with 2 GPUs TensorRT	0	1009	June 27, 2019
Is it possible to run multiple TensorRT model inference on a GPU simultaneously and parallelly? TensorRT tensorrt , cuda	3	1808	August 23, 2022
TRT concurrently Jetson TX2 tensorrt	2	391	September 5, 2021
How to run TensorRT on a muliti-GPU platform. TensorRT	2	3389	October 12, 2021
Concurrently run two or more engine on a tensorrt TensorRT	4	2089	April 14, 2020
Inference Server on Jetson TX2 Triton Inference Server - archived	2	1209	August 9, 2019
How to do two different inference with TensorRT on two different GPU on same machine or PC TensorRT	2	450	September 29, 2023
Using Multiple GPUS TensorRT	0	503	August 20, 2019
Set up multiple Jetson TX2 Developers in parallel Jetson TX2	4	752	October 18, 2021