Recently, i run a trt in single thread cost 6ms, then run the same trt in 2 threads cost 10ms, it looks like the threads nteract with each other, but i put the trt in 2 processes and run at the same time ,it’s ok , cost 6ms. so i want to konw why and what can i do if i need to run trt some times at …

Hi, Asynchronous memory copy run well on x86 machine. Usually, concurrency mechanism requires executing with multiple CUDA stream. Have you launched your TensorRT context with independent CUDA stream? Here is a tutorial for your reference: StreamsAndConcurrencyWebinar.pdf 934.01 KB …

NVIDIA Developer Forums

how to run trt in multithreading？

Autonomous Machines Jetson & Embedded Systems Jetson TX2

380483397 April 26, 2018, 2:13pm 9

i run my program on x86 machine,does the problem still exist?

Latency when running TensorRT engine on two GPU