How to run multiple neural networks in parallel to classify images

[AastaLLL]

Hi,

You can create multiple threads or multiple process for the different models.
A sample for TensorRT with python mulit-threading can be found here:

How to use TensorRT by the multi-threading package of python

Hi all, Purpose: So far I need to put the TensorRT in the second threading. I have read this document but I still have no idea how to exactly do TensorRT part on python. I already have a sample which can successfully run on TRT. Now I just want to run a really simple multi-threading code with TensorRT. (I have done to generate the TensorRT engine, so I will load an engine and do TensorRT inference by multi-threading.) Here is my code below. (Without the Tensorrt code) import threading im…

Thanks.