Description
Good afternoon, I am learning how to work with neural networks using C++ and TensorRT. When it comes to a single neural network, everything works fine and quickly. However, I have two questions. Is it possible to use the same neural network in different threads without having to load the engine and context each time while maintaining the ability for parallel operation (not asynchronous)? Also, is it possible to upload multiple images into the neural network without arranging them into a long sequence, with each one following the previous one? In other words, could I create a list of conditional images and send all of them at once to TensorRT instead of transferring one image at a time?
Environment
TensorRT Version: 10.0.0.6
GPU Type: RTX 3080
Nvidia Driver Version: 535
CUDA Version: 12.4
CUDNN Version: 9.0.0.312
Operating System + Version: Ubuntu 20.04