Running two models in multiple models increases the FPS


I have two tensorrt engines one for segmentation and detection. When I run them individually I get an FPS of 440 and 367 respectively on 2080 Ti. But When I ran the same two models in two separate threads as mentioned in the thread (How to use TensorRT by the multi-threading package of python), I get an FPS of 534 and 429 respectively. Which is like a 20% increase.

I was expecting the FPS to drop because now the resource is shared between multiple threads or at the best would remain at the same FPS. Increased FPS is good, but it is breaking my head without finding the reason


TensorRT Version:
GPU Type: 2080 Tu
Nvidia Driver Version: 450.51.06
CUDA Version: 11.0
CUDNN Version:
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Hi @sksenthilkumar93,
Can you please compare the GPU utilization in both the cases.
You can also use profiler tool get the better idea .