Imagine the task of calculating embeddings of the found faces in the video frames.
I have a CNN that calculates this embeddings and I want to calculate them in batches, the size of which can vary depending on the number of persons in the frame.
If i create network with kEXPLICIT_BATCH then i need to call setBindingDimensions on execution context every time when the number of faces changes, BUT this leads to a significant slowdown in the next call of enqueueV2.
OR I can always call enqueueV2 with maximum batch size, BUT it is also extremely suboptimal.
Could you give me some advice?
TensorRT Version: 7.
GPU Type: 1070
Nvidia Driver Version: 430.64
CUDA Version: 10.0
CUDNN Version: 7.6.3
Operating System + Version: Linux Manjaro