I wanted to give a quick try to TensorRT and ran into the following errors when building the engine from an UFF graph
[TensorRT] ERROR: Tensor: Conv_0/Conv2D at max batch size of 80 exceeds the maximum element count of 2147483647.
To solve this problem I had to reduce the builder
max_batch_sizeparameter to 50 or so. Note that this is much less than the maximum batch size I am able to run using Tensorflow (around 200 before encountering OutOfMemory error). Why is that so?
(the convolution which the errors is referring to is a 3x3x1x64 convolution on patches of size 100x100)
[TensorRT] ERROR: runtime.cpp (24) - Cuda Error in allocate: 2
I have had this error several times and absolutely no clue on what was causing it. One way of getting around was to reduce the
max_workspace_sizeparameter of the builder to let’s say a third of the total GPU memory (5Gb on a P100 with 16GB).
All in all I am not sure that I fully grasped what is behind these max_batch_size and max_workspace_size parameters. Any hints would be greatly appreciated.
Edit: using TRT 18.104.22.168 with Cuda 9.0 and CUDNN 7.3