Dynamic input onnx model fails only with different minShapes, optShapes and maxShapes

Description

I am trying to convert an onnx segmentation model to TensorRT. I want it to have dynamic spatial input sizes so I invoke trtexec as follows:

/usr/src/tensorrt/bin/trtexec \
		--onnx=model.onnx \
		--explicitBatch \
		--saveEngine=model.plan \
		--minShapes=input:1x256x256x3 \
		--optShapes=input:1x512x512x3 \
		--maxShapes=input:1x512x512x3 \
		--buildOnly \
		--workspace=7000 \
		--fp16 \
		--verbose

This fails with

[07/08/2020-15:43:34] [V] [TRT] --------------- Timing Runner: (Unnamed Layer* 168) [ElementWise] (ElementWise)
[07/08/2020-15:43:34] [E] [TRT] ../builder/cudnnBuilderUtils.cpp (360) - Cuda Error in findFastestTactic: 2 (out of memory)
[07/08/2020-15:43:34] [W] [TRT] GPU memory allocation error during getBestTactic: (Unnamed Layer* 168) [ElementWise]
[07/08/2020-15:43:34] [E] [TRT] Internal error: could not find any implementation for node (Unnamed Layer* 168) [ElementWise], try increasing the workspace size with IBuilder::setMaxWorkspaceSize()
[07/08/2020-15:43:34] [E] [TRT] ../builder/tacticOptimizer.cpp (1523) - OutOfMemory Error in computeCosts: 0
[07/08/2020-15:43:34] [E] Engine creation failed
[07/08/2020-15:43:34] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=model.onnx --explicitBatch --saveEngine=model.plan --minShapes=input:1x256x256x3 --optShapes=input:1x512x512x3 --maxShapes=input:1x512x512x3 --buildOnly --workspace=7000 --fp16 --verbose

Note however if I increase minShapes and optShapes to the maxShapes it works:

/usr/src/tensorrt/bin/trtexec \
		--onnx=model.onnx \
		--explicitBatch \
		--saveEngine=model.plan \
		--minShapes=input:1x512x512x3 \
		--optShapes=input:1x512x512x3 \
		--maxShapes=input:1x512x512x3 \
		--buildOnly \
		--workspace=7000 \
		--fp16 \
		--verbose

Above succeeds. Note that I can’t try with higher workspace size due to GPU memory constraints.

Environment

TensorRT Version: libnvinfer-bin/unknown,now 7.0.0-1+cuda10.0
GPU Type: GeForce GTX 1070
Nvidia Driver Version: 440.100
CUDA Version: 10.0
CUDNN Version: 7.6.5.32-1+cuda10.0
Operating System + Version: Ubuntu 18.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Onnx model: [REMOVED]

Hi @copah,
This is a known issue, and fix will be available in future release. Please stay tuned.

Thanks!

@AakankshaS Thank you. Is there a site with the roadmap for TensorRT?

Hi @copah,
We dont have any such page.
For latest TensorRT updates, stay tuned to the TRT official portal.
Thanks!