NVIDIA Developer Forums

Can I use both variable(dynamic) input shape and batching in TensorRT?

AI & Data Science Deep Learning (Training & Inference) TensorRT

tb17074107 October 9, 2024, 8:24am 1

Description

Can I use both variable(dynamic) input shape and batching? And what’s the strategy to padding input when batching?

Environment

TensorRT version: lastest

Topic		Replies	Views	Activity
Does TensorRT support the model has dynamic inputshape and outputshape? TensorRT	2	822	October 12, 2021
How to use dynamic shapes in TensorRT Triton Inference Server - archived	0	1058	July 12, 2021
Can i use two dynamic shapes? TensorRT tensorrt , cuda , c-plus-plus	1	539	March 28, 2023
How to enable dynamic input shapes in TF-TRT Frameworks tensorrt , tensorflow	0	548	May 25, 2020
Tensorrt with dynamic input GPU-Accelerated Libraries tensorrt	0	379	June 8, 2020
Using dynamic shape with space_to_depth implementation TensorRT tensorrt	2	477	October 12, 2021
2 contexts use same dynamic shape profile TensorRT	2	367	October 12, 2021
TensorRT 5.1 support dynamic input size? TensorRT	2	1234	October 12, 2021
Does tensor-rt support input with (seqence, batch, embedding) format TensorRT	1	325	March 9, 2021
Input batch size is smaller than TensorRT engine batch size TensorRT	1	1011	March 28, 2022