Description Hi I am new to TensorRT and I am trying to build a trt engine with dynamic batch size. I already have an onnx model with input shape of -1x299x299x3, but when I was trying to convert onnx to trt with following command: trtexec --onnx=model_Dense201_BM_FP32_Flex.onnx --saveEngine=model…

Hi, thanks for the reply! Sorry I didn’t make my question clear, what I am asking is how to generate a trt engine that accepts dynamic batch inputs when inferencing with enqueueV2, the C++ API instead of how to run an onnx model with trtexec.

Please refer to the following sample which demonstrates how to use dynamic input dimensions in TensorRT. TensorRT/samples/sampleDynamicReshape at main · NVIDIA/TensorRT NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the o…

Dynamic batch size

AI & Data Science Deep Learning (Training & Inference) TensorRT

spolisetty January 23, 2023, 5:25pm 2

Hi,

We recommend that you use the most recent TensorRT version 8.5.2.

In case you would like to build the TRT engine with dynamic shapes using the trexec tool, please refer to the following:

Also, please refer to the following document (the developer guide) for more details.

Thank you.

Topic		Replies	Views
Trtexec and dynamic batch size TensorRT	4	5653	July 22, 2021
Creating a TensorRT Engine with different batch sizes TensorRT python , onnx	12	2978	August 18, 2020
How to support dynamic batch size for TensorRT engine? TensorRT	1	1187	March 3, 2023
ONNX to TensorRT with dynamic batch size in Python TensorRT tensorrt , onnx	4	6400	October 12, 2021
Trtexec create engine failed from onnx when adding dynamic shapes TensorRT	5	2278	June 22, 2021
TensorRT warnings : Dynamic dimensions required for input: input, but no shapes were provided TensorRT	2	1755	October 12, 2021
Dynamic batch size for tensorrt Engine TensorRT tensorrt	1	1901	May 30, 2024
ONNX to TensorRT Python module doesn't generate dynamic batch size engine TensorRT tensorrt , cudnn , onnx	3	1157	October 20, 2023
Tensorrt, convert pytorch onnx module dynamic batch failed General Topics and Other SDKs tensorrt , ubuntu	0	642	February 11, 2022
Create TesorRT with dynamic batch TensorRT	8	2318	September 8, 2020

Dynamic batch size

Related topics