The memory usage of tensorrt algorithm model is different on different hardware!

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Hi！
I used Tensorrt’s python api to load the swin-tiny segmentation model on different hardware, and found that the memory size occupied by the model on the host side was different. What is the reason for this

exp1
exp11582×607 20.7 KB

My guess here is that the 3080ti has stronger computing power, the corresponding data throughput is higher, and the memory usage of the model is higher. I wonder if this guess is correct?

Looking forward to your reply, good luck!

3

Hi,

We don’t have a good explanation about the host memory requirement difference.
Could you please set the workspace size (IBuilderConfig — NVIDIA TensorRT Standard Python API Documentation 8.5.1 documentation) and see if it changes anything.
By default, TRT uses as much workspace as it needs.