Workarounds about TensorRT INT64 datatype

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

In the NLP tasks, there are lots of INT64 ID tensor as input, but currently tensorrt doesn’t support the INT64 datatype. Is there any workarounds or tips on it?

Thanks.

Environment

TensorRT Version: 8.0.1.6
GPU Type: T4
Nvidia Driver Version: 460.32.03
CUDA Version: 11.2
CUDNN Version: 8.0.5.39
Operating System + Version: Ubuntu 18.04.5 LTS

5

Hi,

TensorRT will attempt to cast down INT64 to INT32. For your reference,

Thank you.