Int8 weight extraction

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Dear all, I want to extract the calibrated INT8 weights for offline analysis. Is there any way to get them?

Environment

TensorRT Version: 7.0.0.11
GPU Type: Titan V
Nvidia Driver Version:440.33
CUDA Version: 10.2
CUDNN Version: 7.5
Operating System + Version: ubuntu18.04 LTS
Python Version (if applicable): 3.6
TensorFlow Version (if applicable): I use pytorch
PyTorch Version (if applicable): 1.10+torchvision 0.3
Baremetal or Container (if container which image + tag): N/A

2

Anyone helps me please. I will wait online. Thanks.

3

Hi,
We can’t extract Int8 weights from TRT model.
But it’s easy to convert FP to Int8 weights, just get max absolute value of weight, map it to 127.

Thanks