How to convert UNet model by tensorrt INT8

Description

Could I know how to convert UNet model as tensorrt INT8 on windows?

Environment

TensorRT Version: 8.6.1
GPU Type: RTX A5000
Nvidia Driver Version: 531.14
CUDA Version: 11.6
CUDNN Version:
Operating System + Version: windows10 enterprise
Python Version (if applicable): 3.8
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.13.1
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Hi @a30582733 ,
To convert a UNet model to TensorRT with INT8 precision on Windows in your specified environment (RTX A5000, Windows 10, Python 3.8, PyTorch 1.13.1), you can follow these general steps:

Step-by-Step Guide

  1. Install Necessary Tools and Libraries:

    • Make sure that you have the following installed:
      • NVIDIA CUDA Toolkit (compatible version with your GPU)
      • cuDNN
      • TensorRT
      • PyTorch (1.13.1 in your case)
      • ONNX (for converting the PyTorch model)
      • ONNX Runtime (optional for testing the ONNX model)

    You can install ONNX and other necessary libraries using pip:

    pip install onnx onnxruntime
    
  2. Prepare Your UNet Model:

    • Ensure your UNet model is implemented and trained using PyTorch. For example:
    import torch
    from torchvision import models  # If using any pre-trained networks
    
    class UNet(torch.nn.Module):
        # Define your UNet architecture here
        pass
    
    model = UNet()
    model.load_state_dict(torch.load("path_to_trained_model.pth"))
    model.eval()
    
  3. Convert the UNet Model to ONNX Format:

    • Use PyTorch to export the model to ONNX format. You will need to provide a dummy input tensor with the appropriate shape.
    dummy_input = torch.randn(1, 3, 256, 256)  # Example input shape for UNet
    torch.onnx.export(model, dummy_input, "unet_model.onnx", opset_version=11)
    
  4. Optimize the ONNX Model for TensorRT:

    • Use TensorRT to convert the ONNX model to a TensorRT engine with INT8 precision.
    • You can use the trtexec command-line tool provided by TensorRT:
    trtexec --onnx=unet_model.onnx --saveEngine=unet_model.trt --int8 --shapes=input:1x3x256x256
    

    Note: Ensure you have a calibration dataset for INT8 quantization. You might need to create custom calibration scripts if the default options do not suffice.

  5. Load and Run the TensorRT Engine:

    • After generating the TensorRT engine, you can load it in your application and run inference. Here’s an example of how to do that in Python:
    import pycuda.driver as cuda
    import pycuda.autoinit
    import numpy as np
    import tensorrt as trt
    
    TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
    with open("unet_model.trt", "rb") as f:
        engine = trt.Runtime(TRT_LOGGER).deserialize_cuda_engine(f.read())
    
    context = engine.create_execution_context()
    input_data = np.random.rand(1, 3, 256, 256).astype(np.float32)  # Dummy data
    d_input = cuda.mem_alloc(input_data.nbytes)
    cuda.memcpy_htod(d_input, input_data)
    
    d_output = cuda.mem_alloc(engine.get_binding_size(1))  # Assuming single output
    context.execute(bindings=[int(d_input), int(d_output)])
    
    output_data = np.empty([1, num_classes, 256, 256], dtype=np.float32)  # Replace num_classes
    cuda.memcpy_dtoh(output_data, d_output)
    
  6. Testing and Validation:

    • Validate the performance and accuracy of the INT8 model compared to the original model.

Additional Notes:

  • Calibration for INT8: INT8 precision requires calibration to minimize accuracy loss. Make sure to gather and preprocess a representative dataset for calibration.
  • Error Handling: Implement proper error handling for file I/O and CUDA operations to avoid runtime issues.
  • Documentation: Refer to the official NVIDIA TensorRT documentation for detailed options and configurations related to the use of ONNX and INT8 optimization.

By following the steps outlined, you should be able to convert your UNet model to TensorRT with INT8 precision on Windows. If you encounter specific errors during this process, providing a traceback or error message can help in diagnosing the problem.

Thanks for the reply.

Could you please let me know more detail about INT8 calibration?
or, any example code?

There’s no detail information regarding INT8 calibration on NVIDIA documentation.
It’s very difficult to refer these documents… only saying need calibration, only saying refer to documents…
where are those?

My code is as below. I got a help from Deploying Quantization Aware Trained models in INT8 using Torch-TensorRT — Torch-TensorRT v1.3.0 documentation. However, it doesn’t work. Please don’t say refer to forum…

import os
import torch
import torch_tensorrt
from torch_tensorrt import Input
from cc_weld_dataset import cc_weld_dataset
from unet_origin import UNet
from torch.utils.data import DataLoader
from tqdm import tqdm
from pytorch_quantization import nn as quant_nn
from pytorch_quantization import calib


def compute_amax(model, **kwargs):
    # Load calib result
    for name, module in model.named_modules():
        if isinstance(module, quant_nn.TensorQuantizer):
            if module._calibrator is not None:
                if isinstance(module._calibrator, calib.MaxCalibrator):
                    module.load_calib_amax()
                else:
                    module.load_calib_amax(**kwargs)
            print(F"{name:40}: {module}")
    model.cuda()

def collect_stats(model, data_loader, num_batches):
    """Feed data to the network and collect statistics"""
    # Enable calibrators
    for name, module in model.named_modules():
        if isinstance(module, quant_nn.TensorQuantizer):
            if module._calibrator is not None:
                module.disable_quant()
                module.enable_calib()
            else:
                module.disable()

    # Feed data to the network for collecting stats
    for i, (image, mask, original_image, image_path) in tqdm(enumerate(data_loader), total=num_batches):
        # Forward pass with the image only
        model(image.cuda())
        if i >= num_batches - 1:  # `i` is zero-based
            break

    # Disable calibrators
    for name, module in model.named_modules():
        if isinstance(module, quant_nn.TensorQuantizer):
            if module._calibrator is not None:
                module.enable_quant()
                module.disable_calib()
            else:
                module.enable()

                
def calibrate_model(model, model_name, data_loader, num_calib_batch, calibrator, hist_percentile, out_dir):
    """
    Feed data to the network and calibrate.
    Arguments:
        model: classification model
        model_name: name to use when creating state files
        data_loader: calibration data set
        num_calib_batch: amount of calibration passes to perform
        calibrator: type of calibration to use (max/histogram)
        hist_percentile: percentiles to be used for historgram calibration
        out_dir: dir to save state files in
    """
    if num_calib_batch > 0:
        print("Calibrating model")
        with torch.no_grad():
            collect_stats(model, data_loader, num_calib_batch)

        if calibrator == "max":
            compute_amax(model, method="max")
            calib_output = os.path.join(
                out_dir,
                F"{model_name}-max-{num_calib_batch*data_loader.batch_size}.pth"
            )
            torch.save(model.state_dict(), calib_output)
        elif calibrator == "histogram":
            for percentile in hist_percentile:
                print(F"{percentile} percentile calibration")
                compute_amax(model, method="percentile", percentile=percentile)
                calib_output = os.path.join(
                    out_dir,
                    F"{model_name}-percentile-{percentile}-{num_calib_batch*data_loader.batch_size}.pth"
                )
                torch.save(model.state_dict(), calib_output)
        else:
            raise ValueError(f"Unsupported calibrator type: {calibrator}")
    print("Calibration completed!")



def convert_to_tensorrt_and_benchmark(model, input_shape=(8, 3, 256, 256), output_dir="./output"):

    print("TensorRT starting...")


    trt_compile_spec = {
        "inputs": [torch_tensorrt.Input(input_shape)],  
        "enabled_precisions": {torch.int8},  
    }


    trt_model = torch_tensorrt.compile(model, **trt_compile_spec)
    print("TensorRT convert completed!")


    trt_model_path = os.path.join(output_dir, "model_Unet_trt_int8.pt")
    torch.jit.save(trt_model, trt_model_path)
    print(f"TensorRT INT8 model save completed: {trt_model_path}")
    
    print("TensorRT model inference benchmark starting...")
    input_data = torch.randn(input_shape).to("cuda")
    benchmark_model(trt_model, input_data)



def benchmark_model(model, input_data, num_warmup=10, num_runs=100):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model = model.to(device)
    model.eval()

    print("Warm-up...")
    for _ in range(num_warmup):
        with torch.no_grad():
            _ = model(input_data)

    print("inference time check starting...")
    times = []
    for _ in range(num_runs):
        start_time = torch.cuda.Event(enable_timing=True)
        end_time = torch.cuda.Event(enable_timing=True)

        start_time.record()
        with torch.no_grad():
            _ = model(input_data)
        end_time.record()

        torch.cuda.synchronize()
        times.append(start_time.elapsed_time(end_time))

    print(f"average inference time (ms): {sum(times) / len(times):.2f}")


def main():
    dataset_dir = "D:\\my_UNet"
    output_dir = "D:\\my_UNet\\output"
    os.makedirs(output_dir, exist_ok=True)

    train_dataset = cc_weld_dataset(dataset_dir=os.path.join(dataset_dir, "dataset", "train"), augmentation=None)
    train_loader = DataLoader(dataset=train_dataset, batch_size=8, shuffle=True)

    model_path = os.path.join(output_dir, "model_Unet_origin_best.pt")
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model = UNet(n_channels=3, n_classes=1).to(device)

    model.load_state_dict(torch.load(model_path, map_location=device))
    model.eval() 

    print("Model Calibration starting...")
    calibrate_model(
        model=model,
        model_name="unet",
        data_loader=train_loader,
        num_calib_batch=4, 
        calibrator="max", 
        hist_percentile=[99.9, 99.99, 99.999, 99.9999],
        out_dir=output_dir
    )

    print("Model loading complted and TensorRT convert starting...")
    convert_to_tensorrt_and_benchmark(
        model=model,
        input_shape=(8, 3, 256, 256),
        output_dir=output_dir
    )


if __name__ == "__main__":
    main()