Hi everyone,
I am happy to share YolactEdge, our real-time instance segmentation approach that runs at up to 30.8 FPS on a Jetson AGX Xavier (and 172.7 FPS on an RTX 2080 Ti) with a ResNet-101 backbone on 550x550 resolution images, and achieves 29.5 mAP on COCO’s test-dev.
There are two key aspects that make our model fast and accurate on edge devices: (1) TensorRT optimization while carefully trading off speed and accuracy, and (2) a novel feature warping module to exploit temporal redundancy in videos.
Specifically, for TensorRT optimization, to balance speed and accuracy, instead of converting the whole model to a single precision, we quantize the model weights of four stages (backbone, FPN, proto-net, and prediction head) to INT8 or FP16 precision correspondingly. We have also explored other factors including the number of calibration images used for INT8 quantization. Please refer to our paper for more detailed experiments and explanations.
Hope you like the project and feel free to contribute to it!
Code: GitHub - haotian-liu/yolact_edge: The first competitive instance segmentation approach that runs on small edge devices at real-time speeds.
Paper: [2012.12259] YolactEdge: Real-time Instance Segmentation on the Edge
