SegFormer

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc)
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc)
UNET and SegFormer
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here)
TAO 3.22.05 and TAO 4.0
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

Hi,

I have posted before about adapting UNET for multi-class segmentation, rather than semantic segmentation (UNET Training on Multi-Class Segmentation from Satellite Imagery (DSTL)).

I was wondering if there is a way to do the same for SegFormer? We have about 10+ classes and we are looking to segment the images.

Also, we would use MaskRCNN, but we want to use a more modern algorithm. Are there any plans on the roadmap to implement another SOTA Instance Segmentation algorithm?

Thank you for your time and help,

Derek

3

Yes, it is possible. Refer to SegFormer - NVIDIA Docs and also download latest notebook from TAO Toolkit Getting Started | NVIDIA NGC. More info can be also found in TAO 4 Segformer Input and output dimensions and tensors - #12 by Morganh.

Will sync internally for this feature request. Thanks.