GCViT (Vision transformer) Training in TAO

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Dear @Morganh

Is there any way we can train ViT Pre-trained GCViT NVImageNet Classification weights | NVIDIA NGC model on TAO? Is there any sample application or notebook for ViT model fine-tuning for the multi-task classification and REID task?

I tried some changes but i got issue – ValueError: Unsupported backbone model: gc_vit_xxtiny

Thanks.

2

You can refer to classification notebook. tao_tutorials/notebooks/tao_launcher_starter_kit/classification_pyt/classification.ipynb at main · NVIDIA/tao_tutorials · GitHub. It mentions finetuning with GCViT model.

3

Can we also use this inside the Re-identification and multi-task classification sample application ?