I want to train the cradio+segformer model on more than just a single class now. I have prepared my labels in 8-bit png masks where the bits (0, 1, 2, etc.) correspond to my label_id in dataset.segment.palette and label_transform: None. I also have provided a palette for each label_id as follows: …

I already tried all the above, and seems like no matter the num_class, if I don’t include a palette and set label transform to norm, all foreground classes are combined into a single foreground class. Also, when I set background to 255, my val_loss metric starts reporting only NaN. The only combinat…

[image] kianmehr.ehtiatkar2: - label_id: 0 mapping_class: background rgb: - 0 - 0 - 0 seg_class: background - label_id: 1 mapping_class: Seal rgb: - 255 - 0 - 0 seg_class: Seal - label_id: 2 …

Hi @Morganh , I’m setting the number of classes to not include the background but also including background as a palette with label_id=255. Inspecting the tensorboard event I see that only the first label is being learned, but even that is not done correctly. For instance, the metrics for iou_0 and …

I see this user is also having the same issue: [image] TAO 5.3 Segformer results poor TAO Toolkit For TAO5.2, how about the result? If you did not save the log, you can share the final result for all the classes.

Hi @kianmehr.ehtiatkar2 , Every pixel in the mask must have an integer value that represents the segmentation class label_id ". But I find that some mask files do not have the correct pixel values. Please double check the mask files. Thanks! from PIL import Image import numpy as np #img = Image.…

Hi @Morganh I purged my dataset of background-only images and masks and trained again with the same behavior.

I’m noticing that only the first label or label_id=0 is being learned per the combined visualization image attached. I’ve also attached the mask file for this image for reference showing 2 objects with integers 0 and 1 and background at 255. Only integer 0 is being learned, and the model performance…

Hi @kianmehr.ehtiatkar2 , Firstly, make sure each mask file has corresponding pixel value against the class. For example, one class has pixel-value=0, another class has pixel-value=1. Successful case1: Then if you are going to use palette, please change 1-channel mask png file to 3-channel mask…

@Morgan Huang , thank you for the response. I ran a quick experiment with the first method with removing the palette, and I saw meaningful metrics, so thank you! [image] If I have multiple classes, how do I extract the respective class from my model from the resulting segmentation from inference?…

As we synced offline, if want to contain background + 3-channel RGB png mask + palette, then, dataset: segment: dataset: "SFDataset" root_dir: /path/to/dataset_rgb num_classes: 3 # background + 2 foreground classes img_size: 224 train_split: "train" validation_s…

Training Segformer+Cradiov2 with multiple classes not working (only learning 1 class)

Accelerated Computing Intelligent Video Analytics TAO Toolkit

kianmehr.ehtiatkar2 October 29, 2025, 10:01pm 4

I see this user is also having the same issue:

Topic		Replies	Views
SegFormer fine-tuning TAO Toolkit jetson	32	414	March 3, 2026
Migrating TAO3 unet model to segformer, Foreground has performance of 0.0 ! TAO Toolkit	27	1391	February 27, 2023
Accuracy and mIoU of 1.0 when validating Mask2Former TAO Toolkit	27	778	January 6, 2026
TAO 4 Segformer Input and output dimensions and tensors TAO Toolkit	10	1014	February 21, 2023
TAO Mask2former Binary Instance Segmentation Training TAO Toolkit segmentation , tao	1	150	July 17, 2025
Problem in training unet TAO Toolkit	21	2161	August 30, 2021
Dealing with Class Imbalance in SegFormer Semantic Segmentation TAO Toolkit	1	151	February 27, 2025
Problem in training Unet with multi class labels TAO Toolkit	10	1267	January 16, 2022
Fail with Transfer Learning with Unet Multiclass, Color Images, Semantic Segmentation TAO Toolkit	18	1872	January 18, 2022
Segmentation on instances with holes in an instance mask TAO Toolkit	1	19	July 23, 2026

Training Segformer+Cradiov2 with multiple classes not working (only learning 1 class)

Related topics