Equiangular Tight Frame Transformer for Imbalanced Semantic Segmentation
- 주제(키워드) semantic segmentation , class imbalance , neural collapse
- 주제(DDC) 006.31
- 발행기관 아주대학교 일반대학원
- 지도교수 Yong Seok Heo
- 발행년도 2025
- 학위수여년월 2025. 2
- 학위명 석사
- 학과 및 전공 일반대학원 인공지능학과
- 실제URI http://www.dcollection.net/handler/ajou/000000034383
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
Semantic segmentation often suffers from class imbalance, where the label ratio for each class in the dataset is not uniform. Recent studies have addressed the issue of class imbalance in semantic segmentation by leveraging the neural collapse phenomenon in conjunction with an Equiangular Tight Frame (ETF). While the use of ETF aids in enhancing the discriminability of minor classes, class correlation is another crucial factor that must be taken into account. However, managing the balance between class correlation and discrimination through neural collapse remains challenging, as these properties inherently conflict with one another. Moreover, this control is established during the training stage, resulting in a fixed classifier. There is no guarantee that this classifier will consistently perform well with different input images. To address this problem, we propose an Equiangular Tight Frame Transformer (ETFT), a transformer- based model that jointly processes the features and classifier using ETF structure, and dynamically generates the classifier as a function of the input for imbalanced semantic segmentation. Specifically, the classifier initialized with the ETF structure is jointly processed with the input patch tokens during the attention process. As a result, the transformed patch tokens, aided by the ETF structure, achieve discriminability between classes while preserving contextual correlation. The classifier, initially structured as an ETF, is adjusted to incorporate the correlation information, benefiting from the attention mechanism. Furthermore, the learned classifier is combined with the fixed ETF classifier, leveraging the advantages of both. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods for imbalanced semantic segmentation on both the ADE20K and Cityscapes datasets.
more목차
1 Introduction 1
2 Related Works 6
2.1 Transformer-based semantic segmentation 6
2.2 Class Imbalanced Semantic Segmentation 7
3 Proposed Method 10
3.1 Preliminaries 12
3.2 Feature Attention Constraint 14
3.3 Mixed Classifier 15
4 Experiments and Results 19
4.1 Dataset, Metric and Implementation Details 19
4.1.1 Dataset 19
4.1.2 Metric 20
4.1.3 Implementation Details 21
4.2 Comparison with previous methods 21
4.2.1 Quantitative Comparison 21
4.2.2 Qualitative Comparison 26
4.3 Ablation Study 27
4.3.1 segmenter 27
4.3.2 neural collapse comparison 30
5 Conclusions 35
5.1 Limitations 35
5.2 Conclusions 36
