Quantized Gradual Aware Training for Image Classification
- 주제(키워드) Quantization , Classification , Deep neural network , transfer learning , Knowledge Distillation
- 주제(DDC) 006.31
- 발행기관 아주대학교 일반대학원
- 지도교수 Wonjun Hwang
- 발행년도 2025
- 학위수여년월 2025. 2
- 학위명 석사
- 학과 및 전공 일반대학원 인공지능학과
- 실제URI http://www.dcollection.net/handler/ajou/000000034492
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
With the rapid development of articial intelligence, the size and computational complexity of deep learning models have increased sig- nicantly, necessitating various techniques to efciently manage these models. In particular, quantization techniques play a crucial role in re- ducing memory usage and improving computational speed by convert- ing model weights and activation values to low-bit precision. However, existing Quantization-Aware Training (QAT) methods face limited prac- ticality due to their complex training procedures and additional compu- tational costs. In this paper, we propose a simple yet effective Quan- tized Gradual Aware Training (QGAT) technique to address these limi- tations. QGAT gradually quantizes the channels of a CNN to minimize information loss during the quantization process. This approach aims to maximize the benets of quantization while mitigating the performance degradation of the model. In our experiments, we trained ResNet-20 and ResNet-32 models on the CIFAR-10 and CIFAR-100 datasets, demon- strating that QGAT can enhance the model’s efciency and lightweight characteristics while maintaining a comparable level of accuracy to tra- ditional models. These ndings suggest that the proposed technique can facilitate the deployment of deep learning models in resource-constrained environments. This study highlights the practicality and effectiveness of QGAT, and we plan to extend its application to various models and datasets in future work to verify its generalizability. Keywords: Quantization, Classication, Deep neural network, transfer learning, Knowledge Distillation
more목차
1. Introduction 1
2. Related Work 6
2.1 Quantization-Aware Training 6
2.2 Fisher Information Matrix 7
2.3 Straight-Through Estimator (STE) 8
2.4 Quantized Knowledge Distillation 9
3. Proposed Method 12
3.1 Distillation from High-Bit to Low-Bit Models 13
3.2 Quantization-Gradual Aware Training (QGAT) 15
4. Experimental Results and Discussions 19
4.1 Dataset and Evaluation Protocol 19
4.2 Implemental Details 20
4.3 Performance Comparison Table 21
4.4 Ablation Experiments 23
5. Conclusion 25
Bibliography 26
