Protein Sequence Generation via Uncertainty-based Dynamic Beam Search
- 주제(키워드) De novo protein generation , autoregressive language model , evidential deep learning , uncertainty quantification , dynamic beam search
- 주제(DDC) 621.39
- 발행기관 아주대학교 일반대학원
- 지도교수 Sael Lee
- 발행년도 2026
- 학위수여년월 2026. 8
- 학위명 석사
- 학과 및 전공 일반대학원 컴퓨터공학과
- 실제URI http://www.dcollection.net/handler/ajou/000000036484
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
Protein sequence design is a fundamental challenge in computational biology. Recent advances in deep learning-based autoregressive models have made it possible to explore the vast space of protein sequences efficiently. However, existing generation methods still face a critical trade-off between sequence diversity and structural stability. Deterministic decoding strategies, such as standard beam search, favor high-probability tokens but often produce repetitive or overly conservative sequences. On the other hand, stochastic sampling increases sequence diversity but can generate biologically implausible candidates that fail to fold stably. To address this Limitation: This thesis introduces an uncertainty-aware autoregressive framework for protein generation based on Evidential Deep Learning （EDL）. Our framework measures exactly what the model does not know by using a Dirichlet distribution to track its prediction doubts. The system is trained using a two-stage approach: the first stage focuses on optimizing token prediction accuracy, while the second stage optimizes uncertainty estimation using tailored regularization techniques. Building upon this framework, we propose Uncertainty-Based Dynamic Beam Search （UBDBS）, a decoding algorithm that dynamically adjusts its search width based on real- time uncertainty. UBDBS automatically expands the local search space in highly uncertain regions to encourage broader exploration, while shrinking it in high-confidence regions to preserve structural stability. We evaluate our 37M parameter model on the Swiss-Prot and BRENDA enzyme datasets. The results demonstrate that our approach successfully balances exploration and exploitation, successfully bypassing repetitive loops and structural collapse. Furthermore, our model outperforms baseline language models that are more than twenty times its size across multiple evaluation metrics, proving that explicit uncertainty estimation can compensate for raw parameter scale.
more목차
1 Introduction 1
1.1 Biological Background and Motivation 1
1.2 Deep Learning in Protein Engineering 2
1.3 Decoding Strategies 3
1.4 Thesis Overview and Contributions 4
2 Method 5
2.1 Framework Architecture 5
2.2 Stage I: Softmax Cross-Entropy Framework 8
2.3 Stage II: Evidential Deep Learning Framework 9
2.4 Uncertainty-Based Dynamic Beam Search 13
3 Experiments 17
3.1 Data Processing and Tokenization Pipeline 17
3.2 Baseline Models and Generation Protocol 18
3.3 Evaluation Metrics 19
3.4 Architectural Variants and Decoding Dynamics 21
3.4.1 Softmax vs. Evidential Deep Learning (EDL) Head Formulation 21
3.4.2 Evaluation of Decoding Strategies on Swiss-Prot 22
3.5 Comparison with State-of-the-Art Generative Models 26
3.6 Empirical Study on Functional Enzyme Generation 30
4 Related Work 33
4.1 Language Modeling for De Novo Protein Design 33
4.2 Unconditional and Conditional Autoregressive Baselines 33
4.3 The Uncertainty and Decoding Gap in Protein Language Models 34
5 Conclusion 36
Bibliography 38
