검색 상세

Reference-based Blind Face Restoration via Pivot Direction Gradient Guidance in Diffusion Models

  • 민건 (Geon Min, 아주대학교 일반대학원)

초록/요약

Reference-based blind face restoration (RefBFR) has gained considerable attention because it utilizes additional reference images to restore facial images in situations where the degradation is caused by unknown factors, making it particularly useful in real-world applications. Recently, guided diffusion mod- els have demonstrated exceptional performance in this task without requiring training. They achieve this by integrating gradients of the losses where each loss reflects the different desired properties of the additional external images. However, these approaches fail to consider potential conflicts between gradients of multiple losses, which can lead to sub-optimal results. To address this issue, we introduce Pivot Direction Gradient guidance (PDGrad), a novel gradient ad- justment method for RefBFR within a guided diffusion framework. To this end, we first define the loss function based on both low-level and high-level features. For loss at each feature level, both the coarsely restored image and the reference image are fully integrated. In cases of conflicting gradients, a pivot gradient is established for each level and other gradients are aligned to it, ensuring that the strengths of both images are maximized. Additionally, if the magnitude of the adjusted gradient exceeds that of the pivot gradient, it is adaptively scaled according to the ratio between the two, placing greater emphasis on the pivot. Extensive experimental results on the CelebRef-HQ dataset show that the proposed PDGrad significantly outperforms competitive approaches both quantitatively and qualitatively.

more

목차

1 Introduction 1
2 Related Works 6
3 Proposed Method 10
3.0.1 Preliminary 10
3.0.2 Overview of our method 12
3.0.3 Loss Function 14
3.0.4 The Proposed PDGrad 17
4 Experiments and Results 21
4.0.1 Experimental Setting 22
4.0.2 Quantitative Comparison 23
4.0.3 Qualitative Comparison 27
4.0.4 Ablation Study 30
5 Conclusions 35
Reference 36

more
반출 Meta View 목록