인코더-디코더 모델에서 다양한 한국어 표현방법의 결합을 통한 언어모델링의 성능향상
Improving Performance of Korean Language Modeling by Combining Multi-Representations in Encoder-Decoder Model
- 주제(키워드) Deep Learning , NLP , Korean Language Model , Sequence-to-Sequence , Encoder-Decoder
- 발행기관 아주대학교
- 지도교수 손경아
- 발행년도 2017
- 학위수여년월 2017. 2
- 학위명 석사
- 학과 및 전공 일반대학원 컴퓨터공학과
- 실제URI http://www.dcollection.net/handler/ajou/000000024639
- 본문언어 영어
- 저작권 아주대학교 논문은 저작권에 의해 보호받습니다.
초록/요약
Most successful neural language modeling have mainly focused on English language and operated at the level of words. Although word-level based neural language model performs well, some other languages cannot get word-level token easily especially in Korean language model. In the field of natural language processing for the Korean language, morpheme-level approaches are commonly employed as alternatives to the word in English. However, it causes the dependency to the external morphology analyzer. Accordingly, character-level approaches are preferred for Korean neural language modeling, but there are several ways to represent the Korean language in character-level, as which we call them Korean-letter level and Korean-grapheme level representations. In this thesis, we investigate the best representations of the Korean language with evaluation on three experiments: reconstruction, marker classification, and spelling correction. Since there are no public datasets in the Korean language for the three tasks, we first generate the datasets using a Korean public corpus. Furthermore, we propose an advanced architecture which can effectively employ various representations for sequence-to-sequence problem in the Korean language. In our experiments, we showed that the proposed architecture outperforms the traditional architectures that use only one of the representations.
more목차
I. Introduction 1
A. Background 1
B. Research Objectives 2
C. Thesis organizations 3
II. Related Works 4
A. English word level neural language modeling 4
B. English character level neural language modeling 4
C. Korean morpheme level neural language modeling 5
D. Korean character level neural language modeling 5
III. Methods 7
A. Data Set: Sejong Corpus with Colloquial Style 7
１. Marker classification datasets generated from Sejong Corpus 11
２. Spelling correction datasets generated from Sejong Corpus 12
B. Encoder-Decoder Model 14
１. Sequence-to-Sequence Problem 14
２. Recurrent Neural Networks 15
３. Encoder-Decoder Model 16
C. Proposed Encoders-Decoder Model 18
IV. Results 20
A. Reconstruction Experiment 20
１. Experimental Setup 20
２. Result 21
B. Marker Classification (조사, 助詞, Josa, 토씨) 23
１. Experimental Setup 23
２. Result 24
C. Spelling Correction 25
１. Experimental Setup 25
２. Result 26
３. Candidate Scoring Experiment 30
V. Conclusion 32
References 33
