knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.path = "man/figures/README-", out.width = "100%" )
R package for automatic Korean word spacing.
Python verson can be found here.
Word spacing is one of the important parts of the preprocessing of Korean text analysis. Accurate spacing greatly affects the accuracy of subsequent text analysis. KoSpacing
has fairly accurate automatic word spacing performance, especially good for online text originated from SNS or SMS.
For example.
"아버지가방에들어가신다." can be spaced both of below.
Common sense, the first is the right answer.
KoSpacing
is based on Deep Learning model trained from large corpus(more than 100 million NEWS articles from Chan-Yub Park).
| Test Set | Accuracy | |---|---| | Sejong(colloquial style) Corpus(1M) | 97.1% | | OOOO(literary style) Corpus(3M) | 94.3% |
To install from GitHub, use
install.packages('remotes') remotes::install_github('haven-jeon/KoSpacing') library(KoSpacing) set_env()
library(KoSpacing) spacing("김형호영화시장분석가는'1987'의네이버영화정보네티즌10점평에서언급된단어들을지난해12월27일부터올해1월10일까지통계프로그램R과KoNLP패키지로텍스트마이닝하여분석했다.")
@misc{heewon2018, author = {Heewon Jeon}, title = {KoSpacing: Automatic Korean word spacing}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/haven-jeon/KoSpacing}}
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.