In paithiov909/RcppKagome: 'Rcpp' Interface to Kagome

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  tidy = "styler",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

stopifnot(
  require(RcppMeCab),
  require(RcppKagome)
)
## Initial loads of dynamic libraries ---
pos("test load")
posParallel("test load")
kagome("test load")

RcppKagome

RcppKagome is an R interface to ikawaha/kagome; Self-contained Japanese morphological analyzer written in pure Go.

System Requirements

GNU make
GNU GCC
Go (>= 1.16)
Kagome (v2 or later)

Installation

remotes::install_github(
  "paithiov909/RcppKagome"
  #, INSTALL_opts = "--no-multiarch"  # for windows user
)

Note that the installation RcppKagome from source package requires ikawaha/kagome (v2 or later).

By default, the package uses a static library generated with Cgo, which contains the Japanese IPA dictionary.

However, you can also specify another dictionary to be bundled before build and install the package. If you would like to use another one, please set RCPPKAGOME_DIC as an evironment variable.

Sys.setenv(RCPPKAGOME_DIC = "uni") # for using uni-dic
# Or
Sys.setenv(RCPPKAGOME_DIC = "ko") # for using mecab-ko-dic

Usage

Call Kagome

res <- RcppKagome::kagome("雨にも負けず　風にも負けず")
str(res)

Prettify Output

res <- RcppKagome::kagome(
  c("陽が照って鳥が啼き　あちこちの楢の林も、けむるとき",
    "ぎちぎちと鳴る　汚い掌を、おれはこれからもつことになる"))
res <- RcppKagome::prettify(res)
str(res)

In case using IPA dictionary, prettified outputs have these columns.

doc_id: 文番号（sentence index）
token: 表層形 (surface form)
POS1~POS4: 品詞, 品詞細分類1, 品詞細分類2, 品詞細分類3
X5StageUse1: 活用型（ex. 五段, 下二段…）
X5StageUse2: 活用形（ex. 連用形, 基本形…）
Original: 原形（lemmatised form）
Yomi1: 読み（readings）
Yomi2: 発音（pronunciation）

Performance

Targets of Comparison

paithiov909/RcppKagome
RcppKagome::kagome
paithiov909/RcppMeCab: a fork originally from junhewk/RcppMeCab
RcppMeCab::pos
RcppMeCab::posParallel

Data

Here uses whole text of 'Wagahai Wa Neko Dearu' written by Natsume Souseki. The text is originally from Aozora Bunko.

sentences <- readLines("inst/NekoText.gz", encoding = "UTF-8")
dplyr::glimpse(sentences)

Tokenize Character Scalar

tm <- microbenchmark::microbenchmark(
  pos = RcppMeCab::pos(sentences[30]),
  posParallel = RcppMeCab::posParallel(sentences[30]),
  kagome = RcppKagome::kagome(sentences[30]),
  times = 500L
)
summary(tm)

ggplot2::autoplot(tm)

Tokenize Character Vector

tm <- microbenchmark::microbenchmark(
  pos = RcppMeCab::pos(sentences),
  posParallel = RcppMeCab::posParallel(sentences),
  kagome = RcppKagome::kagome(sentences),
  times = 10L
)
summary(tm)