In statkclee/suggestK: Crawl and Visualize Related Search Terms from the Major Search Engines in Korea

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
knitr::opts_chunk$set(echo = TRUE, warning=FALSE, message=FALSE)

The package crawls search terms from the major search engines in Korea; daum, naver and google. The usage is straightforward.

1. select search engine function and run that function
   • daum: daum_keyword()
   • naver: naver_keyword()
   • google: google_keyword()
2. if you want to visualize search terms relationship, call suggest_list2df()) function which will convert list data structure to dataframe.
3. call visualize_suggest() function which will transform native dataframe into network object automatically and visualize search term relationship; keyword, 1st search terms, and 2nd search terms. networkD3 is a main engine for the newtork visualization.

Install

You could install suggestK package through Github.

``` {r install-suggestK} devtools::install_github("statkclee/suggestK") library(suggestK)

## Crawl Related Search Terms

### Daum

[daum](https://www.daum.net/) search engine provides very flexible related search results in that if a search term is not used frequently, it will return related keywrods quickly, but it could take a couple of minutes in some cases. 

`daum_keyword()` function returns list datatype such that it will be useful to see related keywords with the `jsonedit()` function in `listviewer` package.

In case of the multiple keyword such as "data scientist", you need to add `+` sign between two search keywords. Unless you add `+` sign, you will not get what you expect.
****

``` {r daum}
data_scientist <- daum_keyword("data+scientist")
str(data_scientist)

Naver

naver search engine provides only 10 related keywords, which means that the search query can finish very quickly, but if you expect exhaustive related terms, it will give only partial keywords.

naver_keyword() function returns list datatype such that it will be useful to see related keywords with the jsonedit() function in listviewer package.

``` {r naver} korea <- naver_keyword("korea") str(korea)

### Google

[google](https://www.naver.net/) search engine provides only 10 related keywords such that google suggest query works really fast, but compared with Daum and Naver, **Google Suggest** works differently. You need to be careful when interpreting related terms.

`google_keyword()` function returns list datatype such that it will be useful to see related keywords with the `jsonedit()` function in `listviewer` package.

``` {r google}
korea <- google_keyword("korea")

Convert list to dataframe

Once we have list data from the keyword search, it is necessary to convert list to dataframe. The subsequent data analysis, modeling, and visualization tasks prefer dataframe in most cases.

The suggestK package has two function for the same task because Korean and non-Korean search engines have different interfaces. suggest_list2df() function convert list from NAVER and DAUM search terms to dataframe whereas google_list2df() function convert list from Google search terms to dataframe.

``` {r naver-daum-list2df} suggest_list2df(data_scientist, "data+scientist")[[1]] %>% head()

In case of google, it works similarly.

``` {r google-list2df}
google_list2df(korea, "korea")[[1]] %>% head()

Visualize

visualize_suggest() function enables to visualize Keyword, 1st Search Terms and 2nd Search Terms through networkD3. To get visualized artefacts, pass the search term and search engine name to visualize_suggest() function which will automatically crawl search terms, convert data structure, visualize, and return the results.

{r visualize, fig.width=8, fig.height=8} visualize_suggest("korea", "google")

statkclee/suggestK documentation built on May 21, 2019, 9:22 a.m.