Description Usage Arguments See Also Examples
These functions are a wrapper around unnest_tokens( token = "characters" )
and unnest_tokens( token = "character_shingles" )
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | unnest_characters(
tbl,
output,
input,
strip_non_alphanum = TRUE,
format = c("text", "man", "latex", "html", "xml"),
to_lower = TRUE,
drop = TRUE,
collapse = NULL,
...
)
unnest_character_shingles(
tbl,
output,
input,
n = 3L,
n_min = n,
strip_non_alphanum = TRUE,
format = c("text", "man", "latex", "html", "xml"),
to_lower = TRUE,
drop = TRUE,
collapse = NULL,
...
)
|
tbl |
A data frame |
output |
Output column to be created as string or symbol. |
input |
Input column that gets split as string or symbol. The output/input arguments are passed by expression and support quasiquotation; you can unquote strings and symbols. |
strip_non_alphanum |
Should punctuation and white space be stripped? |
format |
Either "text", "man", "latex", "html", or "xml". If not text, this uses the hunspell tokenizer, and can tokenize only by "word" |
to_lower |
Whether to convert tokens to lowercase. If tokens include
URLS (such as with |
drop |
Whether original input column should get dropped. Ignored if the original input and new output column have the same name. |
collapse |
Whether to combine text with newlines first in case tokens (such as sentences or paragraphs) span multiple lines. If NULL, collapses when token method is "ngrams", "skip_ngrams", "sentences", "lines", "paragraphs", or "regex". |
... |
Extra arguments passed on to tokenizers |
n |
The number of characters in each shingle. This must be an integer greater than or equal to 1. |
n_min |
This must be an integer greater than or equal to 1, and less
than or equal to |
unnest_tokens()
1 2 3 4 5 6 7 8 9 10 | library(dplyr)
library(janeaustenr)
d <- tibble(txt = prideprejudice)
d %>%
unnest_characters(word, txt)
d %>%
unnest_character_shingles(word, txt, n = 3)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.