vocab_builder: A fast unigram vocabulary builder
In text2map: R Tools for Text Matrices, Embeddings, and Networks
vocab_builder R Documentation
A fast unigram vocabulary builder
Description
A streamlined function to take raw texts from a column of a data.frame and
produce a list of all the unique tokens. Tokenizes by the fixed,
single whitespace, and then extracts the unique tokens. This can be used as
input to dtm_builder() to standardize the vocabulary (i.e. the columns)
across multiple DTMs. Prior to building the vocabulary, texts should have
whitespace trimmed, if desired, punctuation removed and terms lowercased.
Usage
vocab_builder(data, text)
Arguments
data
Data.frame with one column of texts
text
Name of the column with documents' text
Value
returns a list of unique terms in a corpus
Author(s)
Dustin Stoltz
text2map documentation built on May 29, 2024, 2:54 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| vocab_builder | R Documentation |
A fast unigram vocabulary builder
Description
A streamlined function to take raw texts from a column of a data.frame and
produce a list of all the unique tokens. Tokenizes by the fixed,
single whitespace, and then extracts the unique tokens. This can be used as
input to dtm_builder() to standardize the vocabulary (i.e. the columns)
across multiple DTMs. Prior to building the vocabulary, texts should have
whitespace trimmed, if desired, punctuation removed and terms lowercased.
Usage
vocab_builder(data, text)
Arguments
data |
Data.frame with one column of texts |
text |
Name of the column with documents' text |
Value
returns a list of unique terms in a corpus
Author(s)
Dustin Stoltz
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.