create_dtm: Document-term matrix construction

Description Usage Arguments Details Value See Also Examples

View source: R/dtm.R

Description

This is a high-level function for creating a document-term matrix.

Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
create_dtm(it, vectorizer, type = c("dgCMatrix", "dgTMatrix",
  "RsparseMatrix"), ...)

## S3 method for class 'itoken'
create_dtm(it, vectorizer, type = c("dgCMatrix",
  "dgTMatrix", "RsparseMatrix"), ...)

## S3 method for class 'itoken_parallel'
create_dtm(it, vectorizer,
  type = c("dgCMatrix", "dgTMatrix", "RsparseMatrix"), ...)

Arguments

it

itoken iterator or list of itoken iterators.

vectorizer

function vectorizer function; see vectorizers.

type

character, one of c("dgCMatrix", "dgTMatrix").

...

placeholder for additional arguments (not used at the moment). over it.

Details

If a parallel backend is registered and first argument is a list of itoken, iterators, function will construct the DTM in multiple threads. User should keep in mind that he or she should split the data itself and provide a list of itoken iterators. Each element of it will be handled in separate thread and combined at the end of processing.

Value

A document-term matrix

See Also

itoken vectorizers

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
## Not run: 
data("movie_review")
N = 1000
it = itoken(movie_review$review[1:N], preprocess_function = tolower,
             tokenizer = word_tokenizer)
v = create_vocabulary(it)
#remove very common and uncommon words
pruned_vocab = prune_vocabulary(v, term_count_min = 10,
 doc_proportion_max = 0.5, doc_proportion_min = 0.001)
vectorizer = vocab_vectorizer(v)
it = itoken(movie_review$review[1:N], preprocess_function = tolower,
             tokenizer = word_tokenizer)
dtm = create_dtm(it, vectorizer)
# get tf-idf matrix from bag-of-words matrix
dtm_tfidf = transformer_tfidf(dtm)

## Example of parallel mode
it = token_parallel(movie_review$review[1:N], tolower, word_tokenizer, movie_review$id[1:N])
vectorizer = hash_vectorizer()
dtm = create_dtm(it, vectorizer, type = 'dgTMatrix')

## End(Not run)

text2vec documentation built on March 26, 2020, 7:48 p.m.