feature_skip_ngrams: Skip ngram features

In sebastian-fox/twitterfeatures: Simple Feature Engineering Functions for Twitter Data

Description

Engineers features related to skip word ngrams in Tweets.

Usage

feature_skip_ngrams(data, doc_id_field, text_field, n_ngrams_min = 2L,
  n_ngrams_max = 4L, n_skip_words = 3L, top_num = 1000L)

Arguments

data	a dataframe or tibble containing the text data and document id
doc_id_field	unquoted field name identifying the field within the data that represents the unique document id
text_field	unquoted field name identifying the field name in data that contains the text of the Tweet
n_ngrams_min	integer; the minimum number in the ngram
n_ngrams_max	integer; the maximum number in the ngram
n_skip_words	integer; the maximum number of words to skip when calculating ngrams
top_num	integer, the top n skip ngrams to create features from

Details

Tweets are first converted to lowercase before the features are calculated.

The ngram features are calculated as a proportion of the total number of skip ngrams in the Tweet that contain the same number of words:

• the number of the top_num skip ngrams of all lengths in all Tweets

Value

A data frame of document ids their associated skip ngrams features

Examples

tweets <- data.frame(status_id = c(1234, 5678),
                     text = c("I tweet about one thing #onething #things",
                              "I tweet about another #anotherthing
                               #things"),
                     stringsAsFactors = FALSE)
feature_skip_ngrams(tweets, status_id, text)

sebastian-fox/twitterfeatures documentation built on Nov. 5, 2019, 8:47 a.m.