feature_ngrams: ngram features

In sebastian-fox/twitterfeatures: Simple Feature Engineering Functions for Twitter Data

Description

Engineers features related to ngrams (both word or character) in Tweets.

Usage

feature_ngrams(data, doc_id_field, text_field, type = "ngrams",
  n_ngrams = 1L, top_num = 1000L)

Arguments

data	a dataframe or tibble containing the text data and document id
doc_id_field	unquoted field name identifying the field within the data that represents the unique document id
text_field	unquoted field name identifying the field name in data that contains the text of the Tweet
type	string; either ngrams of character_shingles
n_ngrams	integer; the number in the ngram
top_num	integer, the top n ngrams to create features from

Details

Tweets are first converted to lowercase before the features are calculated. Then, urls are removed, as well as @usernames. Finally, Tweets with fewer words than n_ngrams will return 0.

The ngram features are calculated as a proportion of the total number of ngrams in the Tweet:

• the number of the top_num ngrams in all Tweets

Value

A data frame of document ids their associated ngrams features

Examples

tweets <- data.frame(status_id = c(1234, 5678),
                     text = c("I tweet about one thing #onething #things",
                              "I tweet about another #anotherthing
                               #things"),
                     stringsAsFactors = FALSE)
feature_ngrams(tweets, status_id, text)

sebastian-fox/twitterfeatures documentation built on Nov. 5, 2019, 8:47 a.m.