get_tokens: Remove stopwords from text

In discoverableresearch: Checks Title, Abstract and Keywords to Optimise Discoverability

Description

Removes stopwords from text in whichever language is specified.

Removes stop words from a text string (adapted from 'litsearchr' https://github.com/elizagrames/litsearchr/) and returns the remaining words as a vector of strings

Usage

get_tokens(text, language = "English")

get_tokens(text, language = "English")

Arguments

text	An input string
language	The language used to look up stop words (default is "English")

Value

Returns the input text with stopwords removed.

A vector of strings consisting of the non-stop words from the 'text' input

Examples

get_tokens("On the Origin of Species", language="English")
text <- "A methodology for systematic mapping in environmental sciences"
tokens <- get_tokens(text)
tokens;

discoverableresearch documentation built on Oct. 23, 2020, 7:13 p.m.