In ropensci/monkeylearn: Accesses the Monkeylearn API for Text Classifiers and Extractors

NOT_CRAN <- identical(tolower(Sys.getenv("NOT_CRAN")), "true")
pat <- Sys.getenv("MONKEYLEARN_KEY")
IS_THERE_KEY <- (pat != "")
NOT_CRAN <- ifelse(IS_THERE_KEY, NOT_CRAN, FALSE)
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  purl = NOT_CRAN,
  eval = NOT_CRAN
)

Intro

This package is an interface to the MonkeyLearn API. MonkeyLearn is a Machine Learning platform on the cloud that allows software companies and developers to easily extract actionable data from text.

The goal of the package is not to support machine learning algorithms development with R or the API, but only to reap the benefits of the existing modules on Monkeylearn. Therefore, there are only two functions, one for using extractors, and one for using classifiers. The difference between extractors and classifiers is that extractors output information about words, whereas classifiers output information about each text as a whole. Named entity recognition is an extraction task, whereas assigning a topic to a text is a classification task.

Setup

To get an API key for MonkeyLearn, register at http://monkeylearn.com/. Note that MonkeyLearn supports registration through GitHub, which makes the registration process really easy. For ease of use, save your API key as an environment variable as described at http://stat545.com/bit003_api-key-env-var.html. You might also want to use the usethis::edit_r_environ() function to modify .Renviron.

All functions of the package will conveniently look for your API key using Sys.getenv("MONKEYLEARN_KEY") so if your API key is an environment variable called "MONKEYLEARN_KEY" you don't need to input it manually.

Please also create a "MONKEYLEARN_PLAN" environment variable indicating whether your Monkeylearn plan is "free", "team", "business" or "custom". If you do not indicate it by default it will be "free" with a message. If your plan is "custom" you'll need a third environment variable "MONKEYLEARN_RATE" indicating the maximum amount of requests per minute that you can make to the API. If you do not indicate it, by default it will be 120 with a message.

So many monkeys/functions

The packages exports monkeylearn_classify, monkey_classify, monkeylearn_extract, monkey_extract. The monkey_ functions are the newer and better ones, so if you don't have legacy code, just start using those!

For inspiration beyond this vignette, you can see external examples of the package in action on this page. In particular you'll find examples using the older set of functions but we now recommend using monkey_extract and monkey_classify, see more later in the vignette.

Extract

A first example

library(monkeylearn)
library(magrittr)

text <- "In the 19th century, the major European powers had gone to great lengths to maintain a balance of power throughout Europe, resulting in the existence of a complex network of political and military alliances throughout the continent by 1900.[7] These had started in 1815, with the Holy Alliance between Prussia, Russia, and Austria. Then, in October 1873, German Chancellor Otto von Bismarck negotiated the League of the Three Emperors (German: Dreikaiserbund) between the monarchs of Austria-Hungary, Russia and Germany."
output <- monkey_extract(input = text,
                         extractor_id = "ex_isnnZRbS")
output
attr(output, "headers")

Parameters

If the documentation of the extractor you use states it has parameters, you can pass them as a named list, see below.

text <- "A panel of Goldman Sachs employees spent a recent Tuesday night at the
Columbia University faculty club trying to convince a packed room of potential
recruits that Wall Street, not Silicon Valley, was the place to be for computer
scientists.\n\n The Goldman employees knew they had an uphill battle. They were
fighting against perceptions of Wall Street as boring and regulation-bound and
Silicon Valley as the promised land of flip-flops, beanbag chairs and million-dollar
stock options.\n\n Their argument to the room of technologically inclined students
was that Wall Street was where they could find far more challenging, diverse and,
yes, lucrative jobs working on some of the worlds most difficult technical problems."

output <- monkey_extract(text,
                        extractor_id = "ex_y7BPYzNG",
                        params = list(max_keywords = 3))
output
output2 <- monkey_extract(text,
                          extractor_id = "ex_y7BPYzNG",
                          params = list(max_keywords = 1))
output2
attr(output2, "headers")

How to find extractors?

You can find extractors and their IDs, including extractors for text in Spanish, at https://app.monkeylearn.com/main/explore

There is no endpoint for automatically finding all extractors, but if you find one in the website you particularly like and use a lot in your language and application, you could choose to save its id as an environment variable as explained here. Reading about extractors on the website will give you a good overview of their characteristics and original application.

Here are a few ones for text in English:

• Entity extractor, extractor_id = "ex_isnnZRbS" (used in the first example). Extract Entities from text using Named Entity Recognition (NER). NER labels sequences of words in a text which are the names of things, such as person and company names. This implementation labels 3 classes: PERSON, ORGANIZATION and LOCATION. This NER tagger is implemented using Conditional Random Field (CRF) sequence models.

• Keyword extractor, extractor_id = "ex_y7BPYzNG". Extract keywords from text in English. Keywords can be compounded by one or more words and are defined as the important topics in your content and can be used to index data, generate tag clouds or for searching. This keyword extraction algorithm employs statistical algorithms and natural language processing technology to analyze your content and identify the relevant keywords.

text <- "A panel of Goldman Sachs employees spent a recent Tuesday night at the Columbia University faculty club trying to convince a packed room of potential recruits that Wall Street, not Silicon Valley, was the place to be for computer scientists.

The Goldman employees knew they had an uphill battle. They were fighting against perceptions of Wall Street as boring and regulation-bound and Silicon Valley as the promised land of flip-flops, beanbag chairs and million-dollar stock options.

Their argument to the room of technologically inclined students was that Wall Street was where they could find far more challenging, diverse and, yes, lucrative jobs working on some of the world’s most difficult technical problems.

“Whereas in other opportunities you might be considering, it is working one type of data or one type of application, we deal in hundreds of products in hundreds of markets, with thousands or tens of thousands of clients, every day, millions of times of day worldwide,” Afsheen Afshar, a managing director at Goldman Sachs, told the students."

monkey_extract(text, extractor_id = "ex_y7BPYzNG")

• Useful data extractor, extractor_id = "ex_dqRio5sG". Extract useful data from text. This algorithm can be used to detect many different useful data: links, phones, ips, prices, times, emails, bitcoin addresses, dates, ipv6s, hex colors and credit cards.

When using this extractor, the format of the API output is a bit different than for other extractors, see below how the output looks like.

text <- "Hi, my email is [e-mail] and my credit card is 4242-4242-4242-4242 so you can charge me with $10. My phone number is 15555 9876. We can get in touch on April 16, at 10:00am"
text2 <- "Hi, my email is [e-mail] and my credit card is 4242-4232-4242-4242. My phone number is 16655 9876. We can get in touch on April 16, at 10:00am"

monkey_extract(c(text, text2), extractor_id = "ex_dqRio5sG", unnest = TRUE)

Classify

A first example

text1 <- "my dog is an avid rice eater"
text2 <- "i want to buy an iphone"
request <- c(text1, text2)

monkey_classify(request, classifier_id = "cl_oFKL5wft")

How to find classifiers?

You can find classifiers and their IDs at https://app.monkeylearn.com/main/explore or you can use the monkeylearn_classifiers function, choosing to show all classifiers or only the private ones with private = TRUE. The first column of the resulting data.frame is the classifier_id to be used in monkeylearn_classify.

monkeylearn_classifiers(private = FALSE)

Here are a few other examples:

• Language detection, classifier_id = "cl_oJNMkt2V". Detect language in text. New languages were added for a total of 48 different languages arranged in language families.

text1 <- "Hauràs de dirigir-te al punt de trobada del grup al que et vulguis unir."
text2 <- "i want to buy an iphone"
text3 <- "Je déteste ne plus avoir de dentifrice."
request <- c(text1, text2, text3)

monkey_classify(request, classifier_id = "cl_oJNMkt2V")

• Profanity and abuse detection, classifier_id = "cl_KFXhoTdt".

text1 <- "I think this is awesome."
text2 <- "Holy shit! You did great!"
request <- c(text1, text2)

monkey_classify(request, classifier_id = "cl_KFXhoTdt")

• General topic classifier, classifier_id = "cl_5icAVzKR".

text1 <- "Let me tell you about my dog and my cat. They are really friendly and like going on walks. They both like chasing mice."
text2 <- "My first R package was probably a disaster but I keep learning how to program."
request <- c(text1, text2)
monkey_classify(request, classifier_id = "cl_5icAVzKR")

Get what you paid for

Monkeylearn offers a different service based on your current plan, that is, "free", "team" or "business". These plans will both influence your rate limiting (how fast?) and your query limiting (how many queries?). See https://monkeylearn.com/pricing/. Thanks to your MONKEYLEARN_PLAN environment variable, the rate will be handled automatically thanks to ratelimitr.

Check the number of remaining calls

After each call to a function you can check how many calls to the API you can still make using attr(output, "headers")$x.query.limit.remaining and attr(output, "headers")$x.query.limit.limit. The period after which attr(output, "headers")$x.query.limit.remaining depends on your subscription and is not included in the output.

Fit monkeylearn into your pipeline!

You can:

• Send a vector of texts or a dataframe and a named column (unquoted)
• Output either a nested or unnested dataframe
  • Nested = 1 row per input; unnested = 1 row per output
• This output
  • Relates each input text to its (usually) multiple classifications/extractions
  • Retains a record of inputs that could not be classified/extracted (e.g., empty strings)
• Batch requests

In a bit more detail

You can classify or extract a vector or dataframe of texts while relating the original input text to its classifications. This is important, because the input:output relationship may not always (and in fact, is not usually) 1:1. These functions retain the tie between each input[^1] element and all of its output elements.

input <- c("Emma Woodhouse, handsome, clever, and rich, with a comfortable home",     
 "and happy disposition, seemed to unite some of the best blessings of",  
 "existence; and had lived nearly twenty-one years in the world with very", 
 "little to distress or vex her.",                                          
 "",                   # <--- note the empty string!                                                   
 "She was the youngest of the two daughters of a most affectionate,",       
 "indulgent father; and had, in consequence of her sister's marriage, been",
 "mistress of his house from a very early period. Her mother had died",     
 "too long ago for her to have more than an indistinct remembrance of",     
 "her caresses; and her place had been supplied by an excellent woman as",  
 "governess, who had fallen little short of a mother in affection.")

That is true even if you have inputs that cannot be processed. For instance, empty string and NA input elements are not sent to the API for classification/extraction. (You'll get a warning of this if verbose = TRUE.) We've got one above to illustrate and elements that returned no classifications/extractions are included in the resulting dataframe. This way you'll know which inputs could not be processed.

(output <- monkey_classify(input, unnest = FALSE))

If there are more than 20 empty inputs, we save your console by messaging only the first 20 indices.

(very_empty_input <- rep("", 25) %>% c(input) %>% sample())

Since the entire original input is represented in the output, if you need to find all of the empty inputs you can easily filter the output to all of the rows containing empty strings.

monkey_classify(very_empty_input, unnest = FALSE)

Configuring the Output

The default output is a nested dataframe with the same number of rows as your input dataframe or the same length as your input vector, depending on which one you sent in.

Let's take a look at the res output column.

output$res

You can easily choose an unnested output by setting the unnest flag to TRUE (which it is by default) to get one row per classification/extraction.

(output_unnested <- monkey_classify(input, verbose = FALSE, unnest = TRUE))

We could have gotten the same result by sending in a dataframe and a named column. If a dataframe is supplied input column is not renamed to req as it is when input is a vector; the original column name is retained.

input_df <- tibble::tibble(text = input) 
output_df_unnested <- monkey_classify(input_df, text, unnest = TRUE, verbose = FALSE) %>% 
    dplyr::rename(req = text)

testthat::expect_equal(output_unnested, output_df_unnested)

If the input is a dataframe, setting the .keep_all option to TRUE allows you to retain all input columns. If FALSE, only the column you specify for classification will be retained.

sw <- dplyr::starwars %>% 
  dplyr::select(name, height) %>% 
  dplyr::sample_n(nrow(input_df))

sw_input_df <- input_df %>% 
  dplyr::bind_cols(sw)

sw_input_df %>% monkey_classify(text, unnest = FALSE, verbose = FALSE)

Batching

Retaining the relationship between input and output doesn't mean you'll need to send requests one-by-one. Batch requests by setting the texts_per_req value which governs the number of texts that are sent per request. Per the MonkeyLearn documentation, the maximum we recommend sending at once is 200 requests.

If texts_per_req is NULL, the default, we try to optimize the response time from the API by setting texts_per_req to 200 when your input has more than 200 texts or to the length of the input if you've got fewer. You'll see a significant speedup by batching your requests this way. However, batching doesn't save you on queries; a batch of 150 texts still uses up 150 queries.

These functions also include some more verbose progress reporting, letting you know what batch you're on out of the total, and which texts are set to be processed in that batch.

one_by_one <- system.time(output <- monkey_classify(input, texts_per_req = 1))

batch_of_five <- system.time(output <- monkey_classify(input, texts_per_req = 5))

How much does sending 5 texts in a batch vs. 1 text improve our processing time?

(speedup <- one_by_one[1] / batch_of_five[1])

A 3-4x speedup isn't so bad! Worth keeping in mind that if you need the blazing fast speeds you might consider upgrading to a higher MonkeyLearn price tier.

---

Meta

• Please report any issues or bugs.
• License: GPL
• Get citation information for monkeylearn in R doing citation(package = 'monkeylearn')
• Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
• This package is part of the rOpenSci project.

[^1]: Thanks to Julia Silge's fantastic janeaustenr package for this text!

ropensci/monkeylearn documentation built on May 18, 2022, 5:38 p.m.