NOT_CRAN <- identical(tolower(Sys.getenv("NOT_CRAN")), "true") pat <- Sys.getenv("MONKEYLEARN_KEY") IS_THERE_KEY <- (pat != "") NOT_CRAN <- ifelse(IS_THERE_KEY, NOT_CRAN, FALSE) knitr::opts_chunk$set( collapse = TRUE, comment = "#>", purl = NOT_CRAN, eval = NOT_CRAN )
This package is an interface to the MonkeyLearn API. MonkeyLearn is a Machine Learning platform on the cloud that allows software companies and developers to easily extract actionable data from text.
The goal of the package is not to support machine learning algorithms development with R or the API, but only to reap the benefits of the existing modules on Monkeylearn. Therefore, there are only two functions, one for using extractors, and one for using classifiers. The difference between extractors and classifiers is that extractors output information about words, whereas classifiers output information about each text as a whole. Named entity recognition is an extraction task, whereas assigning a topic to a text is a classification task.
To get an API key for MonkeyLearn, register at http://monkeylearn.com/. Note that MonkeyLearn supports registration through GitHub, which makes the registration process really easy. For ease of use, save your API key as an environment variable as described at http://stat545.com/bit003_api-key-env-var.html. You might also want to use the usethis::edit_r_environ()
function to modify .Renviron.
All functions of the package will conveniently look for your API key using Sys.getenv("MONKEYLEARN_KEY")
so if your API key is an environment variable called "MONKEYLEARN_KEY" you don't need to input it manually.
Please also create a "MONKEYLEARN_PLAN" environment variable indicating whether your Monkeylearn plan is "free", "team", "business" or "custom". If you do not indicate it by default it will be "free" with a message. If your plan is "custom" you'll need a third environment variable "MONKEYLEARN_RATE" indicating the maximum amount of requests per minute that you can make to the API. If you do not indicate it, by default it will be 120 with a message.
The packages exports monkeylearn_classify
, monkey_classify
, monkeylearn_extract
, monkey_extract
. The monkey_
functions are the newer and better ones, so if you don't have legacy code, just start using those!
For inspiration beyond this vignette, you can see external examples of the package in action on this page. In particular you'll find examples using the older set of functions but we now recommend using monkey_extract
and monkey_classify
, see more later in the vignette.
library(monkeylearn) library(magrittr) text <- "In the 19th century, the major European powers had gone to great lengths to maintain a balance of power throughout Europe, resulting in the existence of a complex network of political and military alliances throughout the continent by 1900.[7] These had started in 1815, with the Holy Alliance between Prussia, Russia, and Austria. Then, in October 1873, German Chancellor Otto von Bismarck negotiated the League of the Three Emperors (German: Dreikaiserbund) between the monarchs of Austria-Hungary, Russia and Germany." output <- monkey_extract(input = text, extractor_id = "ex_isnnZRbS") output attr(output, "headers")
If the documentation of the extractor you use states it has parameters, you can pass them as a named list, see below.
text <- "A panel of Goldman Sachs employees spent a recent Tuesday night at the Columbia University faculty club trying to convince a packed room of potential recruits that Wall Street, not Silicon Valley, was the place to be for computer scientists.\n\n The Goldman employees knew they had an uphill battle. They were fighting against perceptions of Wall Street as boring and regulation-bound and Silicon Valley as the promised land of flip-flops, beanbag chairs and million-dollar stock options.\n\n Their argument to the room of technologically inclined students was that Wall Street was where they could find far more challenging, diverse and, yes, lucrative jobs working on some of the worlds most difficult technical problems." output <- monkey_extract(text, extractor_id = "ex_y7BPYzNG", params = list(max_keywords = 3)) output output2 <- monkey_extract(text, extractor_id = "ex_y7BPYzNG", params = list(max_keywords = 1)) output2 attr(output2, "headers")
You can find extractors and their IDs, including extractors for text in Spanish, at https://app.monkeylearn.com/main/explore
There is no endpoint for automatically finding all extractors, but if you find one in the website you particularly like and use a lot in your language and application, you could choose to save its id as an environment variable as explained here. Reading about extractors on the website will give you a good overview of their characteristics and original application.
Here are a few ones for text in English:
Entity extractor, extractor_id = "ex_isnnZRbS"
(used in the first example). Extract Entities from text using Named Entity Recognition (NER). NER labels sequences of words in a text which are the names of things, such as person and company names. This implementation labels 3 classes: PERSON, ORGANIZATION and LOCATION. This NER tagger is implemented using Conditional Random Field (CRF) sequence models.
Keyword extractor, extractor_id = "ex_y7BPYzNG"
. Extract keywords from text in English. Keywords can be compounded by one or more words and are defined as the important topics in your content and can be used to index data, generate tag clouds or for searching. This keyword extraction algorithm employs statistical algorithms and natural language processing technology to analyze your content and identify the relevant keywords.
text <- "A panel of Goldman Sachs employees spent a recent Tuesday night at the Columbia University faculty club trying to convince a packed room of potential recruits that Wall Street, not Silicon Valley, was the place to be for computer scientists. The Goldman employees knew they had an uphill battle. They were fighting against perceptions of Wall Street as boring and regulation-bound and Silicon Valley as the promised land of flip-flops, beanbag chairs and million-dollar stock options. Their argument to the room of technologically inclined students was that Wall Street was where they could find far more challenging, diverse and, yes, lucrative jobs working on some of the world’s most difficult technical problems. “Whereas in other opportunities you might be considering, it is working one type of data or one type of application, we deal in hundreds of products in hundreds of markets, with thousands or tens of thousands of clients, every day, millions of times of day worldwide,” Afsheen Afshar, a managing director at Goldman Sachs, told the students." monkey_extract(text, extractor_id = "ex_y7BPYzNG")
extractor_id = "ex_dqRio5sG"
. Extract useful data from text. This algorithm can be used to detect many different useful data: links, phones, ips, prices, times, emails, bitcoin addresses, dates, ipv6s, hex colors and credit cards.When using this extractor, the format of the API output is a bit different than for other extractors, see below how the output looks like.
text <- "Hi, my email is john@example.com and my credit card is 4242-4242-4242-4242 so you can charge me with $10. My phone number is 15555 9876. We can get in touch on April 16, at 10:00am" text2 <- "Hi, my email is mary@example.com and my credit card is 4242-4232-4242-4242. My phone number is 16655 9876. We can get in touch on April 16, at 10:00am" monkey_extract(c(text, text2), extractor_id = "ex_dqRio5sG", unnest = TRUE)
text1 <- "my dog is an avid rice eater" text2 <- "i want to buy an iphone" request <- c(text1, text2) monkey_classify(request, classifier_id = "cl_oFKL5wft")
You can find classifiers and their IDs at https://app.monkeylearn.com/main/explore or you can use the monkeylearn_classifiers
function, choosing to show all classifiers or only the private ones with private = TRUE
. The first column of the resulting data.frame is the classifier_id
to be used in monkeylearn_classify
.
monkeylearn_classifiers(private = FALSE)
Here are a few other examples:
classifier_id = "cl_oJNMkt2V"
. Detect language in text. New languages were added for a total of 48 different languages arranged in language families.text1 <- "Hauràs de dirigir-te al punt de trobada del grup al que et vulguis unir." text2 <- "i want to buy an iphone" text3 <- "Je déteste ne plus avoir de dentifrice." request <- c(text1, text2, text3) monkey_classify(request, classifier_id = "cl_oJNMkt2V")
classifier_id = "cl_KFXhoTdt"
.text1 <- "I think this is awesome." text2 <- "Holy shit! You did great!" request <- c(text1, text2) monkey_classify(request, classifier_id = "cl_KFXhoTdt")
classifier_id = "cl_5icAVzKR"
.text1 <- "Let me tell you about my dog and my cat. They are really friendly and like going on walks. They both like chasing mice." text2 <- "My first R package was probably a disaster but I keep learning how to program." request <- c(text1, text2) monkey_classify(request, classifier_id = "cl_5icAVzKR")
Monkeylearn offers a different service based on your current plan, that is, "free", "team" or "business". These plans will both influence your rate limiting (how fast?) and your query limiting (how many queries?). See https://monkeylearn.com/pricing/. Thanks to your MONKEYLEARN_PLAN environment variable, the rate will be handled automatically thanks to ratelimitr
.
After each call to a function you can check how many calls to the API you can still make using attr(output, "headers")$x.query.limit.remaining
and attr(output, "headers")$x.query.limit.limit
. The period after which attr(output, "headers")$x.query.limit.remaining
depends on your subscription and is not included in the output.
monkeylearn
into your pipeline!You can:
You can classify or extract a vector or dataframe of texts while relating the original input text to its classifications. This is important, because the input:output relationship may not always (and in fact, is not usually) 1:1. These functions retain the tie between each input
[^1] element and all of its output elements.
input <- c("Emma Woodhouse, handsome, clever, and rich, with a comfortable home", "and happy disposition, seemed to unite some of the best blessings of", "existence; and had lived nearly twenty-one years in the world with very", "little to distress or vex her.", "", # <--- note the empty string! "She was the youngest of the two daughters of a most affectionate,", "indulgent father; and had, in consequence of her sister's marriage, been", "mistress of his house from a very early period. Her mother had died", "too long ago for her to have more than an indistinct remembrance of", "her caresses; and her place had been supplied by an excellent woman as", "governess, who had fallen little short of a mother in affection.")
That is true even if you have inputs that cannot be processed. For instance, empty string and NA
input elements are not sent to the API for classification/extraction. (You'll get a warning of this if verbose = TRUE
.) We've got one above to illustrate and elements that returned no classifications/extractions are included in the resulting dataframe. This way you'll know which inputs could not be processed.
(output <- monkey_classify(input, unnest = FALSE))
If there are more than 20 empty inputs, we save your console by messaging only the first 20 indices.
(very_empty_input <- rep("", 25) %>% c(input) %>% sample())
Since the entire original input is represented in the output, if you need to find all of the empty inputs you can easily filter the output to all of the rows containing empty strings.
monkey_classify(very_empty_input, unnest = FALSE)
The default output is a nested dataframe with the same number of rows as your input dataframe or the same length as your input vector, depending on which one you sent in.
Let's take a look at the res
output column.
output$res
You can easily choose an unnested output by setting the unnest flag to TRUE (which it is by default) to get one row per classification/extraction.
(output_unnested <- monkey_classify(input, verbose = FALSE, unnest = TRUE))
We could have gotten the same result by sending in a dataframe and a named column. If a dataframe is supplied input column is not renamed to req
as it is when input is a vector; the original column name is retained.
input_df <- tibble::tibble(text = input) output_df_unnested <- monkey_classify(input_df, text, unnest = TRUE, verbose = FALSE) %>% dplyr::rename(req = text) testthat::expect_equal(output_unnested, output_df_unnested)
If the input is a dataframe, setting the .keep_all
option to TRUE allows you to retain all input columns. If FALSE, only the column you specify for classification will be retained.
sw <- dplyr::starwars %>% dplyr::select(name, height) %>% dplyr::sample_n(nrow(input_df)) sw_input_df <- input_df %>% dplyr::bind_cols(sw) sw_input_df %>% monkey_classify(text, unnest = FALSE, verbose = FALSE)
Retaining the relationship between input and output doesn't mean you'll need to send requests one-by-one. Batch requests by setting the texts_per_req
value which governs the number of texts that are sent per request. Per the MonkeyLearn documentation, the maximum we recommend sending at once is 200 requests.
If texts_per_req
is NULL, the default, we try to optimize the response time from the API by setting texts_per_req
to 200 when your input has more than 200 texts or to the length of the input
if you've got fewer. You'll see a significant speedup by batching your requests this way. However, batching doesn't save you on queries; a batch of 150 texts still uses up 150 queries.
These functions also include some more verbose progress reporting, letting you know what batch you're on out of the total, and which texts are set to be processed in that batch.
one_by_one <- system.time(output <- monkey_classify(input, texts_per_req = 1))
batch_of_five <- system.time(output <- monkey_classify(input, texts_per_req = 5))
How much does sending 5 texts in a batch vs. 1 text improve our processing time?
(speedup <- one_by_one[1] / batch_of_five[1])
A 3-4x speedup isn't so bad! Worth keeping in mind that if you need the blazing fast speeds you might consider upgrading to a higher MonkeyLearn price tier.
monkeylearn
in R doing citation(package = 'monkeylearn')
[^1]: Thanks to Julia Silge's fantastic janeaustenr
package for this text!
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.