In r-spark/sparknlp: R Interface to John Snow Labs Spark NLP

This notebook is adapted from John Snow Labs Jupyter/Python getting started notebook. See https://github.com/JohnSnowLabs/spark-nlp-workshop/blob/master/jupyter/annotation/english/match-pattern-pipeline/Pretrained-MatchPattern-Pipeline.ipynb for that version.

• DocumentAssembler
• SentenceDetector
• Tokenizer
• RegexMatcher (match phone numbers)

library(sparklyr)
library(sparknlp)
library(dplyr)

Let's create a Spark connection for our app

version <- Sys.getenv("SPARK_VERSION", unset = "2.4.0")

config <- sparklyr::spark_config()

options(sparklyr.sanitize.column.names.verbose = TRUE)
options(sparklyr.verbose = TRUE)
options(sparklyr.na.omit.verbose = TRUE)
options(sparklyr.na.action.verbose = TRUE)
sc <- sparklyr::spark_connect(master = "local", version = version, config = config)

This Pipeline can extract phone numbers in these formats:

0689912549
[phone]
[phone]
[phone]
(555)-555-5555
555-555-5555
[phone]
[phone]
[phone]
[phone]

pipeline <- nlp_pretrained_pipeline(sc, "match_pattern", lang = "en")

result <- nlp_annotate(pipeline, "You should call Mr. Jon Doe at [phone]")

pull(result, regex)[[1]][[1]][[4]]

result <- nlp_annotate(pipeline, "Ring me up dude! [phone]")

pull(result, regex)[[1]][[1]][[4]]

r-spark/sparknlp documentation built on Oct. 15, 2022, 10:50 a.m.