FuzzExtract: Fuzzy extraction from a sequence

Description Usage Arguments Format Details Methods References Examples

Description

Fuzzy extraction from a sequence

Usage

1
# init <- FuzzExtract$new(decoding = NULL)

Arguments

decoding

either NULL or a character string. If not NULL then the decoding parameter takes one of the standard python encodings (such as 'utf-8'). See the details and references link for more information.

string

a character string.

sequence_strings

a character string vector

contains_dupes

a vector of strings that we would like to dedupe

processor

either NULL or a function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. See the examples for more details.

scorer

a function for scoring matches between the query and an individual processed choice. This should be a function of the form f(query, choice) -> int. By default, FuzzMatcher.WRATIO() is used and expects both query and choice to be strings. See the examples for more details.

limit

An integer value for the maximum number of elements to be returned. Defaults to 5L

score_cutoff

an integer value for the score threshold. No matches with a score less than this number will be returned. Defaults to 0

threshold

the numerical value (0, 100) point at which we expect to find duplicates. Defaults to 70 out of 100

Format

An object of class R6ClassGenerator of length 24.

Details

the decoding parameter is useful in case of non-ascii character strings. If this parameter is not NULL then the force_ascii parameter (if applicable) is internally set to FALSE. Decoding applies only to python 2 configurations, as in python 3 character strings are decoded to unicode by default.

the Extract method selects the best match of a character string vector. It returns a list with the match and it's score.

the ExtractBests method returns a list of the best matches for a sequence of character strings.

the ExtractWithoutOrder method returns the best match of a character string vector (in python it returns a generator of tuples containing the match and it's score).

the ExtractOne method finds the single best match above a score for a character string vector. This is a convenience method which returns the single best choice.

the Dedupe is a convenience method which takes a character string vector containing duplicates and uses fuzzy matching to identify and remove duplicates. Specifically, it uses the Extract method to identify duplicates that score greater than a user defined threshold. Then, it looks for the longest item in the duplicate vector since we assume this item contains the most entity information and returns that. It breaks string length ties on an alphabetical sort. Note: as the threshold DECREASES the number of duplicates that are found INCREASES. This means that the returned deduplicated list will likely be shorter. Raise the threshold for fuzzy_dedupe to be less sensitive.

Methods

FuzzExtract$new(decoding = NULL)
--------------
Extract(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, limit = 5L)
--------------
ExtractBests(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L, limit = 5L)
--------------
ExtractWithoutOrder(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L)
--------------
ExtractOne(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L)
--------------
Dedupe(contains_dupes = NULL, threshold = 70L, scorer = NULL)

References

https://github.com/seatgeek/fuzzywuzzy/blob/master/fuzzywuzzy/process.py, https://docs.python.org/3/library/codecs.html#standard-encodings

Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
if (check_availability()) {


  library(fuzzywuzzyR)

  word = "new york jets"

  choices = c("Atlanta Falcons", "New York Jets", "New York Giants", "Dallas Cowboys")

  duplicat = c('Frodo Baggins', 'Tom Sawyer', 'Bilbo Baggin', 'Samuel L. Jackson',

               'F. Baggins', 'Frody Baggins', 'Bilbo Baggins')

  #------------
  # processor :
  #------------

  init_proc = FuzzUtils$new()

  PROC = init_proc$Full_process    # class process-method

  PROC1 = tolower                  # base R function

  #---------
  # scorer :
  #---------

  init_scor = FuzzMatcher$new()

  SCOR = init_scor$WRATIO


  init <- FuzzExtract$new()

  init$Extract(string = word, sequence_strings = choices, processor = PROC, scorer = SCOR)

  init$ExtractBests(string = word, sequence_strings = choices, processor = PROC1,

                    scorer = SCOR, score_cutoff = 0L, limit = 2L)

  init$ExtractWithoutOrder(string = word, sequence_strings = choices, processor = PROC,

                           scorer = SCOR, score_cutoff = 0L)

  init$ExtractOne(string = word, sequence_strings = choices, processor = PROC,

                  scorer = SCOR, score_cutoff = 0L)

  init$Dedupe(contains_dupes = duplicat, threshold = 70L, scorer = SCOR)


}

fuzzywuzzyR documentation built on May 2, 2019, 8:53 a.m.