lsh_candidates only identifies potential matches, but
cannot estimate the actual similarity of the documents. This function takes a
data frame returned by
lsh_candidates and applies a comparison
function to each of the documents in a corpus, thereby calculating the
document similarity score. Note that since your corpus will have minhash
signatures rather than hashes for the tokens itself, you will probably wish
tokenize to calculate new hashes. This can be done for
just the potentially similar documents. See the package vignettes for
A data frame returned by
A comparison function such as
Display a progress bar while comparing documents.
A data frame with values calculated for
1 2 3 4 5 6 7 8
dir <- system.file("extdata/legal", package = "textreuse") minhash <- minhash_generator(200, seed = 234) corpus <- TextReuseCorpus(dir = dir, tokenizer = tokenize_ngrams, n = 5, minhash_func = minhash) buckets <- lsh(corpus, bands = 50) candidates <- lsh_candidates(buckets) lsh_compare(candidates, corpus, jaccard_similarity)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.