pairs_gold_make: create gold snippet pairs for CrowdFlower

In kbenoit/sophistication: Functions to help measure textual sophistication

Description

Create test questions for CrowdFlower from sample of snippet pairs, where the readability is the most different between pairs.

Usage

pairs_gold_make(
  x,
  n.pairs,
  metric = "Flesch",
  min.diff.quantile = c(0.1, 0.9),
  min.diff.absolute = NULL,
  screeners = FALSE,
  verbose = TRUE,
  seed = NULL
)

Arguments

x	snippet pairs
n.pairs	how many gold pairs to form
metric	measure from quanteda.textstats::textstat_readability() to use to compute difference, default is "Flesch"
min.diff.quantile	use this quantile range of difference in the metric only. This is a relative measure that defaults to the upper and lower deciles: c(.1, .9))
min.diff.absolute	form gold pairs with only at least this absolute value difference in the readability measures of the two texts. Only one of min.diff.quantile and min.diff.absolute may be specified.
screeners	if TRUE, add screener text with random instructions
verbose	if TRUE, output status messages during processing
seed	seed to use for random procedures; set this to ensure replicability

Value

a data.frame of gold pairings, with reasons

kbenoit/sophistication documentation built on May 12, 2021, 5:57 a.m.