Description Usage Arguments Value Note Examples
View source: R/validate_model.R
validate_model
- Check how well a regex model is tagging using human
interaction to assess the model.
assign_validation_task
- Create human assignments to assess how well a
model is functioning. The coder can use the correct
column to assess
how well the tag
fits the text
columns.
1 2 3 4 5 6 7 8 9 10 11 | validate_model(x, n = 20, width = 50, tags = 1, filter = NULL, ...)
assign_validation_task(
x,
n = 20,
checks = 1,
coders = "coder",
out = NULL,
as.list = TRUE,
...
)
|
x |
A |
n |
The number of samples to take from each regex tag assignment. Tags
with less than |
width |
The width of the text display. |
tags |
The number of classifications per row/element to allow. Ties are broken probabilistically by default. |
filter |
Validate a subset of the original tags. Useful for when the user finds a mistake in the classifier and wants to retest only portions of the model. |
checks |
The number of coders needed per tag assignment. |
coders |
A vector of coders to assign tasks to. |
out |
A directory name to create and output csv file(s) to. |
as.list |
logical. Should the assignments be dsplayed as a list of
|
... |
Other arguments passed to |
validate_model
- Returns a data.frame
of the class
'validate_model'
. Note that the pretty print is a tag summarized
version of the model accuracy standard error, and confidence intervals from
summary.validate_model
. n.tagged
is the number of potential
tags from the column sums of the termco
object whereas n.classified
is the number of elements actually classified into that tag group by the
classify
function.
assign_validation_task
- Returns a data.frame
/.csv or
list
of data.frame
s/.csvs. Columns in the data.frame
s
include:
coder |
The assgned coder (person for the task). |
index |
The row/element number of the text. |
correct |
A blank column for coders to dummy/logical code if the tag assignment for that text was accurate. |
tag |
The tag that was assigned to the text. |
text |
The text to which the tag was assigned. |
This function assigns tags using the classify
function. One element may recieve multiple tags.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | ## Not run:
data(presidential_debates_2012)
discoure_markers <- list(
response_cries = c("\\boh", "\\bah", "\\baha", "\\bouch", "yuk"),
back_channels = c("uh[- ]huh", "uhuh", "yeah"),
summons = "hey",
justification = "because"
)
## A model (note: `grouping.var = TRUE` to make a model)
(x <- with(presidential_debates_2012,
term_count(dialogue, grouping.var = TRUE, term.list = discoure_markers)
))
## Requires interaction
out <- validate_model(x)
out
plot(out)
## Validate a subset of the model
out2 <- validate_model(x, filter = c('response_cries', 'summons'))
out2
plot(out2)
## Assign tasks externally
assign_validation_task(x, checks = 3,
coders = c('fred', 'jade', 'sally', 'jim', 'shelly'), out='testing')
assign_validation_task(x, checks = 3,
coders = c('fred', 'jade', 'sally', 'jim', 'shelly'), as.list = FALSE,
out='testing2')
## End(Not run)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.