evaluate_py: run evaluation via python scripts

In gobbios/avutils: various utilities for DiViMe tools and audio processing

Description

run evaluation via python scripts

Usage

evaluate_py(test, reference, metric = "accuracy", task = "detection",
  prefix = NULL, check_python = FALSE, set_ident_flag = FALSE,
  processed_output = TRUE, progressbar = FALSE)

Arguments

test	character, path to test file (must be .rttm), typically with prefix
reference	character, path to reference file (must be .rttm), typically without prefix
metric	character, the metric to be calculated
task	character, the task
prefix	character, the file prefix for the test file (the default option NULL tries to guess it from the file name)
check_python	logical, test whether python is working
set_ident_flag	logical, should the '–identification' flag be set for the call to the python script
processed_output	logical, should the process output be returned (a data.frame) or the raw textual output from the console (a list). Default is TRUE.
progressbar	logical, display a progress bar (default is FALSE)

Details

So far the function only works for a small subset of evaluation task/metric combinations:

• detection/accuracy

• detection/precision

• detection/recall

• identification/ider

• identification/precision

• identification/recall

• diarization/diaer

Pairs of test/reference are determined by the rttm_pairs function.

Also, the function is not very efficient because it has to initialize python for each test/reference pair.

Value

a data.frame with one row per pair of test/reference and a varying number of columns (depending on task and metric).

Examples

## Not run: 
dataloc <- dirname(system.file("spanish.rttm", package = "avutils"))
# list three reference files, one containing a 'typo'
reference <- file.path(dataloc, c("BER_0485_12_07_09123.rttm", "spanish.rttm", "filewithtypo.rttm"))
# and a whole bunch of potential test files (all rttm in the same location)
test <- list.files(dataloc, pattern= ".rttm$", full.names = TRUE)

# will produce a warning that there was one file that couldn't be processed...
res <- evaluate_py(test = test, reference = reference, metric = "accuracy", task = "detection",
                   progressbar = TRUE)
res

## End(Not run)

gobbios/avutils documentation built on Feb. 19, 2020, 9:44 a.m.