compare: Compare fit blockwise with ground truth

In stepR: Multiscale Change-Point Inference

Compare fit blockwise with ground truth

Description

Blockwise comparison of a fitted step function with a known ground truth using different criteria.

Usage

compareBlocks(truth, estimate, dist = 5e3)

Arguments

truth	an object of class stepblock giving the ground truth, or a list of such objects
estimate	corresponding estimated object(s) of class stepblock
dist	a single numeric specifying the distance for at which jumps will be considered as having matched in the qualitative criterion

Value

A data.frame, containing just one row if two single stepblock were given, with columns

true.num, est.num	the true / estimated number of blocks
true.pos, false.pos, false.neg, sens.rate, prec.rate	the number of true / false positive, false negatives, as well as the corresponding sensitivity and precision rates, where an estimated block is considered a true positive if it there is a corresponding block in the ground truth with both endpoints within dist of each other
fpsle	false positive sensitive localization error: for each estimated block's midpoint find into which true block it falls, and sum distances of the respective borders
fnsle	false negative sensitive localization error: for each true block's mid-point find into which estimated block it falls, and sum distances of the respective borders
total.le	total localization error: sum of fpsle and fnsle

Note

No differences between true and fitted parameter values are taking into account, only the precision of the detected blocks is considered; also, differing from the criteria in Elhaik et al.~(2010), no blocks are merged in the ground truth if its parameter values are close, as this may punish sensitive estimators.

Beware that these criteria compare blockwise, i.e. they do not compare the precision of single jumps but for each block both endpoints have to match well at the same time.

References

Elhaik, E., Graur, D., Josić, K. (2010) Comparative testing of DNA segmentation algorithms using benchmark simulations. Molecular Biology and Evolution 27(5), 1015-24.

Futschik, A., Hotz, T., Munk, A. Sieling, H. (2014) Multiresolution DNA partitioning: statistical evidence for segments. Bioinformatics, 30(16), 2255–2262.

See Also

stepblock, stepfit, contMC

Examples

# simulate two Gaussian hidden Markov models of length 1000 with 2 states each
# with identical transition rates being 0.01 and 0.05, resp, signal-to-noise ratio is 5
sim <- lapply(c(0.01, 0.05), function(rate)
  contMC(1e3, 0:1, matrix(c(0, rate, rate, 0), 2), param=1/5))
plot(sim[[1]]$data)
lines(sim[[1]]$cont, col="red")
# use smuceR to estimate fit
fit <- lapply(sim, function(s) smuceR(s$data$y, s$data$x))
lines(fit[[1]], col="blue")
# compare fit with (discretised) ground truth
compareBlocks(lapply(sim, function(s) s$discr), fit)

stepR documentation built on Nov. 14, 2023, 1:09 a.m.