resampleScoreModelPair: Studentized bootstrap test of strength of...

In sigr: Succinct and Correct Statistical Summaries for Reports

Studentized bootstrap test of strength of scoreFn(yValues,model1Values) > scoreFn(yValues,model1Values).

Description

Studentized bootstrap test of strength of scoreFn(yValues,model1Values) > scoreFn(yValues,model1Values) sampled with replacement.

Usage

resampleScoreModelPair(
  model1Values,
  model2Values,
  yValues,
  scoreFn,
  ...,
  na.rm = FALSE,
  returnScores = FALSE,
  nRep = 100,
  parallelCluster = NULL,
  sameSample = FALSE
)

Arguments

model1Values	numeric array of predictions (model to test).
model2Values	numeric array of predictions (reference model).
yValues	numeric/logical array of outcomes, dependent, or truth values
scoreFn	function with signature scoreFn(modelValues,yValues) returning scalar numeric score.
...	not used, forces later arguments to be bound by name.
na.rm	logical, if TRUE remove NA values
returnScores	logical if TRUE return detailed resampledScores.
nRep	integer number of repititions to perform.
parallelCluster	optional snow-style parallel cluster.
sameSample	logical if TRUE use the same sample in computing both scores during bootstrap replication (else use independent samples).

Details

True confidence intervals are harder to get right (see "An Introduction to the Bootstrap", Bradely Efron, and Robert J. Tibshirani, Chapman & Hall/CRC, 1993.), but we will settle for simple p-value estimates.

Value

summaries

Examples

set.seed(25325)
y <- 1:5
m1 <- c(1,1,2,2,2)
m2 <- c(1,1,1,1,2)
cor(m1,y)
cor(m2,y)
f <- function(modelValues,yValues) {
  if((sd(modelValues)<=0)||(sd(yValues)<=0)) {
    return(0)
  }
  cor(modelValues,yValues)
}
resampleScoreModelPair(m1,m2,y,f)

sigr documentation built on Aug. 20, 2023, 9:06 a.m.