ci_wauc: CI for weighted AUC (wAUC) by replicate weights

View source: R/ci.wauc.R

ci_waucR Documentation

CI for weighted AUC (wAUC) by replicate weights

Description

A function to estimate a confidence interval (CI) for wAUC implemented by wROC::wauc() with replicate weights by survey::as.svrepdesign(); bootstrap is Canty and Davison's bootstrap, subbootstrap is Rao and Wu's (n-1) bootstrap, and JK1 and JKn are jackknife methods.

Usage

ci_wauc(
  y,
  yhat,
  tag.event = "1",
  tag.nonevent = "0",
  weights = NULL,
  cluster = NULL,
  strata = NULL,
  design = NULL,
  data = NULL,
  type = c("JK1", "JKn", "bootstrap", "subbootstrap"),
  B = 2000,
  conf.level = 0.95,
  fpc = NULL,
  fpctype = NULL,
  ...,
  compress = TRUE,
  mse = getOption("survey.replicates.mse")
)

Arguments

y

A vector of response variable or a string of its name (requires data to be supplied).

yhat

A vector of predicted probabilities of y or a string of its name (requires data to be supplied).

tag.event

A character of the event of interest in y, e.g. disease, or condition. Default is "1".

tag.nonevent

A character of the non-event in y, e.g. non-disease, or non-condition. Default is "0".

weights

A character string of a sampling weights' identifier. NULL for either a unweighted case or the sampling design inserted in the design argument.

cluster

A character string of a cluster identifier. It could be NULL if the sampling design were plugged in the design argument.

strata

A character string of a strata identifier. It could be NULL if the sampling design were plugged in the design argument.

design

An object of class survey.design generated by survey::svydesign(). It could be NULL if information about cluster, strata, weights and data were given.

data

A data frame including vectors of y, yhat, and (or) weights. cluster,strata can be included depending on the complex sampling structure. weights = NULL assumes equal probability generating ⁠1's⁠. If data=NULL, then specific numerical vectors must be included in y, yhat, or the sampling design should be plugged into the argument design.

type

Type of replicate weights. Choose one of "JK1", "JKn", "bootstrap", or "subbootstrap". Note: JKn for stratified, JK1 for unstratified designs.

B

The number of bootstrap replicates. Default is 2000.

conf.level

The width of confidence interval. Default is 0.95.

fpc, fpctype

Passed to jk1weights, jknweights, brrweights, bootweights, subbootweights, or mrbweights.

...

optional parameters to be passed to the low level function survey::as.svrepdesign().

compress

Use a compressed representation of the replicate weights matrix.

mse

if TRUE, compute variances from sums of squares around the point estimate, rather than the mean of the replicates.

Value

confidence interval (CI) estimates

See Also

svyROC::wauc() for detailed information on the implementation.

survey::as.svrepdesign() for detailed information on replicate weights methods.

Examples


data(nhanes2013_sbc)

# For ad hoc adjustment for a single PSU stratum not to contribute to the variance
options(survey.lonely.psu='remove')

# yhat would typically come from a fitted model (e.g. wElnet(), wXGBoost(),
# wRandomforest(), or any other classifier). Here we generate one directly
# from a simple weighted logistic regression for a self-contained example.
fit <- glm(HBP ~ INDFMPIR + BMXBMI + BMXARML + BMXLEG + BPXPLS,
          data = nhanes2013_sbc,
          family = binomial(), weights = nhanes2013_sbc$WTSAF2YR)
yhat <- predict(fit, type = "response")

# Conveniently, update a dataset including yhat (a vector) estimated
# from any machine learning methods. HBP, SDMVPSU, SDMVSTRA, and WTSAF2YR
# are referenced by name (not position) to avoid any risk of selecting
# the wrong columns.
data1 <- data.frame(
  nhanes2013_sbc[, c("HBP", "SDMVPSU", "SDMVSTRA", "WTSAF2YR")],
  yhat
)

# For estimating bootstrap 95\% CI for a model selected by dCV
wAUC.ci<- ci_wauc("HBP", "yhat",weights="WTSAF2YR",cluster="SDMVPSU",strata="SDMVSTRA",
                 data=data1,type="subbootstrap",B=2000)
#Or equivalently,

des <- survey::svydesign(ids=~SDMVPSU, strata = ~SDMVSTRA, weights = ~WTSAF2YR,
                              nest = TRUE, data = nhanes2013_sbc)

wAUC.ci<- ci_wauc(des$variables$HBP,data1$yhat,design = des,type = "subbootstrap",B=2000)


MLSurvey documentation built on Nov. 18, 2026, 5:11 p.m.