R/wxgboost.R

Defines functions wXGBoost

Documented in wXGBoost

#' Weighted XGBoost prediction models (wXGBoost) for complex survey data
#'
#'@description A model selection function for fitting XGBoost (linear or logistic) models for complex survey data with sampling weights
#'             by a replicating weights method to select optimal tuning parameters by weighted error minimization.
#'             Detailed arguments are referred to in [xgboost::xgboost()].
#'
#' @param data A data frame with information about the response variable and covariates, as well as sampling weights, strata, and cluster indicators. It could be \code{NULL} if the sampling design were plugged into the \code{design} argument.
#' @param y A vector of response variable.
#' @param col.x A numeric vector indicating indices of the covariates or a string vector indicating these column names.
#' @param cluster A string of a cluster identifier. It could be \code{NULL} if the sampling design were plugged in the \code{design} argument.
#' @param strata A string of a strata identifiers. It could be \code{NULL} if the sampling design were plugged in the \code{design} argument.
#' @param weights A string of a sampling weights' identifier. It could be \code{NULL} if the sampling design were plugged in the \code{design} argument.
#' @param design An object of class \code{survey.design} generated by \code{survey::svydesign()}. It could be \code{NULL} if information about \code{cluster}, \code{strata}, \code{weights} and \code{data} were given.
#' @param params The list of parameters, or the list obtained by [xgboost::xgb.params()].  Should include \code{objective} to be a string of either `binary:logistic` for binary classification or `reg:squarederror` for linear regression.
#'               Default evaluation metrics assigned by \code{params:objective} are weighted mse ([evalerror_reg()]) for regression,
#'               weighted mean error ([evalerror_bin()]) for logistic reg (binary classification).  The complete list of parameters is found on
#'               [xgboost::xgb.train()] or \url{https://xgboost.readthedocs.io/en/latest/parameter.html}.
#' @param maximize If `early_stopping_rounds` is set, then this parameter must be set as well. Default is `FALSE` since the weighted loss function is used for minimization.
#'                 This parameter is passed to the \code{xgboost::cb.early.stop} callback.
#' @param nrounds The maximum number of boosting iterations.
#' @param print_every_n Print each n-th iteration evaluation messages when verbose>0. Default is `1` printing all messages. This parameter is passed to the `cb.print.evaluation` callback.
#' @param verbose  `0` for xgboost to stay silent.  Default= `1` for printing information about performance. `2` for printing some additional information out. Note that setting verbose > 0 automatically engages the `cb.print.evaluation`(period=1) callback function.
#' @param early_stopping_rounds \code{NULL} does not trigger the early stopping function. Setting to an integer k will stop training with a validation set unless the performance improve for k rounds.  Setting this parameter engages the `cb.early.stop` callback.
#' @param missing Default is set to `NA`, which means that `NA` values should be considered as 'missing' by the algorithm.  Sometimes, `0` or other extreme values might be used to represent missing values. This parameter is only used when input is a dense matrix.
#' @param final.model Boolean.  Default is \code{TRUE} printing out the object of class \code{xgb.Booster} for the final model selected by replicate weights' CV with corresponding predicted values.
#' @param test.data Optional. An object of `xgb.DMatrix`. It can be used to fit the final model with the best iteration for evaluation and to estimate predicted values only if `final.model = TRUE`.
#' @param method A character string indicating a method of replicate weights. Choose one of these: \code{JKn}, \code{dCV}, \code{bootstrap}, \code{subbootstrap}, \code{BRR}, \code{split}, \code{extrapolation}.
#' @param k An integer. The number of folds for the \code{dCV} method. Default is \code{k=10}.
#' @param R An integer. The number of times the sample is partitioned for \code{dCV}, \code{split} or \code{extrapolation} method. Default is \code{R=1}.
#' @param B An integer. The number of bootstrap re-samples for \code{bootstrap} and \code{subbootstrap} methods. Default is \code{B=200}.
#' @param dCV.sw.test A logical value indicating the method for estimating the error for \code{dCV} method. \code{FALSE}, (the default option) estimates the error for each test set and defines the cross-validated error based on the average strategy.
#'                   \code{TRUE} estimates the cross-validated error based on the pooling strategy.
#' @param train.prob A numeric between 0 and 1, indicating the proportion of clusters (for the method \code{split}) or strata (for the method \code{extrapolation}) to be set in the training sets. Default is \code{train.prob = 0.7}. Only applies to \code{split} and \code{extrapolation} methods.
#' @param method.split A string of one the following replicate weights methods to be implemented under the \code{split} in the \code{method} argument: \code{dCV}, \code{bootstrap} or \code{subbootstrap}.
#' @param print.rw A logical value. If \code{TRUE}, the data set with the replicate weights is saved in the output object. Default \code{print.rw=FALSE}.
#' @param save_name The name or path for periodically saved model file.
#' @param wxgb_model A previously built model to continue the training from. Could be either an object of class \code{xgb.Booster}, i.e., \code{final.model} from the object of class \code{w.xgboost}, its raw data, or the name of a file with a previously saved model.
#' @param save_period When it is non-NULL, model is saved to disk after every `save_period` rounds, `0` means save at the end. The saving is handled by the `cb.save.model` callback.
#' @param callbacks A list of callback functions to perform various task during boosting. See \code{xgboost::callbacks}. Some of the callbacks are automatically created depending on the parameters’ values.
#'                  Users can provide either existing or their own callback methods in order to customize the training process.
#'
#'
#' @seealso [xgboost::xgb.train()] for relevant arguments and return values in detail.
#' @seealso [eval_loss()] for additional customized function for weighted logloss.
#'
#'
#' @return The output object of the function \code{wXGBoost()} is an object of class \code{w.xgboost}:
#' - `CV.iterations`: A list containing information on tuning parameters, `nrounds` of three elements:
#'   - `niter.range.per.R`: A matrix of the range of iterations of replicate-weight CV for `R` replicates.
#'   - `fold.best`: A list of length `R`; each element \code{fold.best[[r]]} is a numeric vector of best iterations (one per fold/replicate-weight column actually produced for replicate `r`, minimizing the average error). The length of this vector depends on the selected \code{method} and is **not** fixed at \code{k}: it equals \code{k} itself only for \code{method = "dCV"}; for \code{"JKn"} it equals the number of PSUs; for \code{"bootstrap"}/\code{"subbootstrap"} it equals \code{B}; for \code{"BRR"} it is a power-of-4 count determined internally by \code{survey::as.svrepdesign()}; and for \code{"split"}/\code{"extrapolation"} it is \code{1}. (Named \code{fold.best}, not \code{k_fold.best}, precisely because it is not always a "k-fold" count in this general sense -- \code{k} itself only has meaning for \code{dCV}.)
#'   - `best_iteration`: An optimal `nround` that minimizes mean test error.
#' - `CV.eval_log`: A list containing information of two elements:
#'   - `CV`: A matrix of iteration-wise mean and sd for training and test sets, respectively.
#'   - `Best_iteration`: A vector of minimum \code{CV.eval_log$CV} to pick up the final model corresponding to \code{CV.iteratons$best_iteration}.
#' - `final.model`: A list containing information on the fitted model by \code{Best_iteration}: an object of class \code{xgb.Booster}.
#'                  Note that the final model is fitted by the original \code{data}, not one of the generated training sets by replicate weights.
#'                  When `test.data` is provided, the final model is fitted by evaluating model performance.
#' - `predicted`: A vector of predicted values with training set, `data` (`test.data = NULL`) or a list of respective predicted values with training set, `data` and test set, `test.data` from `final.model= TRUE`.
#' - `data.rw`: A data frame containing the original data set and the replicate weights added to define training and test sets. Only included in the output object if \code{print.rw=TRUE}.
#' - `call`:The call that executes this function producing the object of \code{w.xgboost}.
#'
#' @note
#' `final.model` can leverage R-\code{xgboost} functionality for comprehensive data analysis for linear/logistic regression but will be
#' extensively used for other models, i.e., Cox, Poisson, etc.
#'
#' @note
#' \code{method = "bootstrap"} and \code{"subbootstrap"} rely on
#' \code{\link{replicate_weights}}'s out-of-bag (OOB) style test weights
#' (\code{sw_r_1_test_*} columns): for each of the \code{B} replicates, a
#' unit is treated as "test" for that replicate if and only if its PSU was
#' not drawn into that resample. The OOB test fraction per replicate is
#' therefore random rather than a fixed, predetermined proportion (unlike
#' \code{dCV} or \code{split}).
#'
#' @note
#' \strong{Breaking change (if upgrading from an earlier version):} the
#' \code{result$CV.iterations} field previously named \code{k_fold.best} is
#' now named \code{fold.best}, and its contents changed from a pre-sized
#' \code{k x R} matrix to a list of length \code{R} (with per-replicate
#' vectors whose length depends on \code{method}, not fixed at \code{k}).
#' This was a necessary bug fix: the old matrix, pre-sized assuming exactly
#' \code{k} folds regardless of \code{method}, either silently corrupted
#' results (for \code{split}/\code{extrapolation}, where the real fold count
#' is 1, leaving most matrix rows as fake zeros) or crashed with "subscript
#' out of bounds" (for \code{JKn}/\code{BRR}/\code{bootstrap}/
#' \code{subbootstrap}, where the real fold count is almost always greater
#' than the default \code{k=10}). Code that previously read
#' \code{fit$CV.iterations$k_fold.best} as a matrix must be updated to read
#' \code{fit$CV.iterations$fold.best} as a list, e.g.
#' \code{fit$CV.iterations$fold.best[[r]]} for replicate \code{r}'s vector
#' of best iterations.
#'
#' @note
#' \strong{On the argument \code{k}:} \code{k} denotes the number of folds
#' \emph{only} for \code{method = "dCV"} (its only meaningful use, matching
#' \code{\link{replicate_weights}}'s own documentation of \code{k}). For every
#' other \code{method}, \code{k} is ignored entirely (a message is printed if
#' a non-default value is supplied), and the real number of replicate-weight
#' columns used internally is determined by the method itself as described
#' under \code{fold.best} above.
#'
#' @examples
#'
#'  # Global options to avoid only one PSU in a stratum in a particular domain or subpopulation
#'    options(survey.adjust.domain.lonely=TRUE)
#'    options(survey.lonely.psu="adjust")
#'
#'data(nhanes2013_sbc)
#'
#' # Selecting optimal (hyper-) parameters for a model development:
#' param <- list(objective = "binary:logistic", max_depth = sample(4:8, 1),
#'              seed = sample.int(10000,1),
#'              eta = runif(1, .01, .3),gamma = runif(1, 0.0, 1),
#'              subsample = 0.5, # much smaller for large N
#'              colsample_bytree = runif(1, .5, .8),
#'              min_child_weight = sample(1:40, 1),
#'              max_delta_step = sample(1:10, 1))  # for very imbalanced case
#'  # Or equivalently,
#' param<- xgboost::xgb.params(objective = "binary:logistic", max_depth = sample(4:8, 1),
#'                        subsample = 0.5)
#'
#' # Variable selection and best iteration (hyper-parameter) selection:
#' # Can add a xgb.DMatrix object of test data (`test.data`) to the function
#' # to obtain predicted values from the final model after extra evaluation with it.
#' \donttest{
#' xgb.cv <- wXGBoost(data = nhanes2013_sbc,
#'               y = nhanes2013_sbc$HBP, col.x = 2:61,
#'               cluster = "SDMVPSU", strata = "SDMVSTRA", weights = "WTSAF2YR",
#'               params = param, nrounds =10000,verbose=0,early_stopping_rounds=5,
#'               method = "dCV", k=10, R=20)
#'               }
#'\donttest{
#' # Or equivalently:
#' des <- survey::svydesign(ids=~SDMVPSU, strata = ~SDMVSTRA, weights = ~WTSAF2YR,
#'                               nest = TRUE, data = nhanes2013_sbc)
#' xgb.cv <- wXGBoost(y = nhanes2013_sbc$HBP, col.x = 2:61, design = des,
#'               params = param, nrounds =10000,verbose=0,early_stopping_rounds=5,
#'               method = "dCV", k=10, R=20)
#'               }
#'@export
wXGBoost <- function(data = NULL, y =NULL, col.x = NULL,missing = NA,final.model=TRUE,test.data=NULL,
                   cluster = NULL, strata = NULL, weights = NULL, design = NULL,
                   params = list(), nrounds, verbose = 0, print_every_n = 1L,
                   maximize = FALSE, early_stopping_rounds = 8,
                   method = c("dCV", "JKn", "bootstrap", "subbootstrap", "BRR", "split", "extrapolation"),
                   k = 10, R = 1, B = 200, dCV.sw.test = FALSE,
                   train.prob = 0.7, method.split = c("dCV", "bootstrap", "subbootstrap"),
                   print.rw = FALSE, save_period = NULL, save_name = "wxgboost.model",
                   wxgb_model = NULL, callbacks = list()){

  # Stops and messages:
  if(is.null(data) & is.null(design)){stop("Information about either the data set ('data') or the sampling design ('design') needed.")}

  if(method == "split"){
    if(is.null(train.prob)){stop("Selected replicate weights method: 'split'.\nPlease, set a value between 0 and 1 for the argument 'train.prob'.")}
    if(train.prob < 0 | train.prob > 1){stop("Selected replicate weights method: 'split'.\nPlease, set a value between 0 and 1 for the argument 'train.prob'.")}
    if(length(method.split)!=1){stop("Selected replicate weights method: 'split'.\nPlease, set a valid method for the argument 'method.split'. Choose between: 'dCV', 'bootstrap' or 'subbootstrap'.")}
  }

  if(method == "extrapolation"){
    if(is.null(train.prob)){stop("Selected replicate weights method: 'extrapolation'.\nPlease, set a value between 0 and 1 for the argument 'train.prob'.")}
    if(train.prob < 0 | train.prob > 1){stop("Selected replicate weights method: 'extrapolation'.\nPlease, set a value between 0 and 1 for the argument 'train.prob'.")}
  }

  if(method %in% c("JKn", "bootstrap", "subbootstrap", "BRR")){
    if(R!=1){message("Selected method: ", method,". For this method, R = 1. Thus, the argument R = ",R, " has been ignored.")}
  }

  if(method %in% c("dCV", "split", "extrapolation")){
    if(R != round(R)){stop("The argument 'R' must be an integer greater or equal to 1. R=",R," is not an integer.\nPlease, set a valid value for 'R' or skip the argument to select the default option R=1.")}
    if(R < 1){stop("The argument 'R' must be an integer greater or equal to 1. R=",R," lower than 1.\nPlease, set a valid value for 'R' or skip the argument to select the default option R=1.")}
  }

  if(method != "dCV"){
    if(!is.null(k) & k!=10){message("Selected method: ", method,". The argument k = ",k, " is not needed and, hence, has been ignored.")}
  }

  if(method == "dCV"){
    if(k != round(k)){stop("The argument 'k' must be an integer. k=",k," is not an integer.\nPlease, set a valid value for 'k' or skip the argument to select the default option k=10.")}
    if(k < 1){stop("The argument 'k' must be a positive integer. k=",k," is not a positive integer.\nPlease, set a valid value for 'k' or skip the argument to select the default option k=10.")}
  }

  if(!(method %in% c("bootstrap", "subbootstrap"))){
    if(!is.null(B) & B!=200){message("Selected method: ", method,". The argument B = ",B, " is not needed and, hence, has been ignored.")}
  }

  if(method %in% c("bootstrap", "subbootstrap")){
    if(B != round(B)){stop("The argument 'B' must be an integer. B=",B," is not an integer.\nPlease, set a valid value for 'B' or skip the argument to select the default option B=200.")}
    if(B < 1){stop("The argument 'B' must be a positive integer. B=",B," is not a positive integer.\nPlease, set a valid value for 'B' or skip the argument to select the default option B=200.")}
  }



  # Step 0: Notation
  if(!is.null(design)){
    cluster <- as.character(design$call$id[2])
    if(cluster == "1" || cluster == "0"){
      cluster <- NULL
    }
    strata <- as.character(design$call$strata[2])
    weights <- as.character(design$call$weights[2])
    data <- design$variables
  }


  # Step 1: Generate replicate weights based on the method
  newdata <- replicate_weights(data = data, method = method,
                               cluster = cluster, strata = strata, weights = weights,
                               k = k, R = R, B = B,
                               train.prob = train.prob, method.split = method.split,
                               rw.test = TRUE, dCV.sw.test = dCV.sw.test)


  # Step 2: Do CV to find the best iteration
  mat<- as.matrix(data[,col.x])

  xgb_model <- NULL
  Niter<-NULL; Niter.max<-NULL
  # fold.best.iter is a list (length R), not a pre-sized matrix: the
  # number of folds/replicate-weight columns actually produced per
  # replicate 'r' depends on the selected method (k for dCV, number of
  # PSUs for JKn, B for bootstrap/subbootstrap, a power-of-4 count for
  # BRR, 1 for split/extrapolation) and is not necessarily equal to the
  # 'k' argument, which is specific to dCV's fold count. A matrix
  # pre-sized with nrow = k would silently corrupt output (when the real
  # fold count < k) or error with "subscript out of bounds" (when the
  # real fold count > k, e.g. almost always true for JKn/BRR/bootstrap).
  # Named fold.best (not k_fold.best) since it does not represent a
  # "k-fold" count in this general sense -- 'k' itself is only meaningful
  # for method = "dCV".
  fold.best.iter <- list()
  k.fold.error <- list();

  #attach eval.nonpara.R for testing measure
  # param custom_metric User defined evaluation function. Returns list(metric='metric-name', value='metric-value') with given prediction and dtrain.
  #                     Default evaluation metric is assigned by \code{params:objective}: weighted mse ([evalerror_reg()]) for regression,
  #                     weighted mean error ([evalerror_bin()]) for logistic reg or binary classification.
  # if(is.null(custom_metric)){
   if(params$objective == "binary:logistic"){
        eval.metric<- evalerror_bin
          # eval.metric<- eval.loss
    }else {
       eval.metric<- evalerror_reg
    }
  # }else {
  #     eval.metric<-custom_metric
  #   }
for (r in 1:R){
    rwtraincols <- grep(paste0("r_",r,"_train"), colnames(newdata))
    rwtestcols <- grep(paste0("r_",r,"_test"),names(newdata))
    eval.log<-list();total.iter <- NULL;

    # Actual fold/replicate count for this r, used throughout instead of
    # the dCV-specific 'k' argument.
    n.folds <- length(rwtraincols)
    fold.best.iter[[r]] <- numeric(n.folds)

  for(k.w in 1:n.folds){

      #generate supporting type data
      dtrain<- xgb.DMatrix(data = mat, label = y, weight = newdata[,rwtraincols[k.w]], nthread = params$nthread, missing = missing)
      dtest <- xgb.DMatrix(data = mat, label = y, weight = newdata[,rwtestcols[k.w]], nthread = params$nthread, missing = missing)
      watchlist <- list(train=dtrain, eval=dtest)

    model <- xgboost::xgb.train(params = params, data = dtrain, evals = watchlist,
                               nrounds=nrounds, custom_metric = eval.metric,
                              verbose = verbose, print_every_n = print_every_n,
                              early_stopping_rounds = early_stopping_rounds, maximize = maximize,
                              save_period = save_period, save_name = save_name,
                              xgb_model = wxgb_model, callbacks = list())

       eval.log[[length(eval.log)+1]]<-attributes(model)$evaluation_log
       names(eval.log)[[length(eval.log)]]<- names(newdata)[rwtraincols[k.w]]


    # if(!is.null(early_stopping_rounds)){
        fold.best.iter[[r]][k.w] <- attributes(model)$early_stop$best_iteration
        total.iter <- c(total.iter,dim(attributes(model)$evaluation_log)[1])
      # } else{ total.iter <- model$niter
      # }

  }
    #find min error per r to pick best nrounds from the replicate-weight CV

    min.niter<- min(total.iter)
      train.eval<- NULL
      test.eval <- NULL
    # Use the actual number of folds produced for this replicate
    # (length(eval.log), == n.folds above), not the 'k' argument -- see
    # note on k.fold.best.iter above for why this must not be hardcoded.
    for (j in 1:n.folds){
          train.eval <- cbind(train.eval,eval.log[[j]][1:min.niter,2])
          test.eval <- cbind(test.eval,eval.log[[j]][1:min.niter,3])
    }
    #collect min niters and corresponding iteration-wise mean for k-folds per r
      Niter<- c(Niter, min.niter); Niter.max<-c(Niter.max,max(total.iter))
      k.fold.error[[r]] <- cbind(apply(train.eval,1,mean,na.rm=TRUE),apply(test.eval,1,mean,na.rm=TRUE))
}
    min_nstops <- min(Niter)
    train.mean.error<-NULL;test.mean.error<-NULL
    for(r in 1:R){
     train.mean.error<- cbind(train.mean.error,k.fold.error[[r]][1:min_nstops,1])
     test.mean.error<- cbind(test.mean.error,k.fold.error[[r]][1:min_nstops,2])
    }

     mean.train.error<- apply(train.mean.error,1,mean,na.rm=TRUE)
     sd.train.error <- apply(train.mean.error,1,sd,na.rm=TRUE)
     mean.test.error <- apply(test.mean.error,1,mean,na.rm=TRUE)
      sd.test.error <- apply(test.mean.error,1,sd,na.rm=TRUE)

    Eval.log <- cbind(mean.train.error,sd.train.error,mean.test.error,sd.test.error)
    Eval.log<- data.frame(itr=1:min_nstops,Eval.log)

    best_iter <- which.min(mean.test.error)


    #step 3: run the final model
    if(final.model){
     #generate supporting data type
    dtrain<- xgb.DMatrix(data = mat, label = y, weight = data[,weights],nthread = params$nthread, missing=missing )

    # Always include at least 'train' in eval.list. This ensures 'evals' is
    # never empty -- required by xgboost whenever early_stopping_rounds is
    # non-NULL. We also disable early stopping for this final fit below,
    # since nrounds = best_iter was already determined to be optimal by the
    # replicate-weights CV above; there is nothing left to "stop early" on,
    # and re-enabling it here could only truncate training before best_iter.
    if(is.null(test.data)){
      eval.list<- list(train = dtrain)
    }else{ #!is.null(test.data) case:
      eval.list <-list(train=dtrain,test=test.data)
       }

    model <- xgboost::xgb.train(params = params, data = dtrain, evals = eval.list,
                               nrounds=best_iter, custom_metric = eval.metric,
                              verbose = verbose, print_every_n = print_every_n,
                              maximize = maximize,early_stopping_rounds = NULL,
                              save_period = save_period, save_name = save_name,
                               callbacks = list())

    if(is.null(test.data)){
      yhat <- predict(model, newdata=dtrain)
    }else{
      yhat<-list(train = predict(model, newdata=dtrain),
                 test = predict(model, newdata=test.data) )
    }

 }
  result <- list()
  result$CV.iterations <- list(niter.range.per.R = rbind(Niter,Niter.max),
                               fold.best=fold.best.iter,
                               best_iteration = best_iter)
  result$CV.eval_log <- list(CV = Eval.log,
                             Best_iteration =  Eval.log[best_iter,])
  if(final.model) {result$final.model <- model
                   result$predicted <- yhat}

  result$call <- match.call()

  if(print.rw == TRUE){result$data.rw <- newdata}

  class(result) <- "w.xgboost"

  return(result)

}

Try the MLSurvey package in your browser

Any scripts or data that you put into this service are public.

MLSurvey documentation built on Nov. 18, 2026, 5:11 p.m.