R/welnet.par.R

Defines functions print.par.w.elnet par_wElnet

Documented in par_wElnet

#' Parallel cross-validated Weighted Elastic Net across a sequence of alpha values
#'
#' @description Fits [wElnet()] once per value of \code{alpha}, assigning each
#'              alpha to its own CPU core via the \code{parallel} package, and
#'              then selects the overall best (alpha, lambda) combination by
#'              minimizing the average weighted error across all fits.
#'
#'              This function always uses a \code{"PSOCK"} cluster, on every
#'              platform (Windows, Mac, Linux), because it is designed to run
#'              safely inside RStudio. RStudio manages a graphics device and
#'              other GUI state in the main R process; forking that process
#'              (\code{"FORK"} clusters) is explicitly discouraged by R's own
#'              \code{parallel} documentation in GUI-based sessions and can
#'              crash the session. \code{"PSOCK"} workers are fresh, independent
#'              R processes, so this risk does not apply, at the cost of a
#'              small amount of serialization overhead when the cluster starts.
#'
#'              Because this function is part of the \code{MLSurvey} package,
#'              each worker simply attaches \code{MLSurvey} (along with
#'              \code{glmnet} and \code{survey}) via \code{library()}, which
#'              makes [wElnet()], [replicate_weights()], and all of their
#'              internal helper functions available on every worker
#'              automatically -- no manual \code{clusterExport()} of function
#'              definitions is required.
#'
#' @param alpha A numeric vector of elastic net mixing parameters to evaluate
#'              in parallel, one per core. Default is the standard 11-point
#'              grid \code{c(0, 0.001, 0.008, 0.027, 0.064, 0.125, 0.216,
#'              0.343, 0.512, 0.729, 1)}.
#' @param n.cores Integer number of CPU cores to use. Defaults to
#'                \code{min(parallel::detectCores() - 1, length(alpha))}.
#' @param blas.threads Integer number of threads each worker's BLAS library is
#'                      allowed to use. Defaults to \code{1}, to avoid
#'                      oversubscribing CPU cores when \code{n.cores} worker
#'                      processes each also try to run multithreaded linear
#'                      algebra inside \code{glmnet}. Requires the
#'                      \code{RhpcBLASctl} package; if it is not installed,
#'                      this is silently skipped with a message.
#' @param data A data frame with information about the response variable and covariates,
#'             as well as sampling weights, strata, and cluster indicators. Passed to
#'             [wElnet()] for every alpha. It could be \code{NULL} if the sampling design
#'             were supplied in the \code{design} argument instead.
#' @param col.y A numeric value indicating the index of the response variable or a
#'              string of the response variable name.
#' @param col.x A numeric vector indicating indices of covariates or a vector of strings
#'              indicating these column names.
#' @param cluster A string of a cluster identifier. It could be \code{NULL} if the
#'                sampling design were plugged in the \code{design} argument.
#' @param strata A string of a strata identifier. It could be \code{NULL} if the
#'               sampling design were plugged in the \code{design} argument.
#' @param weights A string of a sampling weights' identifier. It could be \code{NULL}
#'                if the sampling design were included in the \code{design} argument.
#' @param design An object of class \code{survey.design} generated by
#'               \code{\link[survey]{svydesign}}. It could be \code{NULL} if information
#'               about \code{cluster}, \code{strata}, \code{weights}, and \code{data}
#'               were given instead.
#' @param family A string indicating a \code{glm} family object for fitting weighted
#'               elastic net models. Choose between \code{gaussian} (linear models) or
#'               \code{binomial} (logistic models).
#' @param lambda A sequence of lambda supplied by a user. If \code{NULL} (default),
#'               [wElnet()] generates its own grid for each alpha.
#' @param nlambda An integer. The number of lambda grid values to generate when
#'                \code{lambda} is \code{NULL}. Default is \code{100}.
#' @param lambda.min.ratio The smallest value for \code{lambda}, as a fraction of the
#'                          maximum lambda. If \code{NULL} (default), [wElnet()]'s own
#'                          internal default is used.
#' @param method A string indicating a method of replicate weights. Choose one of:
#'               \code{JKn}, \code{dCV}, \code{bootstrap}, \code{subbootstrap}, \code{BRR},
#'               \code{split}, \code{extrapolation}.
#' @param k An integer. The number of folds for the \code{dCV} method. Default is \code{k=10}.
#' @param R An integer. The number of times the sample is partitioned for \code{dCV},
#'          \code{split}, or \code{extrapolation} methods. Default is \code{R=1}.
#' @param B An integer. The number of bootstrap re-samples for \code{bootstrap} and
#'          \code{subbootstrap} methods. Default is \code{B=200}.
#' @param dCV.sw.test A logical value indicating the method for estimating the error for
#'                    the \code{dCV} method. Default is \code{FALSE}.
#' @param train.prob A numeric between 0 and 1 setting the proportion of clusters (for
#'                    \code{split}) or strata (for \code{extrapolation}) in training sets.
#'                    Default is \code{train.prob = 0.7}.
#' @param method.split A string indicating which replicate weights method to use under
#'                      the \code{split} method: \code{dCV}, \code{bootstrap}, or
#'                      \code{subbootstrap}.
#' @param print.rw A logical value. If \code{TRUE}, each fitted \code{w.elnet} object's
#'                  replicate-weights data set is retained in its output. Default \code{FALSE}.
#' @param standardize A logical value passed to \code{\link[glmnet]{glmnet}} indicating
#'                     whether predictors should be standardized before fitting. Default
#'                     \code{TRUE}.
#' @param offset An optional offset vector passed to \code{\link[glmnet]{glmnet}}.
#'               Default \code{NULL}.
#' @param ... Any further arguments accepted by \code{\link[glmnet]{glmnet}} (beyond
#'            \code{standardize} and \code{offset}), forwarded to every [wElnet()] call.
#'
#' @return An object of class \code{par.w.elnet}, a list with:
#' - `results`: a named list of \code{w.elnet} objects (the output of [wElnet()]), one per alpha.
#' - `summary`: a data frame with columns \code{alpha}, \code{lambda.min}, and \code{min.error},
#'              one row per alpha, sorted in the order \code{alpha} was supplied.
#' - `best`: the single \code{w.elnet} object with the lowest \code{min.error} across all alphas.
#' - `best.alpha`: the alpha value corresponding to \code{best}.
#'
#' @seealso [wElnet()] for the single-alpha model fit that this function parallelizes.
#'
#' @examples
#' \donttest{
#' library(MLSurvey)
#' data(nhanes2013_sbc)
#' alpha <- c(0.000, 0.001, 0.008, 0.027, 0.064, 0.125, 0.216, 0.343, 0.512, 0.729, 1.000)
#'
#' en.par <- par_wElnet(alpha = alpha,
#'                       data = nhanes2013_sbc,
#'                       col.y = "HBP", col.x = 2:61,
#'                       family = "binomial",
#'                       cluster = "SDMVPSU", strata = "SDMVSTRA", weights = "WTSAF2YR",
#'                       method = "dCV", k = 10, R = 20)
#'
#' en.par$summary      # error per alpha
#' en.par$best.alpha    # winning alpha
#' en.par$best          # winning w.elnet object (same structure as wElnet() output)
#' }
#'
#' @importFrom parallel makeCluster stopCluster clusterEvalQ parLapply detectCores
#' @export
par_wElnet <- function(alpha = c(0.000, 0.001, 0.008, 0.027, 0.064, 0.125, 0.216,
                                 0.343, 0.512, 0.729, 1.000),
                       n.cores = NULL,
                       blas.threads = 1,
                       data = NULL, col.y = NULL, col.x = NULL,
                       cluster = NULL, strata = NULL, weights = NULL, design = NULL,
                       family = c("gaussian", "binomial"),
                       lambda = NULL,
                       nlambda = 100,
                       lambda.min.ratio = NULL,
                       method = c("dCV", "JKn", "bootstrap", "subbootstrap", "BRR", "split", "extrapolation"),
                       k = 10, R = 1, B = 200,
                       dCV.sw.test = FALSE,
                       train.prob = 0.7, method.split = c("dCV", "bootstrap", "subbootstrap"),
                       print.rw = FALSE, standardize = TRUE, offset = NULL, ...){

  family      <- match.arg(family)
  method      <- match.arg(method)
  method.split<- match.arg(method.split)

  if(is.null(data) & is.null(design)){
    stop("Information about either the data set ('data') or the sampling design ('design') needed.")
  }
  if(length(alpha) < 1){
    stop("'alpha' must contain at least one value.")
  }

  n.alpha <- length(alpha)

  # --- Respect CRAN's parallel-processing policy --------------------------
  # R CMD check (and CRAN's own automated checks) set _R_CHECK_LIMIT_CORES_
  # when running examples/tests/vignettes, and parallel::makeCluster() will
  # hard-error via .check_ncores() if more than 2 workers are requested in
  # that context. We detect this and cap n.cores at 2 regardless of what the
  # user supplied, so the \donttest{} example in this file's documentation
  # (and any test suite that calls par_wElnet()) passes under CRAN's checks
  # without requiring the end user to think about this at all.
  check_limit <- Sys.getenv("_R_CHECK_LIMIT_CORES_", "")
  on_cran_check <- nzchar(check_limit) && check_limit != "FALSE"

  if(is.null(n.cores)){
    n.cores <- if(on_cran_check){
      min(2L, n.alpha)
    } else {
      max(1, min(parallel::detectCores() - 1, n.alpha))
    }
  }
  n.cores <- min(n.cores, n.alpha)      # never spin up more workers than alphas
  # hard CRAN ceiling, even if user set n.cores explicitly
  if(on_cran_check) n.cores <- min(n.cores, 2L)


  # Explicitly force every argument that fit_one_alpha() below will capture
  # as a closure. Without this, R's lazy evaluation means some arguments
  # may still be unresolved *promises* (holding an unevaluated expression
  # plus a pointer back to par_wElnet()'s *caller's* environment) at the
  # moment parLapply() serializes fit_one_alpha() to a PSOCK worker. If a
  # promise is shipped unforced, the worker can only resolve it by
  # re-evaluating that expression in the original caller's environment --
  # which does not exist on a separate worker process, producing errors
  # like "object 'col.x' not found" even though col.x was clearly supplied
  # by the caller. force() converts each argument into a concrete value
  # bound directly in par_wElnet()'s own execution frame, which IS
  # correctly captured and serialized along with the closure.
  #
  # This is not optional/defensive-only: it was confirmed to cause real
  # failures under devtools::check() (which loads the package as a properly
  # installed, namespace-isolated package) even though the same code could
  # appear to work fine under devtools::load_all()'s looser scoping.
  force(data); force(col.y); force(col.x)
  force(cluster); force(strata); force(weights); force(design)
  force(family); force(lambda); force(nlambda); force(lambda.min.ratio)
  force(method); force(k); force(R); force(B); force(dCV.sw.test)
  force(train.prob); force(method.split); force(print.rw)
  force(standardize); force(offset)


  # --- Always PSOCK, on every platform -----------------------------------
  # Safe to run inside RStudio on Windows, Mac, and Linux alike: PSOCK
  # workers are independent fresh R processes, so forking the RStudio
  # session's GUI/graphics-device state (a risk with "FORK" clusters) never
  # comes into play.
  message("Fitting ", n.alpha, "alpha value(s) across ", n.cores,
      " CPU core(s) using a PSOCK cluster (safe for RStudio on all platforms)...")

  cl <- parallel::makeCluster(n.cores, type = "PSOCK")
  on.exit(parallel::stopCluster(cl), add = TRUE)

  # Attach MLSurvey (which brings wElnet(), replicate_weights(), and all of
  # their internal helpers along with it), plus glmnet and survey, on every
  # worker. Since this function now lives inside the MLSurvey package itself,
  # there is no need to clusterExport() any function definitions by name --
  # library(MLSurvey) makes the whole package namespace available on each
  # fresh worker process.
  parallel::clusterEvalQ(cl, {
    library(MLSurvey)
    library(glmnet)
    library(survey)
    TRUE
  })

  # Cap each worker's BLAS threads to avoid oversubscribing cores when
  # n.cores workers are each also trying to run multithreaded linear algebra
  # inside glmnet. Skipped silently if RhpcBLASctl isn't installed.
  if(requireNamespace("RhpcBLASctl", quietly = TRUE)){
    parallel::clusterExport(cl, varlist = "blas.threads", envir = environment())
    parallel::clusterEvalQ(cl, {
      RhpcBLASctl::blas_set_num_threads(blas.threads)
      TRUE
    })
  } else {
    message("Package 'RhpcBLASctl' not installed; skipping per-worker BLAS thread limiting. ",
            "Install it with install.packages('RhpcBLASctl') to avoid potential CPU oversubscription.")
  }

  # --- One wElnet() call per alpha, run on its own core -----------------
  fit_one_alpha <- function(a){

    call.args <- list(data = data, col.y = col.y, col.x = col.x,
                      cluster = cluster, strata = strata, weights = weights,
                      design = design, family = family, lambda = lambda,
                      alpha = a, nlambda = nlambda,
                      method = method, k = k, R = R, B = B,
                      dCV.sw.test = dCV.sw.test, train.prob = train.prob,
                      method.split = method.split, print.rw = print.rw,
                      standardize = standardize, offset = offset)

    # Only pass lambda.min.ratio through if the user actually set it;
    # otherwise let wElnet() apply its own internal default.
    if(!is.null(lambda.min.ratio)){
      call.args$lambda.min.ratio <- lambda.min.ratio
    }

    extra.args <- list(...)
    call.args <- c(call.args, extra.args)

    do.call(MLSurvey::wElnet, call.args)
  }

  results <- parallel::parLapply(cl, alpha, fit_one_alpha)
  names(results) <- paste0("alpha_", alpha)

  # --- Summarize across alphas -------------------------------------------
  summary.df <- data.frame(
    alpha      = alpha,
    lambda.min = vapply(results, function(r) r$lambda$min, numeric(1)),
    min.error  = vapply(results, function(r) min(r$error$average), numeric(1))
  )

  best.idx    <- which.min(summary.df$min.error)
  best.result <- results[[best.idx]]

  out <- list(
    results    = results,
    summary    = summary.df,
    best       = best.result,
    best.alpha = alpha[best.idx]
  )
  class(out) <- "par.w.elnet"

  return(out)
}

#' @export
print.par.w.elnet <- function(x, ...){
  cat("Parallel Weighted Elastic Net over", nrow(x$summary), "alpha value(s)\n\n")
  print(x$summary[order(x$summary$alpha), ])
  cat("\nBest alpha:", x$best.alpha, "\n")
  invisible(x)
}

Try the MLSurvey package in your browser

Any scripts or data that you put into this service are public.

MLSurvey documentation built on Nov. 18, 2026, 5:11 p.m.