imbalanced.rfsrc: Random Forests for Imbalanced Two-Class Classification

In randomForestSRC: Fast Unified Random Forests for Survival, Regression, and Classification (RF-SRC)

Random Forests for Imbalanced Two-Class Classification

Description

Fits a random forest for a two-class outcome using the random forest quantile classifier (RFQ), balanced random forest (BRF), or standard classification. RFQ changes the probability threshold used to assign class labels; BRF changes the sampling used to grow the trees. The splitting rule, performance measure, and variable importance can be specified separately.

Companion helpers summarize performance, select a probability threshold, or extract the class-frequency threshold from response labels.

Usage

## S3 method for class 'rfsrc'
imbalanced(formula, data, ntree = 3000, 
  method = c("rfq", "brf", "standard"), splitrule = "auc",
  perf.type = NULL, block.size = NULL, fast = FALSE,
  ratio = NULL, ...)

get.imbalanced.performance(obj, prob = NULL, threshold = NULL,
                           confusion = FALSE, robust = FALSE)
get.imbalanced.optimize(obj, prob = NULL, newdata = NULL,
                        measure = c("gmean", "F1", "F1mod", "F1modgmean"),
                        ngrid = 1000, plot.it = TRUE)
get.rfq.threshold(y)

## S3 method for class 'imbalanced.performance'
print(x, digits = 4,
                                      show.confusion = TRUE, ...)

Arguments

formula	A model formula with one factor response having two levels and one or more predictors.
data	A data frame containing the response and predictors. Data-frame subclasses are converted to a plain data frame.
ntree	Number of trees to grow. The default is 3000.
method	Classification method. "rfq", the default, uses the RFQ decision rule of O'Brien and Ishwaran (2019). "brf" uses class-weighted sampling with replacement and a reduced tree sample size. "standard" uses ordinary random forest classification without enabling the RFQ rule. See Details.
splitrule	Splitting rule passed to the forest grow function. The default is "auc". Other choices include "gini" and "entropy"; see rfsrc for supported classification rules. This setting is separate from perf.type.
perf.type	Performance measure used by the forest and its variable importance calculations. The default is "gmean" for RFQ and BRF, and "default" (misclassification error) for standard classification. Choices accepted by this wrapper are "none", "default", "standard", "misclass", "brier", "gmean", and the legacy spelling "g.mean". The latter is converted to "gmean". Use "none" to omit forest performance calculations.
block.size	Block size passed to the selected forest grow function for performance and VIMP calculations; see rfsrc. This wrapper passes NULL explicitly when the argument is not set. Specify the same block size when comparing VIMP across methods.
fast	Logical. Use rfsrc.fast for the RFQ or standard method when TRUE. The BRF branch always calls rfsrc and does not use this argument.
ratio	Optional, experimental majority-class undersampling for the RFQ and standard methods. Supply a number between zero and one. The majority class is sampled without replacement, while the minority class is bootstrapped with replacement. Small requested fractions may be increased by the sampling helper. This is a majority-class sampling fraction, rather than a requested final minority prevalence. Ignored for BRF. See Details for OOB support.
obj	For the performance and optimization helpers, a grow or prediction object with a single two-class outcome and observed responses, or a two-level response factor when prob is supplied.
prob	Optional probabilities aligned with the responses in obj. A numeric vector or one-column matrix or data frame contains minority-class probabilities. A two-column matrix or data frame contains both class probabilities; named columns are matched to factor levels, and unnamed columns follow factor-level order. NULL uses the object's predictions as described in Details.
threshold	Cutoff for minority-class probabilities in get.imbalanced.performance. NULL, the default, uses the minority proportion among the observed evaluation responses. "auto" uses that prevalence for an RFQ object and 0.5 otherwise. "bayes" always uses 0.5. A single finite numeric value specifies the cutoff directly; values outside [0,1] are clipped to that interval. With direct response and probability inputs, "auto" uses 0.5 because no fitted classifier is supplied.
confusion	Return a list including the confusion matrix instead of the default named vector of performance measures?
robust	Add one to the diagonal counts when calculating the four classification rates and their derived summaries? See Details.
newdata	Optional threshold-selection data for get.imbalanced.optimize, supplied with a grow object and prob = NULL. Must contain the predictors and observed response. NULL uses predictions already in obj. Ignored when prob is supplied.
measure	Performance measure to maximize during threshold selection: "gmean" (the default), "F1", "F1mod", or "F1modgmean". See Details for definitions.
ngrid	Number of equally spaced thresholds from zero to one, including both endpoints. Must be an integer of at least two.
plot.it	Draw the four threshold-search diagnostic panels?
y	Class-label vector for get.rfq.threshold, typically a two-level factor. Missing responses are omitted when counting classes.
...	Additional arguments passed to rfsrc or, when selected, rfsrc.fast. Examples include mtry, nodesize, nsplit, importance, and na.action. BRF sets its own case.wt, sampsize, and samptype. When ratio is supplied, the wrapper constructs samp and sets bootstrap = "by.user". Additional arguments to the print method are currently unused.
x	For the print method, the result of get.imbalanced.performance.
digits	Number of decimal places used in the performance display.
show.confusion	Print model and expected random-reference confusion tables when a confusion matrix is present in x? Obtain that matrix with confusion = TRUE in get.imbalanced.performance. This option does not recompute it.

Details

Choice of classifier

Let \pi be the training proportion of the minority class and \widehat p(x) its predicted probability. RFQ assigns an observation to the minority class when \widehat p(x) \geq \pi. The class-frequency threshold can identify minority observations whose predicted probability is below one half. The standard method instead uses ordinary class assignment based on the largest predicted probability. RFQ here is a classification rule, distinct from the continuous-response quantile estimation performed by quantreg.

BRF calls the grow function with replacement sampling, a tree sample size of twice the smaller observed class count, and case weights that give the two classes equal total sampling weight. The wrapper does not construct separate fixed-size samples from each class. Both fast and ratio are ignored in this branch.

Performance and variable importance

G-mean is the geometric mean of minority sensitivity and majority specificity, \sqrt{\mathrm{sensitivity}\, \mathrm{specificity}}. Larger values indicate better balance between the two class-specific rates. Misclassification and Brier scores are error measures for which smaller values are better. Changing perf.type selects the forest performance measure; it does not select the RFQ decision rule.

Request permutation VIMP explicitly with importance = "permute". For comparisons, use a common perf.type, splitting rule, and block.size, unless the comparison is intended to study those settings. Increase ntree as needed to assess the stability of OOB performance and VIMP.

Sampling and missing values

With ratio = NULL, the RFQ and standard branches pass missing-data handling to the selected grow function. BRF and calls using ratio first apply complete-case omission to the supplied data frame. This omission currently includes columns that are not selected by the formula and occurs before the grow function handles na.action.

With ratio supplied, each tree uses a bootstrap sample of the minority observations and a without-replacement sample of majority observations. At ratio = 1, every majority observation is in-bag for every tree, so majority-class OOB performance is unavailable. A single minority observation is also always in-bag in this sampling scheme. Use a separate evaluation sample when OOB support is absent.

Detailed performance summaries

get.imbalanced.performance accepts a fitted or predicted forest object, or a two-level factor supplied as obj together with prob. For a forest object, it prefers predicted.oob when that component exists and is not entirely missing; otherwise it uses predicted. A partially missing OOB component is not filled with full-ensemble predictions.

A numeric vector or one-column matrix or data frame represents minority-class probabilities, with one value per response. No column name is needed for these inputs. A two-column matrix or data frame contains both class probabilities. Named columns are matched to the response factor levels; unnamed columns follow factor-level order. The response and probability rows must already be aligned.

Finite probabilities are restricted to [0,1] using the existing clipping rule. Finite two-column rows must sum to one within numerical tolerance. Rows with a missing response or a nonfinite probability are excluded from scoring; both columns must be finite for a two-column input. Unavailable scores are returned as NA. Class counts, minority identity, and the default threshold are determined from all observed evaluation responses before excluding unavailable predictions.

The helper determines the minority class and its proportion from the response supplied for evaluation. If the class counts tie, the first factor level is selected. The default threshold = NULL uses this evaluation proportion for all forest methods.

With threshold = "auto", the saved forest$rfq flag selects the cutoff: TRUE uses the same evaluation prevalence as NULL, and otherwise the cutoff is 0.5. This applies to both grow and prediction objects. For RFQ test predictions the prevalence is therefore calculated from the evaluation responses, not substituted from the training responses. Splitting rules and perf.type do not determine whether the classifier is RFQ. Direct response/probability inputs have no fitted-classifier flag, so "auto" uses 0.5. Use threshold = "bayes" to request 0.5 explicitly for any classifier.

All settings threshold the probabilities rather than reading the stored class component. Supply a single finite numeric threshold to examine a different cutoff; finite values outside [0,1] are clipped to that interval. Threshold ties are assigned to the minority class. A supplied threshold changes the cutoff, not how the minority class is identified or how the forest was fitted.

Besides the four rates sens, spec, prec, and npv, the summary includes misclass, brier, brier.norm, auc, logloss, and pr.auc. For complementary binary probabilities, brier is the mean squared error of the minority probability and brier.norm is four times this value. The current log-loss helper excludes infinite losses rather than clipping zero probabilities.

F1 is the harmonic mean of precision and sensitivity; F1mod is the harmonic mean of sensitivity, specificity, precision, and negative predictive value. F1gmean averages F1 and G-mean; F1modgmean averages F1mod and G-mean. A harmonic mean is zero when all its component rates are defined and at least one is zero. It is NA when any required rate is undefined.

robust = TRUE adds one to the two diagonal counts when calculating the four rates and their derived summaries. It does not change the returned confusion counts, misclassification error, or probability-based scores. Use confusion = TRUE to return the confusion matrix in addition to the metrics. Its rows are observed classes and columns are predicted classes, with 0 denoting the majority and 1 the minority.

Random-score reference

The performance summary has a "rand" attribute containing reference metrics and expected confusion counts. This reference uses a score U uniformly distributed on [0,1], independent of class, and predicts the minority class when U reaches the chosen threshold. For threshold t, its sensitivity is 1-t, specificity is t, and ROC AUC is 1/2. The separately returned PR-AUC reference is the proportion of positive observations among the rows scored by the PR helper. Its unnormalized Brier, normalized Brier, and log-loss references are 1/3, 4/3, and 1, respectively. These differ from a predictor that always returns probability 1/2.

Nonlinear rate summaries are computed from the reference rates (or smoothed expected counts under robust = TRUE); they are not finite-sample expectations of those nonlinear statistics. The performance print method compares the model and reference with Delta and percentage Gain, oriented so that positive values indicate improvement. The fitted-forest printer has its own probability reference; it need not use the same baseline.

Threshold selection

get.imbalanced.optimize searches ngrid equally spaced thresholds from zero to one, using G-mean, F1, F1mod, or F1modgmean as selected by measure. It uses the first grid point attaining the largest available value. It returns one row of performance metrics at the selected threshold and does not change the forest or its stored class predictions. plot.it = FALSE suppresses its four diagnostic panels. ngrid must be an integer of at least two. When the chosen measure is unavailable at every grid point, the function stops with an explanatory error. A valid optimum of zero is retained. The caller's graphics settings are restored after plotting.

A grow object ordinarily supplies OOB probabilities for this search. Supplying newdata searches using that data's observed responses and predictions, making it a threshold-selection sample. For an independent assessment of the chosen threshold, apply it unchanged to a separate test sample. The examples select a threshold from training OOB predictions and carry it forward to the test data.

Printing performance

Printing a performance result displays two metric tables. The first contains classification rates and their F1 and G-mean combinations; the second contains overall performance, including G-mean, AUC, PR-AUC, misclassification, Brier scores, and log loss. The same G-mean row appears in both tables; F1-related combinations appear only in the first.

Delta and percentage Gain compare the model with its random-score reference, with positive values indicating improvement. Larger rates, G-mean, and AUC values are better; smaller errors and losses are better. digits controls the display precision, and show.confusion = FALSE omits the optional confusion tables. These settings do not change the stored performance values.

Value

imbalanced returns an object from the selected forest grow function. Usual components include the class-probability matrices predicted and predicted.oob, class assignments class and class.oob, and requested performance and importance values. Availability depends on the selected grow interface and sampling options; see rfsrc and rfsrc.fast.

Performance and threshold helpers

get.imbalanced.performance

A named numeric vector of class counts, imbalance ratio, resolved numeric threshold, and performance measures. With confusion = TRUE, a named list also includes confusion, the confusion matrix with a class.error column. Both forms have class "imbalanced.performance" and a "rand" attribute containing the random-reference metrics and expected confusion counts.

get.imbalanced.optimize

A one-row data frame of performance measures at the selected threshold. Extract the cutoff from the threshold column; the row name is its grid index.

get.rfq.threshold

The smaller class count divided by the total observed class count, as a numeric scalar. Returns NULL unless the class-frequency table has exactly two entries. No forest or probability predictions are required.

The performance and optimization helpers return NULL when the evaluation response is not a two-level factor.

print.imbalanced.performance displays the summaries and returns x invisibly.

Author(s)

Hemant Ishwaran and Udaya B. Kogalur

References

Chen, C., Liaw, A. and Breiman, L. (2004). Using random forest to learn imbalanced data. University of California, Berkeley, Technical Report 110.

Kubat, M., Holte, R. and Matwin, S. (1997). Learning when negative examples abound. Machine Learning, ECML-97: 146-153.

O'Brien, R. and Ishwaran, H. (2019). A random forests quantile classifier for class imbalanced data. Pattern Recognition, 90, 232-249.

See Also

rfsrc, rfsrc.fast, get.imbalanced.performance, subsample

Examples

## ------------------------------------------------------------
## RFQ, BRF, and standard classification
## ------------------------------------------------------------
data(breast, package = "randomForestSRC")
dta <- na.omit(breast)
f <- status ~ .

set.seed(19)
rfq <- imbalanced(f, dta, ntree = 300)
print(rfq)
print(get.imbalanced.performance(rfq, confusion = TRUE))

## Extract the class-frequency threshold from the observed responses.
print(get.rfq.threshold(rfq$yvar))

brf <- imbalanced(f, dta, ntree = 300, method = "brf")
standard <- imbalanced(f, dta, ntree = 300, method = "standard")

## Compare all three using the helper's default evaluation rule.
metrics <- c("sens", "spec", "gmean", "misclass", "brier", "auc")
comparison <- rbind(
  RFQ = as.numeric(get.imbalanced.performance(rfq)[metrics]),
  BRF = as.numeric(get.imbalanced.performance(brf)[metrics]),
  Standard = as.numeric(get.imbalanced.performance(standard)[metrics])
)
colnames(comparison) <- metrics
print(comparison)

## Supply a cutoff explicitly to inspect another probability rule.
print(get.imbalanced.performance(standard, threshold = .5))

## Let the saved classifier setting select prevalence or 0.5.
print(get.imbalanced.performance(rfq, threshold = "auto"))
print(get.imbalanced.performance(standard, threshold = "auto"))
print(get.imbalanced.performance(brf, threshold = "auto"))

## Request the 0.5 Bayes cutoff explicitly, including for RFQ.
print(get.imbalanced.performance(rfq, threshold = "bayes"))

## Vectors, one-column objects, and named two-column matrices.
class.frequency <- table(rfq$yvar)
minority <- names(class.frequency)[which.min(class.frequency)]
p <- rfq$predicted.oob[, minority]
print(get.imbalanced.performance(rfq$yvar, p))
print(get.imbalanced.performance(rfq$yvar, data.frame(p = p)))
print(get.imbalanced.performance(
  rfq$yvar, rfq$predicted.oob[, rev(levels(rfq$yvar)), drop = FALSE]
))

## ------------------------------------------------------------
## Train/test assessment with a training-selected threshold
## ------------------------------------------------------------
set.seed(23)
trn.id <- sort(unlist(lapply(
  split(seq_len(nrow(dta)), dta$status),
  function(idx) idx[sample.int(length(idx),
    size = floor(2 * length(idx) / 3))]
)))
trn <- dta[trn.id, , drop = FALSE]
tst <- dta[-trn.id, , drop = FALSE]
print(rbind(training = table(trn$status), test = table(tst$status)))

fit <- imbalanced(f, trn, ntree = 300)
selected <- get.imbalanced.optimize(fit, measure = "gmean",
                                     ngrid = 501, plot.it = FALSE)
print(selected)
threshold.selected <- selected$threshold

pred <- predict(fit, newdata = tst)
print(get.imbalanced.performance(pred))
print(get.imbalanced.performance(pred, threshold = threshold.selected))

## Predicting labels for new rows does not require their outcomes.
class.frequency <- table(trn$status)
minority <- names(class.frequency)[which.min(class.frequency)]
majority <- setdiff(levels(trn$status), minority)
pred.x <- predict(fit, newdata = tst[, fit$xvar.names, drop = FALSE])
selected.class <- factor(
  ifelse(pred.x$predicted[, minority] >= threshold.selected,
         minority, majority),
  levels = levels(trn$status)
)
print(head(selected.class))

## ------------------------------------------------------------
## Explicit permutation VIMP and blocking
## ------------------------------------------------------------
rfq.vimp <- imbalanced(f, dta, ntree = 300,
                       importance = "permute", block.size = 1)
rfq.block <- imbalanced(f, dta, ntree = 300,
                        importance = "permute", block.size = 10)
print(cbind(unblocked = rfq.vimp$importance[, 1],
            blocked = rfq.block$importance[, 1]))

## Small B is for illustration; increase it for final inference.
smp <- subsample(rfq.block, B = 25, verbose = FALSE)
print(extract.subsample(smp)$var.sel.Z)
plot.subsample(smp, cex.axis = .7, alpha = .5)

## ------------------------------------------------------------
## Optional sampling and grow-interface choices
## ------------------------------------------------------------
rfq.sample <- imbalanced(f, dta, ntree = 300, ratio = .25)
print(get.imbalanced.performance(rfq.sample))
rfq.fast <- imbalanced(f, dta, ntree = 300, fast = TRUE)
print(get.imbalanced.performance(rfq.fast))

randomForestSRC documentation built on Sept. 16, 2026, 5:06 p.m.