plot_test_perf_VS_rand: plot_test_perf_VS_rand

In shihuang047/crossRanger: Cross-Application of Ranger Models using microbiome data

plot_test_perf_VS_rand

Description

This outputs a histogram and a p-value showing if the performance of a real regression model significantly better than null models.

Usage

plot_test_perf_VS_rand(
  rf_model,
  newx,
  newy,
  prefix = "test",
  target_field = "",
  metric = "MAE",
  permutation = 1000,
  n_features = NA,
  outdir = NULL
)

Arguments

rf_model	A trained rf model object, which should be generated from /coderf.cross.validation or /coderf.out.of.bag.
newx	A data.matrix or data.frame with the new data for rf model testing.
newy	The data label of new data.
prefix	The prefix for the dataset in the training or testing.
target_field	A string indicating the target field of the metadata for machine-learning analysis.
metric	The regression performance metric applied, including MAE, RMSE, MSE, R_squared, Adj_R_squared, or Separman_rho.
permutation	The permutation times for a random guess of regression performance.
n_features	The number of features in the training data.
outdir	The output directory.

Author(s)

Shi Huang

Examples

set.seed(123)
x <- data.frame(rbind(t(rmultinom(7, 75, c(.201,.5,.02,.18,.099))),
            t(rmultinom(8, 75, c(.201,.4,.12,.18,.099))),
            t(rmultinom(15, 75, c(.011,.3,.22,.18,.289))),
            t(rmultinom(15, 75, c(.091,.2,.32,.18,.209))),
            t(rmultinom(15, 75, c(.001,.1,.42,.18,.299)))))
y<- 1:60

newx <- data.frame(rbind(t(rmultinom(7, 75, c(.201,.5,.02,.18,.099))),
            t(rmultinom(8, 75, c(.201,.4,.12,.18,.099))),
            t(rmultinom(15, 75, c(.001,.1,.42,.18,.299)))))
newy<- 4:33
rf_model<-rf.out.of.bag(x, y)
p<-plot_perf_VS_rand(x=x, y=y, predicted_y=rf_model$predicted, prefix="train", nfolds=5,
permutation=100, metric="MAE", target_field="age", n_features=5)
p
p_test<-plot_test_perf_VS_rand(rf_model, newx, newy, permutation=100,
                               metric="MAE", target_field="age", n_features=5)
p_test

shihuang047/crossRanger documentation built on Feb. 7, 2023, 10:03 p.m.