orsf_vi: ORSF variable importance
In aorsf: Accelerated Oblique Random Survival Forests
orsf_vi R Documentation
ORSF variable importance
Description
Estimate the importance of individual variables using oblique random
survival forests.
Usage
orsf_vi(
object,
group_factors = TRUE,
importance = NULL,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_negate(
object,
group_factors = TRUE,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_permute(
object,
group_factors = TRUE,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_anova(object, group_factors = TRUE, ...)
Arguments
object
(orsf_fit) a trained oblique random survival forest
(see orsf).
group_factors
(logical) if TRUE, the importance of factor variables will be reported overall by aggregating the importance of individual levels of the factor. If FALSE, the importance of individual factor levels will be returned.
importance
(character) Indicate method for variable importance:
'anova': compute analysis of variance (ANOVA) importance
'negate': compute negation importance
'permute': compute permutation importance
oobag_fun
(function) to be used for evaluating out-of-bag prediction accuracy after negating coefficients
(if importance = 'negate') or permuting the values of a predictor
(if importance = 'permute')
When oobag_fun = NULL (the default), Harrell's C-statistic (1982) is used to evaluate accuracy.
if you use your own oobag_fun note the following:
-
oobag_fun should have two inputs: y_mat and s_vec
-
y_mat is a two column matrix with first column named 'time', second named 'status'
-
s_vec is a numeric vector containing predicted survival probabilities.
-
oobag_fun should return a numeric output of length 1
the same oobag_fun should have been used when you created object
so that the initial value of out-of-bag prediction accuracy is
consistent with the values that will be computed while variable
importance is estimated.
For more details, see the out-of-bag
vignette.
n_thread
(integer) number of threads to use while computing predictions. Default is one thread. To use the maximum number of threads that your system provides for concurrent execution, set n_thread = 0.
verbose_progress
(logical) if TRUE, progress messages are
printed in the console. If FALSE (the default), nothing is printed.
...
Further arguments passed to or from other methods (not currently used).
Details
When an orsf_fit object is fitted with importance = 'anova', 'negate', or
'permute', the output will have a vector of importance values based on
the requested type of importance. However, you may still want to call
orsf_vi() on this output if you want to group factor levels into one
overall importance value.
orsf_vi() is a general purpose function to extract or compute variable
importance estimates from an 'orsf_fit' object (see orsf).
orsf_vi_negate(), orsf_vi_permute(), and orsf_vi_anova() are wrappers
for orsf_vi(). The way these functions work depends on whether the
object they are given already has variable importance estimates in it
or not (see examples).
Value
orsf_vi functions return a named numeric vector.
Names of the vector are the predictor variables used by object
Values of the vector are the estimated importance of the given predictor.
The returned vector is sorted from highest to lowest value, with higher
values indicating higher importance.
Variable importance methods
negation importance: Each variable is assessed separately by multiplying the variable's coefficients by -1 and then determining how much the model's performance changes. The worse the model's performance after negating coefficients for a given variable, the more important the variable. This technique is promising b/c it does not require permutation and it emphasizes variables with larger coefficients in linear combinations, but it is also relatively new and hasn't been studied as much as permutation importance. See Jaeger, (2023) for more details on this technique.
permutation importance: Each variable is assessed separately by randomly permuting the variable's values and then determining how much the model's performance changes. The worse the model's performance after permuting the values of a given variable, the more important the variable. This technique is flexible, intuitive, and frequently used. It also has several known limitations
analysis of variance (ANOVA) importance: A p-value is computed for each coefficient in each linear combination of variables in each decision tree. Importance for an individual predictor variable is the proportion of times a p-value for its coefficient is < 0.01. This technique is very efficient computationally, but may not be as effective as permutation or negation in terms of selecting signal over noise variables. See Menze, 2011 for more details on this technique.
Examples
ANOVA importance
The default variable importance technique, ANOVA, is calculated while
you fit an ORSF ensemble.
fit <- orsf(pbc_orsf, Surv(time, status) ~ . - id)
fit
## ---------- Oblique random survival forest
##
## Linear combinations: Accelerated Cox regression
## N observations: 276
## N events: 111
## N trees: 500
## N predictors total: 17
## N predictors per node: 5
## Average leaves per tree: 21
## Min observations in leaf: 5
## Min events in leaf: 1
## OOB stat value: 0.84
## OOB stat type: Harrell's C-statistic
## Variable importance: anova
##
## -----------------------------------------
ANOVA is the default because it is fast, but it may not be as decisive
as the permutation and negation techniques for variable selection.
Raw VI values
the ‘raw’ variable importance values can be accessed from the fit object
attr(fit, 'importance_values')
## edema_1 ascites_1 bili copper albumin age
## 0.53189300 0.49950642 0.39598881 0.30443254 0.26028060 0.24758399
## protime stage chol edema_0.5 spiders_1 ast
## 0.22874192 0.20974576 0.20353982 0.18401760 0.18090452 0.17457962
## hepato_1 sex_f trig alk.phos platelet trt_placebo
## 0.16402406 0.14803440 0.13009809 0.11627907 0.07853659 0.06939410
these are ‘raw’ because values for factors have not been aggregated into
a single value. Currently there is one value for k-1 levels of a k level
factor. For example, you can see edema_1 and edema_0.5 in the importance
values above because edema is a factor variable with levels of 0, 0.5,
and 1.
Collapse VI across factor levels
To get aggregated values across all levels of each factor,
access the importance element from the orsf fit:
fit$importance
## ascites bili edema copper albumin age protime
## 0.49950642 0.39598881 0.32482431 0.30443254 0.26028060 0.24758399 0.22874192
## stage chol spiders ast hepato sex trig
## 0.20974576 0.20353982 0.18090452 0.17457962 0.16402406 0.14803440 0.13009809
## alk.phos platelet trt
## 0.11627907 0.07853659 0.06939410
use orsf_vi() with group_factors set to TRUE (the default)
orsf_vi(fit)
## ascites bili edema copper albumin age protime
## 0.49950642 0.39598881 0.32482431 0.30443254 0.26028060 0.24758399 0.22874192
## stage chol spiders ast hepato sex trig
## 0.20974576 0.20353982 0.18090452 0.17457962 0.16402406 0.14803440 0.13009809
## alk.phos platelet trt
## 0.11627907 0.07853659 0.06939410
Note that you can make the default returned importance values ungrouped
by setting group_factors to FALSE in the orsf_vi functions or the
orsf function.
Add VI to an ORSF
You can fit an ORSF without VI, then add VI later
fit_no_vi <- orsf(pbc_orsf,
Surv(time, status) ~ . - id,
importance = 'none')
# Note: you can't call orsf_vi_anova() on fit_no_vi because anova
# VI can only be computed while the forest is being grown.
orsf_vi_negate(fit_no_vi)
## bili copper sex protime stage albumin
## 0.118368091 0.048934519 0.037084634 0.027045537 0.023883035 0.021168322
## age ascites chol ast hepato edema
## 0.020510253 0.014990139 0.014700024 0.011445420 0.007705758 0.007216976
## spiders trig alk.phos trt platelet
## 0.006373687 0.003230074 0.002825097 0.002487956 0.001558061
orsf_vi_permute(fit_no_vi)
## bili copper protime albumin ascites
## 0.0546262009 0.0248983335 0.0154588025 0.0135496100 0.0134187625
## age stage chol edema ast
## 0.0119125415 0.0114086074 0.0074840971 0.0052955192 0.0051264428
## hepato spiders sex trig alk.phos
## 0.0050509057 0.0046277553 0.0039395847 0.0024127938 0.0009590829
## platelet trt
## 0.0004366080 -0.0018115005
ORSF and VI all at once
fit an ORSF and compute vi at the same time
fit_permute_vi <- orsf(pbc_orsf,
Surv(time, status) ~ . - id,
importance = 'permute')
# get the vi instantly (i.e., it doesn't need to be computed again)
orsf_vi_permute(fit_permute_vi)
## bili copper age ascites albumin
## 5.354589e-02 2.608966e-02 1.286604e-02 1.250721e-02 1.206669e-02
## protime stage ast chol edema
## 1.083811e-02 1.056282e-02 8.224237e-03 6.016773e-03 5.831833e-03
## spiders hepato sex trig alk.phos
## 4.736005e-03 3.446193e-03 3.440979e-03 2.277363e-03 2.224448e-03
## platelet trt
## 1.424653e-03 -1.351124e-05
You can still get negation VI from this fit, but it needs to be computed
orsf_vi_negate(fit_permute_vi)
## bili copper sex stage protime albumin
## 0.124731632 0.052319855 0.038669229 0.027453339 0.022757244 0.022220823
## age ast ascites chol spiders edema
## 0.020417138 0.013403277 0.013193815 0.011895708 0.008962837 0.007361044
## hepato trt trig alk.phos platelet
## 0.006428047 0.004371012 0.004240743 0.003232336 0.002575995
References
Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the Yield of Medical Tests. JAMA 1982; 247(18):2543-2546. DOI: 10.1001/jama.1982.03320430047030
Breiman L. Random forests. Machine learning 2001 Oct; 45(1):5-32. DOI: 10.1023/A:1010933404324
Menze BH, Kelm BM, Splitthoff DN, Koethe U, Hamprecht FA. On oblique random forests. Joint European Conference on Machine Learning and Knowledge Discovery in Databases 2011 Sep 4; pp. 453-469. DOI: 10.1007/978-3-642-23783-6_29
Jaeger BC, Welden S, Lenoir K, Speiser JL, Segar MW, Pandey A, Pajewski NM. Accelerated and interpretable oblique random survival forests. Journal of Computational and Graphical Statistics Published online 08 Aug 2023. DOI: 10.1080/10618600.2023.2231048
aorsf documentation built on Oct. 26, 2023, 5:08 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
| orsf_vi | R Documentation |
ORSF variable importance
Description
Estimate the importance of individual variables using oblique random survival forests.
Usage
orsf_vi(
object,
group_factors = TRUE,
importance = NULL,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_negate(
object,
group_factors = TRUE,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_permute(
object,
group_factors = TRUE,
oobag_fun = NULL,
n_thread = 1,
verbose_progress = FALSE,
...
)
orsf_vi_anova(object, group_factors = TRUE, ...)
Arguments
object |
(orsf_fit) a trained oblique random survival forest (see orsf). |
group_factors |
(logical) if |
importance |
(character) Indicate method for variable importance:
|
oobag_fun |
(function) to be used for evaluating out-of-bag prediction accuracy after negating coefficients (if importance = 'negate') or permuting the values of a predictor (if importance = 'permute')
For more details, see the out-of-bag vignette. |
n_thread |
(integer) number of threads to use while computing predictions. Default is one thread. To use the maximum number of threads that your system provides for concurrent execution, set |
verbose_progress |
(logical) if |
... |
Further arguments passed to or from other methods (not currently used). |
Details
When an orsf_fit object is fitted with importance = 'anova', 'negate', or
'permute', the output will have a vector of importance values based on
the requested type of importance. However, you may still want to call
orsf_vi() on this output if you want to group factor levels into one
overall importance value.
orsf_vi() is a general purpose function to extract or compute variable
importance estimates from an 'orsf_fit' object (see orsf).
orsf_vi_negate(), orsf_vi_permute(), and orsf_vi_anova() are wrappers
for orsf_vi(). The way these functions work depends on whether the
object they are given already has variable importance estimates in it
or not (see examples).
Value
orsf_vi functions return a named numeric vector.
Names of the vector are the predictor variables used by
objectValues of the vector are the estimated importance of the given predictor.
The returned vector is sorted from highest to lowest value, with higher values indicating higher importance.
Variable importance methods
negation importance: Each variable is assessed separately by multiplying the variable's coefficients by -1 and then determining how much the model's performance changes. The worse the model's performance after negating coefficients for a given variable, the more important the variable. This technique is promising b/c it does not require permutation and it emphasizes variables with larger coefficients in linear combinations, but it is also relatively new and hasn't been studied as much as permutation importance. See Jaeger, (2023) for more details on this technique.
permutation importance: Each variable is assessed separately by randomly permuting the variable's values and then determining how much the model's performance changes. The worse the model's performance after permuting the values of a given variable, the more important the variable. This technique is flexible, intuitive, and frequently used. It also has several known limitations
analysis of variance (ANOVA) importance: A p-value is computed for each coefficient in each linear combination of variables in each decision tree. Importance for an individual predictor variable is the proportion of times a p-value for its coefficient is < 0.01. This technique is very efficient computationally, but may not be as effective as permutation or negation in terms of selecting signal over noise variables. See Menze, 2011 for more details on this technique.
Examples
ANOVA importance
The default variable importance technique, ANOVA, is calculated while you fit an ORSF ensemble.
fit <- orsf(pbc_orsf, Surv(time, status) ~ . - id) fit
## ---------- Oblique random survival forest ## ## Linear combinations: Accelerated Cox regression ## N observations: 276 ## N events: 111 ## N trees: 500 ## N predictors total: 17 ## N predictors per node: 5 ## Average leaves per tree: 21 ## Min observations in leaf: 5 ## Min events in leaf: 1 ## OOB stat value: 0.84 ## OOB stat type: Harrell's C-statistic ## Variable importance: anova ## ## -----------------------------------------
ANOVA is the default because it is fast, but it may not be as decisive as the permutation and negation techniques for variable selection.
Raw VI values
the ‘raw’ variable importance values can be accessed from the fit object
attr(fit, 'importance_values')
## edema_1 ascites_1 bili copper albumin age ## 0.53189300 0.49950642 0.39598881 0.30443254 0.26028060 0.24758399 ## protime stage chol edema_0.5 spiders_1 ast ## 0.22874192 0.20974576 0.20353982 0.18401760 0.18090452 0.17457962 ## hepato_1 sex_f trig alk.phos platelet trt_placebo ## 0.16402406 0.14803440 0.13009809 0.11627907 0.07853659 0.06939410
these are ‘raw’ because values for factors have not been aggregated into a single value. Currently there is one value for k-1 levels of a k level factor. For example, you can see edema_1 and edema_0.5 in the importance values above because edema is a factor variable with levels of 0, 0.5, and 1.
Collapse VI across factor levels
To get aggregated values across all levels of each factor,
access the
importanceelement from theorsffit:fit$importance
## ascites bili edema copper albumin age protime ## 0.49950642 0.39598881 0.32482431 0.30443254 0.26028060 0.24758399 0.22874192 ## stage chol spiders ast hepato sex trig ## 0.20974576 0.20353982 0.18090452 0.17457962 0.16402406 0.14803440 0.13009809 ## alk.phos platelet trt ## 0.11627907 0.07853659 0.06939410
use
orsf_vi()with group_factors set toTRUE(the default)orsf_vi(fit)
## ascites bili edema copper albumin age protime ## 0.49950642 0.39598881 0.32482431 0.30443254 0.26028060 0.24758399 0.22874192 ## stage chol spiders ast hepato sex trig ## 0.20974576 0.20353982 0.18090452 0.17457962 0.16402406 0.14803440 0.13009809 ## alk.phos platelet trt ## 0.11627907 0.07853659 0.06939410
Note that you can make the default returned importance values ungrouped
by setting group_factors to FALSE in the orsf_vi functions or the
orsf function.
Add VI to an ORSF
You can fit an ORSF without VI, then add VI later
fit_no_vi <- orsf(pbc_orsf,
Surv(time, status) ~ . - id,
importance = 'none')
# Note: you can't call orsf_vi_anova() on fit_no_vi because anova
# VI can only be computed while the forest is being grown.
orsf_vi_negate(fit_no_vi)
## bili copper sex protime stage albumin ## 0.118368091 0.048934519 0.037084634 0.027045537 0.023883035 0.021168322 ## age ascites chol ast hepato edema ## 0.020510253 0.014990139 0.014700024 0.011445420 0.007705758 0.007216976 ## spiders trig alk.phos trt platelet ## 0.006373687 0.003230074 0.002825097 0.002487956 0.001558061
orsf_vi_permute(fit_no_vi)
## bili copper protime albumin ascites ## 0.0546262009 0.0248983335 0.0154588025 0.0135496100 0.0134187625 ## age stage chol edema ast ## 0.0119125415 0.0114086074 0.0074840971 0.0052955192 0.0051264428 ## hepato spiders sex trig alk.phos ## 0.0050509057 0.0046277553 0.0039395847 0.0024127938 0.0009590829 ## platelet trt ## 0.0004366080 -0.0018115005
ORSF and VI all at once
fit an ORSF and compute vi at the same time
fit_permute_vi <- orsf(pbc_orsf,
Surv(time, status) ~ . - id,
importance = 'permute')
# get the vi instantly (i.e., it doesn't need to be computed again)
orsf_vi_permute(fit_permute_vi)
## bili copper age ascites albumin ## 5.354589e-02 2.608966e-02 1.286604e-02 1.250721e-02 1.206669e-02 ## protime stage ast chol edema ## 1.083811e-02 1.056282e-02 8.224237e-03 6.016773e-03 5.831833e-03 ## spiders hepato sex trig alk.phos ## 4.736005e-03 3.446193e-03 3.440979e-03 2.277363e-03 2.224448e-03 ## platelet trt ## 1.424653e-03 -1.351124e-05
You can still get negation VI from this fit, but it needs to be computed
orsf_vi_negate(fit_permute_vi)
## bili copper sex stage protime albumin ## 0.124731632 0.052319855 0.038669229 0.027453339 0.022757244 0.022220823 ## age ast ascites chol spiders edema ## 0.020417138 0.013403277 0.013193815 0.011895708 0.008962837 0.007361044 ## hepato trt trig alk.phos platelet ## 0.006428047 0.004371012 0.004240743 0.003232336 0.002575995
References
Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the Yield of Medical Tests. JAMA 1982; 247(18):2543-2546. DOI: 10.1001/jama.1982.03320430047030
Breiman L. Random forests. Machine learning 2001 Oct; 45(1):5-32. DOI: 10.1023/A:1010933404324
Menze BH, Kelm BM, Splitthoff DN, Koethe U, Hamprecht FA. On oblique random forests. Joint European Conference on Machine Learning and Knowledge Discovery in Databases 2011 Sep 4; pp. 453-469. DOI: 10.1007/978-3-642-23783-6_29
Jaeger BC, Welden S, Lenoir K, Speiser JL, Segar MW, Pandey A, Pajewski NM. Accelerated and interpretable oblique random survival forests. Journal of Computational and Graphical Statistics Published online 08 Aug 2023. DOI: 10.1080/10618600.2023.2231048
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.