predict.xgb.Booster | R Documentation |

Predicted values based on either xgboost model or model handle object.

## S3 method for class 'xgb.Booster' predict( object, newdata, missing = NA, outputmargin = FALSE, ntreelimit = NULL, predleaf = FALSE, predcontrib = FALSE, approxcontrib = FALSE, predinteraction = FALSE, reshape = FALSE, training = FALSE, iterationrange = NULL, strict_shape = FALSE, ... ) ## S3 method for class 'xgb.Booster.handle' predict(object, ...)

`object` |
Object of class |

`newdata` |
takes For single-row predictions on sparse data, it's recommended to use CSR format. If passing a sparse vector, it will take it as a row vector. |

`missing` |
Missing is only used when input is dense matrix. Pick a float value that represents missing values in data (e.g., sometimes 0 or some other extreme value is used). |

`outputmargin` |
whether the prediction should be returned in the for of original untransformed
sum of predictions from boosting iterations' results. E.g., setting |

`ntreelimit` |
Deprecated, use |

`predleaf` |
whether predict leaf index. |

`predcontrib` |
whether to return feature contributions to individual predictions (see Details). |

`approxcontrib` |
whether to use a fast approximation for feature contributions (see Details). |

`predinteraction` |
whether to return contributions of feature interactions to individual predictions (see Details). |

`reshape` |
whether to reshape the vector of predictions to a matrix form when there are several prediction outputs per case. This option has no effect when either of predleaf, predcontrib, or predinteraction flags is TRUE. |

`training` |
whether is the prediction result used for training. For dart booster, training predicting will perform dropout. |

`iterationrange` |
Specifies which layer of trees are used in prediction. For
example, if a random forest is trained with 100 rounds. Specifying
'iterationrange=(1, 21)', then only the forests built during [1, 21) (half open set)
rounds are used in this prediction. It's 1-based index just like R vector. When set
to |

`strict_shape` |
Default is |

`...` |
Parameters passed to |

Note that `iterationrange`

would currently do nothing for predictions from gblinear,
since gblinear doesn't keep its boosting history.

One possible practical applications of the `predleaf`

option is to use the model
as a generator of new features which capture non-linearity and interactions,
e.g., as implemented in `xgb.create.features`

.

Setting `predcontrib = TRUE`

allows to calculate contributions of each feature to
individual predictions. For "gblinear" booster, feature contributions are simply linear terms
(feature_beta * feature_value). For "gbtree" booster, feature contributions are SHAP
values (Lundberg 2017) that sum to the difference between the expected output
of the model and the current prediction (where the hessian weights are used to compute the expectations).
Setting `approxcontrib = TRUE`

approximates these values following the idea explained
in http://blog.datadive.net/interpreting-random-forests/.

With `predinteraction = TRUE`

, SHAP values of contributions of interaction of each pair of features
are computed. Note that this operation might be rather expensive in terms of compute and memory.
Since it quadratically depends on the number of features, it is recommended to perform selection
of the most important features first. See below about the format of the returned results.

The return type is different depending whether `strict_shape`

is set to `TRUE`

. By default,
for regression or binary classification, it returns a vector of length `nrows(newdata)`

.
For multiclass classification, either a `num_class * nrows(newdata)`

vector or
a `(nrows(newdata), num_class)`

dimension matrix is returned, depending on
the `reshape`

value.

When `predleaf = TRUE`

, the output is a matrix object with the
number of columns corresponding to the number of trees.

When `predcontrib = TRUE`

and it is not a multiclass setting, the output is a matrix object with
`num_features + 1`

columns. The last "+ 1" column in a matrix corresponds to bias.
For a multiclass case, a list of `num_class`

elements is returned, where each element is
such a matrix. The contribution values are on the scale of untransformed margin
(e.g., for binary classification would mean that the contributions are log-odds deviations from bias).

When `predinteraction = TRUE`

and it is not a multiclass setting, the output is a 3d array with
dimensions `c(nrow, num_features + 1, num_features + 1)`

. The off-diagonal (in the last two dimensions)
elements represent different features interaction contributions. The array is symmetric WRT the last
two dimensions. The "+ 1" columns corresponds to bias. Summing this array along the last dimension should
produce practically the same result as predict with `predcontrib = TRUE`

.
For a multiclass case, a list of `num_class`

elements is returned, where each element is
such an array.

When `strict_shape`

is set to `TRUE`

, the output is always an array. For
normal prediction, the output is a 2-dimension array `(num_class, nrow(newdata))`

.

For `predcontrib = TRUE`

, output is `(ncol(newdata) + 1, num_class, nrow(newdata))`

For `predinteraction = TRUE`

, output is `(ncol(newdata) + 1, ncol(newdata) + 1, num_class, nrow(newdata))`

For `predleaf = TRUE`

, output is `(n_trees_in_forest, num_class, n_iterations, nrow(newdata))`

Scott M. Lundberg, Su-In Lee, "A Unified Approach to Interpreting Model Predictions", NIPS Proceedings 2017, https://arxiv.org/abs/1705.07874

Scott M. Lundberg, Su-In Lee, "Consistent feature attribution for tree ensembles", https://arxiv.org/abs/1706.06060

`xgb.train`

.

## binary classification: data(agaricus.train, package='xgboost') data(agaricus.test, package='xgboost') train <- agaricus.train test <- agaricus.test bst <- xgboost(data = train$data, label = train$label, max_depth = 2, eta = 0.5, nthread = 2, nrounds = 5, objective = "binary:logistic") # use all trees by default pred <- predict(bst, test$data) # use only the 1st tree pred1 <- predict(bst, test$data, iterationrange = c(1, 2)) # Predicting tree leafs: # the result is an nsamples X ntrees matrix pred_leaf <- predict(bst, test$data, predleaf = TRUE) str(pred_leaf) # Predicting feature contributions to predictions: # the result is an nsamples X (nfeatures + 1) matrix pred_contr <- predict(bst, test$data, predcontrib = TRUE) str(pred_contr) # verify that contributions' sums are equal to log-odds of predictions (up to float precision): summary(rowSums(pred_contr) - qlogis(pred)) # for the 1st record, let's inspect its features that had non-zero contribution to prediction: contr1 <- pred_contr[1,] contr1 <- contr1[-length(contr1)] # drop BIAS contr1 <- contr1[contr1 != 0] # drop non-contributing features contr1 <- contr1[order(abs(contr1))] # order by contribution magnitude old_mar <- par("mar") par(mar = old_mar + c(0,7,0,0)) barplot(contr1, horiz = TRUE, las = 2, xlab = "contribution to prediction in log-odds") par(mar = old_mar) ## multiclass classification in iris dataset: lb <- as.numeric(iris$Species) - 1 num_class <- 3 set.seed(11) bst <- xgboost(data = as.matrix(iris[, -5]), label = lb, max_depth = 4, eta = 0.5, nthread = 2, nrounds = 10, subsample = 0.5, objective = "multi:softprob", num_class = num_class) # predict for softmax returns num_class probability numbers per case: pred <- predict(bst, as.matrix(iris[, -5])) str(pred) # reshape it to a num_class-columns matrix pred <- matrix(pred, ncol=num_class, byrow=TRUE) # convert the probabilities to softmax labels pred_labels <- max.col(pred) - 1 # the following should result in the same error as seen in the last iteration sum(pred_labels != lb)/length(lb) # compare that to the predictions from softmax: set.seed(11) bst <- xgboost(data = as.matrix(iris[, -5]), label = lb, max_depth = 4, eta = 0.5, nthread = 2, nrounds = 10, subsample = 0.5, objective = "multi:softmax", num_class = num_class) pred <- predict(bst, as.matrix(iris[, -5])) str(pred) all.equal(pred, pred_labels) # prediction from using only 5 iterations should result # in the same error as seen in iteration 5: pred5 <- predict(bst, as.matrix(iris[, -5]), iterationrange=c(1, 6)) sum(pred5 != lb)/length(lb)

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.