# compareModels: Compare accuracy of alternative classification methods In gamclass: Functions and Data for a Course on Modern Regression and Classification

## Description

Compare, between models, probabilities that the models assign to membership in the correct group or class. Probabilites should be estimated from cross-validation or from bootstrap out-of-bag data or preferably for test data that are completely separate from the data used to dervive the model.

## Usage

 ```1 2``` ```compareModels(groups, estprobs = list(lda = NULL, rf = NULL), gpnames = NULL, robust = TRUE, print = TRUE) ```

## Arguments

 `groups` Factor that specifies the groups `estprobs` List whose elements (with names that identify the models) are matrices that give for each observation (row) estimated probabilities of membership for each of the groups (columns). `gpnames` Character: names for groups, if different from `levels(groups)` `robust` Logical, `TRUE` or `FALSE` `print` Logical. Should results be printed?

## Details

The estimated probabilities are compared directly, under normal distribution assumptions. An effect is fitted for each observation, plus an effect for the method. Comparison on a logit scale may sometimes be preferable. An option to allow this is scheduled for incorporation in a later version.

## Value

 `modelAVS` Average accuracies for models `modelSE` Approximate average SE for comparing models `gpAVS` Average accuracies for groups `gpSE` Approximate average SE for comparing groups `obsEff` Effects assigned to individual observations

## Note

The analysis estimates effects due to model and group (`gp`), after accounting for differences between observations.

John Maindonald

## Examples

 ```1 2 3 4 5 6 7 8 9``` ```library(MASS) library(DAAG) library(randomForest) ldahat <- lda(species ~ length+breadth, data=cuckoos, CV=TRUE)\$posterior qdahat <- qda(species ~ length+breadth, data=cuckoos, CV=TRUE)\$posterior rfhat <- predict(randomForest(species ~ length+breadth, data=cuckoos), type="prob") compareModels(groups=cuckoos\$species, estprobs=list(lda=ldahat, qda=qdahat, rf=rfhat), robust=FALSE) ```

### Example output

```Loading required package: lattice

Attaching package: 'DAAG'

The following object is masked from 'package:MASS':

hills

randomForest 4.6-14
Type rfNews() to see new features/changes/bug fixes.
[1] "Average accuracies for groups:"
0.1370          0.5257          0.1328          0.1458          0.1379
gpwren
0.5464
Approx sed
0.0253
[1] "Average accuracies for methods:"
lda    qda     rf
0.3260 0.3385 0.3396
Approx sed
0.0126
```

gamclass documentation built on Nov. 14, 2020, 1:07 a.m.