vuong: Non-nested model tests

In games: Statistical Estimation of Game-Theoretic Models

Description

Perform Vuong's (1989) or Clarke's (2007) test for non-nested model selection.

Usage

vuong(model1, model2, outcome1=NULL, outcome2=NULL, level=0.05,
digits=2)

clarke(model1, model2, outcome1=NULL, outcome2=NULL, level=0.05, digits=2)

Arguments

model1	A fitted statistical model of class "game", "lm", or "glm"
model2	A fitted statistical model of class "game", "lm", or "glm" whose dependent variable is the same as that of model1
outcome1	Optional: if model1 is of class "game", specify an integer to restrict attention to a particular binary outcome (the corresponding column of predict(model1)). For ultimatum models, "offer" or "accept" may also be used. See "Details" below for more information on when to specify an outcome. If model1 is not of class "game" and outcome1 is non-NULL, it will be ignored and a warning will be issued.
outcome2	Optional: same as outcome1, but corresponding to model2.
level	Numeric: significance level for the test.
digits	Integer: number of digits to print

Details

These tests are for comparing two statistical models that have the same dependent variable, where neither model can be expressed as a special case of the other (i.e., they are non-nested). The null hypothesis is that the estimated models are the same Kullback-Leibler distance from the true model. To adjust for potential differences in the dimensionality of the models, the test statistic for both vuong and clarke is corrected using the Bayesian information criterion (see Clarke 2007 for details).

It is crucial that the dependent variable be exactly the same between the two models being tested, including the order the observations are placed in. The vuong and clarke functions check for such discrepancies, and stop with an error if any is found. Models with non-null weights are not yet supported.

When comparing a strategic model to a (generalized) linear model, you must take care to ensure that the dependent variable is truly the same between models. This is where the outcome arguments come into play. For example, in an ultimatum model where acceptance is observed, the dependent variable for each observation is the vector consisting of the offer size and an indicator for whether it was accepted. This is not the same as the dependent variable in a least-squares regression of offer size, which is a scalar for each observation. Therefore, for a proper comparison of model1 of class "ultimatum" and model2 of class "lm", it is necessary to specify outcome1 = "offer". Similarly, consider an egame12 model on the war1800 data, where player 1 chooses whether to escalate the crisis and player 2 chooses whether to go to war. The dependent variable for each observation in this model is the vector of each player's choice. By contrast, in a logistic regression where the dependent variable is whether war occurs, the dependent variable for each observation is a scalar. To compare these models, it is necessary to specify outcome1 = 3.

Value

Typical use will be to run the function interactively and examine the printed output. The functions return an object of class "nonnest.test", which is a list containing:

stat

The test statistic

test

The type of test ("vuong" or "clarke")

level

Significance level for the test

digits

Number of digits to print

loglik1

Vector of observationwise log-likelihoods for model1

loglik2

Vector of observationwise log-likelihoods for model2

nparams

Integer vector containing the number of parameters fitted in model1 and model2 respectively

nobs

Number of observations of the dependent variable being modeled

Author(s)

Brenton Kenkel ([e-mail])

References

Quang H. Vuong. 1989. "Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses." Econometrica 57(2): 307–333.

Kevin Clarke. 2007. "A Simple Distribution-Free Test for Nonnested Hypotheses." Political Analysis 15(3): 347–363.

Examples

data("war1800")

## Balance of power model
f1 <- esc + war ~ balanc + s_wt_re1 | 0 | balanc | balanc + s_wt_re1
m1 <- egame12(f1, data = war1800, subset = !is.na(regime1) & !is.na(regime2))

## Regime type model
f2 <- esc + war ~ regime1 | 0 | regime1 + regime2 | regime1 + regime2
m2 <- egame12(f2, data = war1800)

## Comparing two strategic models
vuong(model1 = m1, model2 = m2)
clarke(model1 = m1, model2 = m2)

## Comparing strategic model to logit - must specify `outcome1` appropriately
logit1 <- glm(war ~ balanc + s_wt_re1, data = m1$model, family=binomial)
vuong(model1 = m1, outcome1 = 3, model2 = logit1)
clarke(model1 = m1, outcome1 = 3, model2 = logit1)

logit2 <- glm(sq ~ regime1 + regime2, data = war1800, family=binomial)
vuong(model1 = m2, outcome1 = 1, model2 = logit2)
clarke(model1 = m2, outcome1 = 1, model2 = logit2)

## Ultimatum model
data(data_ult)
f3 <- offer + accept ~ w1 + w2 + x1 + x2 | w1 + w2 + z1 + z2
m3 <- ultimatum(f3, maxOffer = 15, data = data_ult)
ols1 <- lm(offer ~ w1 + w2 + x1 + x2 + z1 + z2, data = data_ult)
vuong(model1 = m3, outcome1 = "offer", model2 = ols1)
clarke(model1 = m3, outcome1 = "offer", model2 = ols1)

Example output

Loading required package: maxLik
Loading required package: miscTools

Please cite the 'maxLik' package as:
Henningsen, Arne and Toomet, Ott (2011). maxLik: A package for maximum likelihood estimation in R. Computational Statistics 26(3), 443-458. DOI 10.1007/s00180-010-0217-1.

If you have questions, suggestions, or comments regarding the 'maxLik' package, please use a forum or 'tracker' at maxLik's R-Forge site:
https://r-forge.r-project.org/projects/maxlik/
Loading required package: Formula
Loading required package: MASS
Warning message:
In egame12(f1, data = war1800, subset = !is.na(regime1) & !is.na(regime2)) :
  Hessian is not negative definite; coefficients may not be locally identified

Vuong test for non-nested models

Model 1 log-likelihood: -291
Model 2 log-likelihood: -289
Observations: 282
Test statistic: 1.7

Neither model is significantly preferred (p = 0.087)


Clarke test for non-nested models

Model 1 log-likelihood: -291
Model 2 log-likelihood: -289
Observations: 282
Test statistic: 158 (56%)

Model 1 is preferred (p = 0.049)


Vuong test for non-nested models

Model 1 log-likelihood: -154
Model 2 log-likelihood: -156
Observations: 282
Test statistic: -15

Model 2 is preferred (p < 2e-16)


Clarke test for non-nested models

Model 1 log-likelihood: -154
Model 2 log-likelihood: -156
Observations: 282
Test statistic: 4 (1%)

Model 2 is preferred (p < 2e-16)


Vuong test for non-nested models

Model 1 log-likelihood: -188
Model 2 log-likelihood: -188
Observations: 282
Test statistic: -33

Model 2 is preferred (p < 2e-16)


Clarke test for non-nested models

Model 1 log-likelihood: -188
Model 2 log-likelihood: -188
Observations: 282
Test statistic: 3 (1%)

Model 2 is preferred (p < 2e-16)


Vuong test for non-nested models

Model 1 log-likelihood: -3844
Model 2 log-likelihood: -4385
Observations: 1500
Test statistic: 16

Model 1 is preferred (p < 2e-16)


Clarke test for non-nested models

Model 1 log-likelihood: -3844
Model 2 log-likelihood: -4385
Observations: 1500
Test statistic: 863 (58%)

Model 1 is preferred (p = 5.9e-09)

games documentation built on May 2, 2019, 3:26 p.m.