fFtest: Fast (Weighted) F-test for Linear Models (with Factors)

In SebKrantz/collapse: Advanced and Fast Data Transformation

Fast (Weighted) F-test for Linear Models (with Factors)

Description

fFtest computes an R-squared based F-test for the exclusion of the variables in exc, where the full (unrestricted) model is defined by variables supplied to both exc and X. The test is efficient and designed for cases where both exc and X may contain multiple factors and continuous variables. There is also an efficient 2-part formula method.

Usage

fFtest(...) # Internal method dispatch: formula if is.call(..1) || is.call(..2)

## Default S3 method:
fFtest(y, exc, X = NULL, w = NULL, full.df = TRUE, ...)

## S3 method for class 'formula'
fFtest(formula, data = NULL, weights = NULL, ...)

Arguments

y	a numeric vector: the dependent variable.
exc	a numeric vector, factor, numeric matrix or list / data frame of numeric vectors and/or factors: variables to test / exclude.
X	a numeric vector, factor, numeric matrix or list / data frame of numeric vectors and/or factors: covariates to include in both the restricted (without exc) and unrestricted model. If left empty (X = NULL), the test amounts to the F-test of the regression of y on exc.
w	numeric. A vector of (frequency) weights.
formula	a 2-part formula: y ~ exc | X, where both exc and X are expressions connected with +, and X can be omitted. Note that other operators (:, *, ^, -, etc.) are not supported, you can interact variables using standard functions like finteraction/itn or magrittr::multiply_by inside the formula e.g. log(y) ~ x1 + itn(x2, x3) | x4 or log(y) ~ x1 + multiply_by(x2, x3) | x4.
data	a named list or data frame.
weights	a weights vector or expression that results in a vector when evaluated in the data environment.
full.df	logical. If TRUE (default), the degrees of freedom are calculated as if both restricted and unrestricted models were estimated using lm() (i.e. as if factors were expanded to matrices of dummies). FALSE only uses one degree of freedom per factor.
...	other arguments passed to fFtest.default or to fhdwithin. Sensible options might be the lm.method argument or further control parameters to fixest::demean, the workhorse function underlying fhdwithin for higher-order centering tasks.

Details

Factors and continuous regressors are efficiently projected out using fhdwithin, and the option full.df regulates whether a degree of freedom is subtracted for each used factor level (equivalent to dummy-variable estimator / expanding factors), or only one degree of freedom per factor (treating factors as variables). The test automatically removes missing values and considers only the complete cases of y, exc and X. Unused factor levels in exc and X are dropped.

Note that an intercept is always added by fhdwithin, so it is not necessary to include an intercept in data supplied to exc / X.

Value

A 5 x 3 numeric matrix of statistics. The columns contain statistics:

1. the R-squared of the model

2. the numerator degrees of freedom i.e. the number of variables (k) and used factor levels if full.df = TRUE

3. the denominator degrees of freedom: N - k - 1.

4. the F-statistic

5. the corresponding P-value

The rows show these statistics for:

1. the Full (unrestricted) Model (y ~ exc + X)

2. the Restricted Model (y ~ X)

3. the Exclusion Restriction of exc. The R-squared shown is simply the difference of the full and restricted R-Squared's, not the R-Squared of the model y ~ exc.

If X = NULL, only a vector of the same 5 statistics testing the model (y ~ exc) is shown.

See Also

flm, fhdwithin, Data Transformations, Collapse Overview

Examples

## We could use fFtest as a simple seasonality test:
fFtest(AirPassengers, qF(cycle(AirPassengers)))         # Testing for level-seasonality
fFtest(AirPassengers, qF(cycle(AirPassengers)),         # Seasonality test around a cubic trend
        poly(seq_along(AirPassengers), 3))
fFtest(fdiff(AirPassengers), qF(cycle(AirPassengers)))  # Seasonality in first-difference

## A more classical example with only continuous variables
fFtest(mpg ~ cyl + vs | hp + carb, mtcars)
fFtest(mtcars$mpg, mtcars[c("cyl","vs")], mtcars[c("hp","carb")])
 
## Now encoding cyl and vs as factors
fFtest(mpg ~ qF(cyl) + qF(vs) | hp + carb, mtcars)
fFtest(mtcars$mpg, lapply(mtcars[c("cyl","vs")], qF), mtcars[c("hp","carb")])

## Using iris data: A factor and a continuous variable excluded
fFtest(Sepal.Length ~ Petal.Width + Species | Sepal.Width + Petal.Length, iris)
fFtest(iris$Sepal.Length, iris[4:5], iris[2:3])

## Testing the significance of country-FE in regression of GDP on life expectancy
fFtest(log(PCGDP) ~ iso3c | LIFEEX, wlddev)
fFtest(log(wlddev$PCGDP), wlddev$iso3c, wlddev$LIFEEX)
 
## Ok, country-FE are significant, what about adding time-FE
fFtest(log(PCGDP) ~ qF(year) | iso3c + LIFEEX, wlddev)
fFtest(log(wlddev$PCGDP), qF(wlddev$year), wlddev[c("iso3c","LIFEEX")])

# Same test done using lm:
data <- na_omit(get_vars(wlddev, c("iso3c","year","PCGDP","LIFEEX")))
full <- lm(PCGDP ~ LIFEEX + iso3c + qF(year), data)
rest <- lm(PCGDP ~ LIFEEX + iso3c, data)
anova(rest, full)

SebKrantz/collapse documentation built on Dec. 16, 2024, 7:26 p.m.