In MATHSTATSOU/Intro2MLR: Introduction to MLR

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#ws>"
)

library(Intro2MLR)

Introduction

The F statistic is central to MLR and we will need to both derive and use it well.

These notes are not a complete account of the derivation of the f statistic and rely completely on the class work developed previously.

Expressions for F

The test for model adequacy is usually presented as the test against

$$H_0: \beta_1=\beta_2=\ldots =\beta_k=0$$ The F statistic is calculated by finding

$$F = \frac{(RSS_{H_0}-RSS)/k}{RSS/(n-(k+1))}$$ Now $RSS_{H_0}=(Y-\hat{Y})^{'}(Y-\hat{Y})$ where $\hat{Y}$ is calculated under the assumption of $H_0$.

So $Y=X\beta + \epsilon$ where $X= 1_n$ and hence $\hat{Y}=X\hat{\beta}$

$$\hat{Y}=X(X^{'}X)^{-1}X^{'} Y = 1_{n}(n)^{-1}1^{'}_n Y=1/nJ_n Y=\bar{Y}$$

This means that $RSS_{H_0}=TSS$

Since $TSS=MSS + RSS$

$$F = \frac{MSS/k}{RSS/(n-(k+1))}$$ We will now simplify this expression in terms of $R^2= \frac{MSS}{TSS}$.

The first thing to do is re-express $MSS$ and $RSS$ in terms of $R^2$

$R^2$

$$MSS = R^2 TSS$$

and

$$R^2 = 1-\frac{RSS}{TSS}$$ Therefore

$$RSS = (1-R^2)TSS$$

F as a function of $R^2$ when $H_0: \beta_1=\ldots=\beta_k=0$

$$F = \frac{R^2TSS/k}{(1-R^2)TSS/(n-(k+1))}$$ and hence

$$F = \frac{R^2/k}{(1-R^2)/(n-(k+1))}$$

Comparing a larger model with a nested version.

We developed in class the following result

$$F=\frac{(RSS_{H_0}-RSS)/(k-g)}{RSS/(n-(k+1))}$$

where $H_0: Y=\beta_0 + \beta_1x_1 + \ldots + \beta_gx_g + \epsilon$ and the alternate

$H_1: Y=\beta_0 + \beta_1x_1 + \ldots + \beta_gx_g + \beta_{g+1}x_{g+1}+\ldots \beta_kx_k+\epsilon$

We could express the NULL uses in the test as $\beta_{g+1} =\ldots =\beta_k=0$

If we make $k=g+1$ then $k-g=1$ and $H_0: \beta_{g+1}=0$

$$F=\frac{\chi^2_1/1}{\chi^{2}{n-(k+1)}/(n-(k+1))}=\frac{Z^2}{\chi^{2}{n-(k+1)}/(n-(k+1))}=T^2$$

MATHSTATSOU/Intro2MLR documentation built on Dec. 11, 2020, 9:01 p.m.