cv.nperm: Conduct k-fold cross validation with multiple permutations

In marchtaylor/sinkr: Collection of functions with emphasis on multivariate data analysis

Conduct k-fold cross validation with multiple permutations

Description

Conduct k-fold cross validation with multiple permutations

Usage

cv.nperm(modelfun = "lm", k = 4, nperm = 10, argList = NULL)

Arguments

modelfun	Model function to use ("lm", "glm", "gam", "gls")
k	Number of folds to use (see kfold)
nperm	Number of permutations
argList	List of arguments to pass to modelFun

Value

dataframe of validation (i.e. response) and prediction values

Examples

# make data
set.seed(1111)
n <- 100
x <- sort(rlnorm(n, meanlog = 0.25, sdlog = 1.5))
a <- 0
b <- 100
cv <- 0.5
y <- (a + x*b) * rlnorm(n, 0, cv)

# fit models
fitLM <- lm(y ~ x - 1)
fitLogLM <- lm(log(y) ~ log(x))
fitGLM <- glm(y ~ x - 1, family=Gamma(link="identity"))

# plot
plot(x, y)
lines(x, predict(fitLM), col=2, lty=2)
lines(x, exp(predict(fitLogLM)), col=3, lty=3)
lines(x, predict(fitGLM, type="response"), col=4, lty=4)
legend("bottomright", legend=c("LM", "logLM", "GLM (Gamma)"), col=2:4, lty=2:4)

# Compare with Aikaike Information Criterium (AIC)
AIC(fitLM, fitLogLM, fitGLM) # not an appropriate comparison



# LM
set.seed(1)
res <- cv.nperm(modelfun="lm", k=5, nperm=9,
  argList=list(
    data = data.frame(x,y), 
    formula = formula(y ~ x - 1)
  )
)
sqrt(mean((res[,1] - res[,2])^2)) # root mean squared error
median(abs((res[,1] - res[,2])/res[,2] * 100)) # median absolute percent error


# log LM
set.seed(1)
res <- cv.nperm(modelfun="lm", k=5, nperm=9,
  argList=list(
    data = data.frame(x,y), 
    formula = formula(log(y) ~ log(x))
  )
)
sqrt(mean((exp(res[,1]) - exp(res[,2]))^2)) # root mean squared error
median(abs((exp(res[,1]) - exp(res[,2]))/exp(res[,2]) * 100)) # median absolute percent error


# GLM
set.seed(1)
res <- cv.nperm(modelfun="glm", k=5, nperm=9,
  argList=list(
    data = data.frame(x,y), 
    formula = formula(y ~ x - 1),
    family = Gamma(link = "identity")
  )
)
sqrt(mean((res[,1] - res[,2])^2)) # root mean squared error
median(abs((res[,1] - res[,2])/res[,2] * 100)) # median absolute percent error

marchtaylor/sinkr documentation built on June 15, 2025, 1:17 a.m.