gradient_descent: Do k-fold cross-validation for gradient descent method

In Zebedial/bis557: The package created for BIS557 Homework 1

Description

This function helps to conduct a k-fold cross-validation for the gradient descent method given model and data. The user need to specified input for the function gradient_descent. More details see gradient_descent.

Implement gradient descent for ordinary least square. Gradient descent can only handle design matrix with full rank. For design matrix with problem of collinearity, if perfect collinearity presents, the OLS estimate computed by gradient descent contains redundant estimate corresponding to variables should be omitted; For other cases with strong collinearity, the method may not be convergent (i.e. the lm_patho data). This function will pass the data to the the function "linear_model" when it cannot be able to handle the problem

Usage

gradient_descent(
  formula,
  data,
  contrasts = NULL,
  gamma = 1e-04,
  maxiter = 1e+06,
  tolt = 1e-12
)

gradient_descent(
  formula,
  data,
  contrasts = NULL,
  gamma = 1e-04,
  maxiter = 1e+06,
  tolt = 1e-12
)

Arguments

formula	A symbolic description of the model to be fitted. This should be a formula class argument.
data	Specification of a dataframe that contains the variables in the model.
contrasts	A list of contrasts.
gamma	Specification of a learning rate that adjust the OLS estimates along gradient.
maxiter	Maximum number of iterations for the updating process of OLS estimates.
tolt	A tolerance that bounds the difference between the current SSR and the updated SSR.
...	other inputs that need to be passed to the function ridge_regression.
folds	Number of folds used in the k-fold cross-validation

Value

a list containing the raw vector of residuals and the computed mean squared error (out of sample accuracy)

A list of component that imitates the output of lm() function. Including estimated coefficients for predictors specified in the formula. also may return a warning if the iterations exceed the maximum iteration number.

Examples

data(iris)
cv.gradient_descent(Sepal.Length ~ ., iris)
data(iris)
gradient_descent(Sepal.Length ~ ., iris)

Zebedial/bis557 documentation built on Dec. 21, 2020, 2:16 a.m.