# Introduction to RCANE" In rcane: Different Numeric Optimizations to Estimate Parameter Coefficients

```knitr::opts_chunk\$set(
collapse = TRUE,
comment = "#>"
)
```

The goal of RCANE is to create an easy-to-use interface for gradient descent algorithms.

## Introduction

Rcane is a package which contains different numeric optimization algorithms for parameter estimation in linear regression.

4 algorithms are:

• Batch Gradient Descent

• Stochastic Gradient Descent

• Mini-batch Gradient Descent

• Coordinate Descent

Gradient descent is a first-order iterative optimization algorithm for finding minimum of a function. More information on Gradient Descent can be found here

## Package flow

The flow can be given as follows ```library(rcane)
```

## rlm for Batch Gradient Descent

### without boldDriver

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
```

### with boldDriver

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd", boldDriver = TRUE)
```

## rlm for Stochastic Gradient Descent

### without AdaGrad

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "sgd")
```

### with Adagrad

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "sgd", AdaGrad = TRUE)
```

## rlm for Mini-batch Gradient Descent

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "mini-bgd")
```

## rlm for Coordinate Descent

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "cd")
```

## coef

Get the coefficients of the model

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
coef(bgd)
```

## fitted

Get the fitted values of the model

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
head(fitted(bgd))
```

## formula

Get the formula of the applied model

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
formula(bgd)
```

## print

Print the coefficients of the model

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
print(bgd)
```

## resid

Get the residuals of the fitted model

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
head(resid(bgd))
```

## predict

Apply the predictions on the new data set

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
head(predict(bgd, newdata=iris))
```

## learning rate

This is the rate at which the algorithm would converge. The learning rate should be chosen carefully. If the learning rate is too high, the algorithm would overshoot and never converge. If the learning rate is too low, the algorithm would get slower and not converge before max.iter iterations.

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd", alpha=0.2)
```

## max iterations

The function would terminate after iterations

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd", alpha=0.2, max.iter = 500)
```

## precision

The function would terminate if the parameter estimates do not change by more than \<precision> for a given iteration.

```rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd", precision = 0.0002)
```

## plotLoss

The function prints the loss function for each iteration

```bgd <- rlm(formula = Petal.Length ~ Petal.Width, data = iris, method = "bgd")
plotLoss(bgd)
```

## Try the rcane package in your browser

Any scripts or data that you put into this service are public.

rcane documentation built on May 1, 2019, 10:11 p.m.