options(htmltools.dir.version = FALSE) # see: https://github.com/yihui/xaringan # install.packages("xaringan") # see: # https://github.com/yihui/xaringan/wiki # https://github.com/gnab/remark/wiki/Markdown options(width=110) options(digits = 4)
.pull-left3[
Given data, an algorithm tunes its parameters to match the data, understand how it works, and make predictions for what will occur in the future.
knitr::include_graphics("images/mldiagram_A.png")
]
.pull-right3[
knitr::include_graphics("images/machinelearningcartoon.png")
]
.pull-left3[
How does Google know what search results you want?
How does Amazon know what products to recommend?
How does Netflix decide what shows you'll want to watch next?
How do Tesla cars recognize objects and predict accidents?
Machine learning drives our algorithms for demand forecasting, product search ranking, product and deals recommendations, merchandising placements, fraud detection, translations, and much more. ~ Jeff Bezos, Amazon founder
]
.pull-right3[
knitr::include_graphics("images/mlexamples.png")
]
knitr::include_graphics("images/MLdiagram.png")
.pull-left3[
Just because an algorithm can fit past (training) data well, does not necessarily mean that it will predict new data well.
knitr::include_graphics("images/stockpen.jpg")
]
.pull-right3[
"Prediction is difficult, especially when it is about the future" ~ Niels Bohr
knitr::include_graphics("images/bohr.jpg")
"An economist is an expert who will know tomorrow why the things he predicted yesterday didn't happen today." ~ Evan Esar
"A prediction about the direction of the stock market tells you nothing about where stocks are headed, but a whole lot about the person doing the predicting" ~ Warren Buffett
]
knitr::include_graphics("images/fittingpredictiondarts_A.png")
knitr::include_graphics("images/fittingpredictiondarts_B.png")
knitr::include_graphics("images/fittingpredictiondarts_C.png")
knitr::include_graphics("images/fittingpredictiondarts_D.png")
set.seed(100) x <- seq(0, 5, length.out = 50) noise <- rnorm(50, mean = 0, sd = 2.5) model_fun <- function(x) {x ^ 3 - 4 * x ^ 2 + .5 * x + 5} y <- model_fun(x) y_obs <- model_fun(x) + noise par(mar = c(3, 4, 3, 1)) par(mfrow = c(1, 3)) plot(x, y_obs, main = "Data", xlab = "", ylab = "", col = "black") # segments(x, y, x, y_obs) # lines(y, y = y_obs) # Plot 1 plot(x, y_obs, main = "Signal", xlab = "", ylab = "", col = "lightgray") curve(model_fun, from = 0, to = 5, add = TRUE, col = "green", lwd = 2) # segments(x, y, x, y_obs) # lines(y, y = y_obs) # Plot 2 plot(x, y_obs, main = "Noise", xlab = "", ylab = "") curve(model_fun, from = 0, to = 5, add = TRUE, col = "darkgray", lwd = .5) segments(x, y, x, y_obs) # lines(y, y = y_obs) # # # Plot 3 # # plot(x, y_obs, main = "A bad model tries to fit everything", xlab = "", ylab = "") # curve(model_fun, from = 0, to = 5, add = TRUE, col = "darkgray", lwd = .5) # # text(.5, 20, "Hey I can draw a line through all points\nI don't have any error!", adj = 0) # # # lines(x, y_obs) # # # Plot 4 # # plot(x, y_obs, main = "A good model will try to focus on the signal", xlab = "", ylab = "") # curve(model_fun, from = 0, to = 5, add = TRUE, col = "blue", lwd = 2) # # text(.5, 20, "I won't try to fit all points because\nI think there is random error", adj = 0) # lines(x, y_obs)
set.seed(100) x <- seq(0, 5, length.out = 50) noise <- rnorm(50, mean = 0, sd = 2.5) model_fun <- function(x) {x ^ 3 - 4 * x ^ 2 + .5 * x + 5} y <- model_fun(x) y_obs <- model_fun(x) + noise par(mar = c(3, 4, 3, 1)) par(mfrow = c(1, 3)) plot(x, y_obs, main = "Data", xlab = "", ylab = "", col = "black") # segments(x, y, x, y_obs) # lines(y, y = y_obs) # Plot 1 plot(x, y_obs, main = "Good Model", xlab = "", ylab = "", col = "darkgray") lines(x, y, col = "blue") # curve(model_fun, from = 0, to = 5, add = TRUE, col = "green", lwd = 2) text(.5, 20, "Fitting error = Medium", adj = 0) text(.5, 15, "Prediction error = Low", adj = 0) # segments(x, y, x, y_obs) # lines(y, y = y_obs) # Plot 2 plot(x, y_obs, main = "Bad Model", xlab = "", ylab = "") text(.5, 20, "Fitting error = None", adj = 0) text(.5, 15, "Prediction error = High", adj = 0) lines(x, y_obs, col = "red") # lines(y, y = y_obs) # # # Plot 3 # # plot(x, y_obs, main = "A bad model tries to fit everything", xlab = "", ylab = "") # curve(model_fun, from = 0, to = 5, add = TRUE, col = "darkgray", lwd = .5) # # text(.5, 20, "Hey I can draw a line through all points\nI don't have any error!", adj = 0) # # # lines(x, y_obs) # # # Plot 4 # # plot(x, y_obs, main = "A good model will try to focus on the signal", xlab = "", ylab = "") # curve(model_fun, from = 0, to = 5, add = TRUE, col = "blue", lwd = 2) # # text(.5, 20, "I won't try to fit all points because\nI think there is random error", adj = 0) # lines(x, y_obs)
.pull-left3[
There are hundreds if not thousands of machine learning algorithms from many different fields.
In this section, we will focus on 4:
| Algorithm|Complexity?| |:------|:----| | Regression| Low / Medium | | Decision Trees| Low | | Random Forests| High | | Support Vector Machines| High |
]
.pull-right3[
Wikipedia lists 57 Categories of machine learning algorithms, each with dozens of examples
knitr::include_graphics("images/wikipediaml.png")
]
.pull-left3[
A_model <- A_fun(formula = y ~., data = data_train, ...)
| Argument| Description| Note |
|------:|:----|:---|
| formula| Formula indicating variables to use| y ~ .
is often used as a catch-all |
| data| The dataset for model training| |
| ...| Optional other arguments| See the function help page for details|
]
.pull-right3[
# Common ways to explore / use a model A_model # Print generic information names(A_model) # Show attributes summary(A_model) # Print summary information predict(A_model, # Predict test data newdata = data_test) plot(A_model) # Visualize the model
]
glm()
.pull-left3[
In regression, the criterion is modeled as the weighted sum of predictors times weights $\beta_{1}$, $\beta_{2}$
One could model the risk of defaulting on a loan as:
$$Risk = Age \times \beta_{age} + Income \times \beta_{income} + ...$$
Training a model means finding values of $\beta_{Age}$ and $\beta_{Income}$ that 'best' match the training data.
knitr::include_graphics("images/regression.png")
]
.pull-right3[
Create regressions using the glm()
function (part of base-R)
# glm() function for regression glm(formula = y ~., # Formula data = data_train, # Training data family, ...) # Optional arguments # Train glm model glm_mod <- glm(formula = risk ~ ., data = data_train) # Predict new data with glm model glm_pred <- predict(glm_mod, newdata = data_test)
]
FFTrees()
.pull-left3[
In decision trees, the criterion is modeled as a sequence of logical Yes or No questions.
knitr::include_graphics("images/defaultfft.png")
]
.pull-right3[
Create decision trees using the FFTrees
package
# Load the FFTrees package library(FFTrees) # Main Function FFTrees(formula = y ~ ., data, ...) # Train FFTrees model FFTrees_mod <- FFTrees(formula = default ~ ., data = loan_data, main = "Default Risk") # Plot the tree plot(FFTrees_mod) # Predict new data with FFTrees model FFTrees_loan_pred <- predict(FFTrees_mod, data = data_test)
]
.pull-left3[
e1071::svm()
knitr::include_graphics("images/supportvectormachine.png")
# Creating support vector machine model library(e1071) svm_mod <- svm(formula = risk ~ ., data = loan_data, kernel, degree, gamma, ...)
]
.pull-right3[
randomForest::randomForest()
knitr::include_graphics("images/randomforest.png")
# Creating random forest model library(randomForest) rf_mod <- randomForest(formula = risk ~ ., data = loan_data, mtry, cutoff, sampsize, ...)
]
.pull-left3[
If you're really into machine learning, packages such as mlr
and caret
can automate much of the the machine learning process.
knitr::include_graphics("images/mlrcaret.png")
]
.pull-right3[
In the practical, we will go through the basic steps "by hand" so you can see the process:
# Create training and test data data_train <- ... data_test <- ... # Train models on training data model_A <- A_fun(formula = y ~ ., data = data_train) # Model A predictions pred_A <- predict(model_A, newdata = data_test) # Calculate Model A error pred_err_A <- mean(abs(pred_A - data_test$y)) # Compare to Models B, C, D...
]
1805 - 1809: Legendre and Gauss discover least squares. Soon after Galton defines Regression in a biological context, followed by Pearson for purely statistical analyses.
1952: Arthur Samuel creates first computer learning program for learning checkers and coins the term Machine Learning in 1959.
1957: Frank Rosenblatt creates first Neural Network to simulate the thought process of the human brain.
1963: First algorithm for Support Vector Machines is developed by Vapnik & Chervonenkis.
1967: Nearest neighbor algorithm is developed for classification
1984: Breiman & Olshen publish the CART algorithm for Decision Trees, followed by Quinlan who publishes the ID3 algorithm followed by C4.5
1986: Rina Dechter introduces Deep Learning, with many subsequent updates in the 2000s.
1995: Tin Kam Ho develops first algorithm for Random Forests
Sources: Wikipedia, Bernard Marr, "A Short History of Machine Learning", Forbes.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.