In emeyers/SDS230: Tools for the class Data Exploration and Analysis

$\$

# install.packages("gapminder")

SDS230::download_data("amazon.rda")

SDS230::download_image("qq-plot.png")

# install.packages("latex2exp")

library(latex2exp)

library(dplyr)


#options(scipen=999)


knitr::opts_chunk$set(echo = TRUE)

# hide all plot output - useful for printing the code
# knitr::opts_chunk$set(fig.show='hide')

set.seed(123)

$\$

Overview

• ggplot bonus features
• visual hypothesis tests
• functions
• qqplots

$\$

Part 1: Review of ggplot

ggplot2 is a R package created by Hadley Wickham that implements Leland Wilkinson's concept of a grammar of graphics. In the grammar of graphics (and ggplot) data graphics are composed of:

• a frame
• glyphs
• scales and guides
• facets
• layers
• themes

$\$

Part 1.1: Color scales using scale_color_brewer

Let's use the gapminder data to construct a visualization using this grammar that has:

1. A basic frame constructed from called the ggplot() function

2. A geom of points where:

3. gdpPercap is mapped to the x position
4. lifeExp is mapped to the y position

5. continent is mapped to color

6. Transform the scales so that

7. The x values are on a $log_10$ scale
8. The color scale using a different qualitative color brewer pallet

library(ggplot2)
library(gapminder)


                   # create the basic frame
                     # add a geom with 3 aesthetic mappings 
                         # change the scales
                   # change the color pallet

$\$

Part 1.2: ggplot themes

Let's continue to use the gapminder data to construct a visualization using this grammar that has:

1. The gapminder data filtered to only have data from France, Haiti and the United States

2. A geom of line where:

3. year is mapped to the x position

4. lifeExp is mapped to the y position

5. Faceting based on the country

6. A second glyph layer with a red vertical line at the year 1969

7. The Wall Street Journal theme from the ggthemes package

library(dplyr)
library(ggthemes)


# plot comparing countries with different themes

$\$

Part 1.3: geom_text()

We can also create: - Text as a glyph using geom_text() - Annotations using annotate(geom_type, geom_properties) - Manually modify the theme using theme() with different arguments

Let's create a plot where:

1. It only has data from 2007

2. A text glyph with the following properties:

3. gdpPercap is mapped to the x position
4. lifeExp is mapped to the y position
5. continent is mapped to color

6. country is mapped to the text label

7. Let's use a log 10 scale

8. Let's turn off the legend by modifying the theme

9. Let's add a text annotation with the following attributes

10. x = 600
11. y = 80
12. label = "this plot is messy :("

# plot using a text geom...

$\$

Part 2: Additional ggplot themes and features

$\$

Part 2.1: Plotly

The plotly package can be used to create interactive visualizations.

Let's recreate an interactive version of the gapminder data from 2007 using a geom_point() and:

• gdpPercap is mapped to the x position
• lifeExp is mapped to the y position
• continent is mapped to color
• country is mapped to the name

We will save our plot to an object g and then we can use ggplotly(g) to create an interactive visualization.

library(gapminder)
library(plotly)



# plot 2007 gapminder data





# create an interactive version of the plot using plotly

$\$

Part 2.2: emojis

Everyone loves emojis!

Let's use the motor trends car data to plot miles per gallon (mpg) as a function of weight using the proper glyph.

# devtools::install_github("dill/emoGG")


library(emoGG)


# use emojis as geoms

$\$

Part 2.3: animations

We can crate animated gifs too...

#install.packages('png')
#install.packages('gifski')
#install.packages('gganimate')


# library(gganimate)
# 
# 
# ggplot(gapminder, aes(gdpPercap, lifeExp, size = pop, colour = country)) +
#   geom_point(alpha = 0.7, show.legend = FALSE) +
#   scale_colour_manual(values = country_colors) +
#   scale_size(range = c(2, 12)) +
#   scale_x_log10() +
#   facet_wrap(~continent) +
#   # Here comes the gganimate specific bits
#   labs(title = 'Year: {frame_time}', x = 'GDP per capita', y = 'life expectancy') +
#   transition_time(year) +
#   ease_aes('linear')

If you would like more practice with ggplot on a very challenging problem, please try out optional homework 100. You will not turn this homework in and it will not count toward your grade in the class, but it is a good way to get more practice with ggplot.

You can download the homework in the using way using: SDS230::download_data("unisex_names.rda")

$\$

Part 3: Visual hypothesis tests

In "visual hypothesis tests", we assess whether we can identify which plot contains relationships in the data, from plots that show scrambled versions of the data. For more information about these tests, and an R package that implements these tests, see:

• https://cran.r-project.org/web/packages/nullabor/vignettes/nullabor.html
• https://vita.had.co.nz/papers/inference-infovis.pdf

$\$

Part 3.1: An example problem

Let's run a visual hypothesis test to see if there is a correlation between the number of pages in a book and the price. We can start by stating the null and alternative hypothesis:

In words:

Null hypothesis:

Alternative hypothesis:

In symbols:

$H_0: $

$H_A: $

Let's now run the visual hypothesis test. Where we have:

• The "observed visualization" (step 2) is the plot of the actual data.

• The "null visualizations" (step 3) are visualizations of data where the relationship between weight and mpg has been shuffled.

• The "p-value/decision" (step 4-5) is whether we can identify the real plot from the shuffled plots.

$\$

Part 3.2: Running the analysis

Let's use the nullabor package to run the analysis. We will use the null_permute() function to randomly shuffle the List.Price variable and we will use the lineup() to create a data frame that has the real data and 19 other randomly shuffled data sets. We can then plot the real and shuffled data sets using ggplot() to see if we can identify the real one. We can also use the decrypt() function to reveal the answer about which data set is the real one.

# install.packages("nullabor")

library(nullabor)
library(ggplot2)

load("amazon.rda")



# create the lineup




# visualize the lineup

The nullabor package can be used to examine other types of relationships, including examining differences in word useage (word clouds) and assessing whether there are trends on a map.

$\$

Part 4: Writing functions in R and conditional statements

Part 4.1: Writing functions

We can write functions in R using the function() function!

Let's write a function called cube_it() that takes a value x and returns x cubed.

# the square root function



# a function that takes a value to the third power

$\$

Part 4.2: Addtional function arguments

We can specify additional arguments which can also have default values.

Let's write a function called power_it which takes a value x and a value pow and returns x to the pow power. Let's also have the default value of the pow argument be 3.

# a function that takes a value to the third power

$\$

Part 4.3: Writing our own permutation test function for correlation

Let's write our permutation test for testing whether there is a significant correlation in data.

We can then test it to see if there is a correlation between the price of a book and the number of pages in a book.

# write a cor_permtest function that tests where there is a significant correlation
# between the values in two vectors



  # calculate the observed statistic



  # create the null distribution 





  # Calculate the two-tailed p-value




  # return the p-value




# end the function




# test the function to see if there is a correlation between the number of pages in a book
# and the price of a book 

Try this at home: write a function that can run a permutation test comparing two means from two different data samples (e.g., a function that can run a permutation test to assess whether cognitive abilities different depending on whether participants took a Gingko supplement or a placebo).

$\$

Part 5: Examining how data is distributed using quantile-quantile plots

We can check if empirical data seems to come from a particular distribution using quantile-quantile plots (qqplots). qqplots plot the sorted data values in your sample as a function of the theoretical quantile values (at evenly spaced probability areas).

Below is an illustration of quantile values for 10 data points that come from a normal distribution. If we have 10 data points in our sample, then to create a qqplot comparing our data to a standard normal distribution we would plot our data as a function of these theoretical quantile values. If the plot falls along a diagonal line, this indicates our data comes from the standard normal distribution

Also see this explanation

$\$

Part 5.1

Let's create a qqplot to assess whether the thickness of books on Amazon are normally distributed.

# get the heights of all baseball players
load("amazon.rda")
thickness <- amazon$Thick



# view a histogram of book thicknesses



# create an sequence of values between 0 and 1 at even spaces



# get the quantiles from these values



# create the qqplot

$\$

Part 5.2

We can also use the qqnorm() function to do this more easily when comparing data to the normal distribution. Or, if we want a better visualization, we can use the qqPlot() function in the car package.

# use the qqnorm function


# use the qqPlot function from the car package
#install.packages("car")

This data is pretty normal as can see in the plots above. Let's look at some highly skewed data.

# data that is skewed to the right
exp_data <- rexp(1000)

# visuialize a histogram of the data
hist(exp_data, breaks = 50, 
     main = "Data from an exponential distribution", 
     xlab = "Data values")


# create a qqplot of the data using the qqnorm function





# data that is skewed to the left
exp_data <- -1 * rexp(1000)

hist(exp_data, breaks = 50, 
     main = "Left skewed data", 
     xlab = "Data values")



# create a qqplot of the data using the qqnorm function

emeyers/SDS230 documentation built on Jan. 18, 2024, 1:01 a.m.