In UBC-MDS/tidyplusR: An extension on tidyverse to perform specified tasks

Introduction

The tidyplusR package is an essential data cleaning package with features like missing value treatment, data manipulation and displaying data as markdown table for documents. The package adds a few additional functionality on the existing data wrangling packages in popular statistical software like R. The objective of this package is to provide a few specific functions to solve some of the pressing issues in data cleaning.

Contributors:

• Akshi Chaudhary : akshi8
• Tina Qian : TinaQian2017
• Xinbin Huang: xinbinhuang

Installation

You can install tidyplusR from github using the following:

# install.packages("devtools")
devtools::install_github("UBC-MDS/tidyplusR")

Functions included:

Three main parts include different functions in tidyplusR

• Data Manipulation : Datatype cleansing
• typemix
  • The function helps to find the columns containing different types of data, like character and numeric. The input of the function is a data frame, and the output of the function will be a list of 3 data frames.

• cleanmix

  • The function helps to clean our data frame. After knowing the location of discrepancy of data types, one can use this function to keep a type of data in certain columns.
  • Here, the input will be the output by typemix function, name of the column (a vector of the name of columns) that they want to clean, the type of data they want to work on, and if we want to keep or delete the certain type. The output will be a data frame like the original type but with specified data type in certain columns deleted.

• Missing Value Treatment : Basic Imputation using impute

  • Imputation: replace missing values in a column of a dataframe, or multiple columns of dataframe based on the method of imputation

  • (Method = 'Mean') replace using mean

  • (Method = 'Median') replace using median
  • (Method = 'Mode') replace using mode

• Markdown Table:

• md_new(): This function creates a bare bone for generating a markdown table. Alignments, and size of the table can be input by users.

  • Input: the size of table (number of rows and number of columns)
  • Output: a character vector of the source code.
• md_data(): This function converts a dataframe or matrix into a markdown table format.
  • Input: a matrix or dataframe
  • Output: a character vector of the source code.

Examples

This is a basic example which shows you how to solve a common problem:

Datatype cleansing

The section has two functions, typemix and cleanmix.

• The input for typemix function is a data frame, and the output is a list of 3 data frames. The first one is the same as the input data frame, the second one tells you the location and types of data in the columns where there is type mixture. The third data frame is a summary of the second data frame.

• The input for cleanmix function is the result from typemix function, the column(s) you want to work on, the type(s) of data you want to keep/delete, and if you want to keep/delete the instances specified.

library(tidyplusR)


dat<-data.frame(x1=c(1,2,3,"1.2.3"),
                x2=c("test","test",1,TRUE),
                x3=c(TRUE,TRUE,FALSE,FALSE))
#Input data with mixed datatypes
dat 

#Identified and cleaned(removed) datatypes based on the types mentioned
tidyplusR::cleanmix(typemix(dat),column=c(1,2),type=c("number","character"))

Missing Value imputation

• This function requires a dataframe as an input for missing value treatment using mean/median/mode

### Dummy dataframe
dat <- data.frame(x=sample(letters[1:3],20,TRUE), 
                  y=sample(letters[1:3],20,TRUE),
                  w=as.numeric(sample(0:50,20,TRUE)),
                  z=sample(letters[1:3],20,TRUE), 
                  b = as.logical(sample(0:1,20,TRUE)),
                  a=sample(0:100,20,TRUE),
                  stringsAsFactors=FALSE)

dat[c(5,10,15),1] <- NA
dat[c(3,7),2] <- NA
dat[c(1,3,5),3] <- NA
dat[c(4,5,9),4] <- NA
dat[c(4,5,9),5] <- NA
dat[,4] <- factor(dat[,4] )
dat[c(4,5,9),6] <- NA
#Input data with missing values
dat 

#### Calling impute function
#Missing value replaced with method = mode
tidyplusR::impute(dat,method = "mode")   ## method can be replaced by median and mean as well

Markdown table

• md_new() can create an empty markdown table by specifying the number of columns and number of rows.

## default: ncol = 2 and nrow = 2, alignment = "l"
md_new()

## 3 by 3 table
md_new(nrow = 3, ncol = 3)

## different alignments:
md_new(nrow = 1, align = "c")
md_new(nrow = 1, align = "r")

## providing header
h <- c("foo", "boo")
md_new(header = h)

• md_data() can create an markdown table given input as matrix of data frame.

md_data(mtcars, row.index = 1:3, col.index = 1:4)

## alignment to right
md_data(mtcars, row.index = 1:3, col.index = 1:4, align = "r")

## provide header
md_data(mtcars, row.index = 1:3, col.index = 1:4, header = c("a","b","c","d"))

## not include row names
md_data(mtcars, row.index = 1:3, col.index = 1:4, row.names = F)

Existing features in R ecosystem similar to tidyplusR

• Data Manipulation

• dplyr and tidyverse these R libraries have very powerful data wrangling tools but with tidyplus user can explicitly perform string processing/ datatype conversion without affecting the overall column type (which is convenient when you have really messed up data with mix of strings and numbers)

• Missing Value treatment

• R doesn't have imputation methods which use EM algorithm for missing value treatment, which in fact is very efficient and accurate MICE package in R do provide limited imputation using mean, mode, etc.
• Markdown table in R
• R has library Kable which can output a dataset in the form of a markdown table but with tidyplus user will have more freedom with data types and formatting.

License

MIT

Contributing

This is an open source project. Please follow the guidelines below for contribution. - Open an issue for any feedback and suggestions. - For contributing to the project, please refer to Contributing for details.

UBC-MDS/tidyplusR documentation built on May 25, 2019, 1:36 p.m.