diagnose_outlier.tbl_dbi: Diagnose outlier of numerical variables in the DBMS

In dlookr: Tools for Data Diagnosis, Exploration, Transformation

Diagnose outlier of numerical variables in the DBMS

Description

The diagnose_outlier() produces outlier information for diagnosing the quality of the numerical(INTEGER, NUMBER, etc.) column of the DBMS table through tbl_dbi.

Usage

## S3 method for class 'tbl_dbi'
diagnose_outlier(.data, ..., in_database = FALSE, collect_size = Inf)

Arguments

.data	a tbl_dbi.
...	one or more unquoted expressions separated by commas. You can treat variable names like they are positions. Positive values select variables; negative values to drop variables. If the first expression is negative, diagnose_outlier() will automatically start with all variables. These arguments are automatically quoted and evaluated in a context where column names represent column positions. They support unquoting and splicing.
in_database	Specifies whether to perform in-database operations. If TRUE, most operations are performed in the DBMS. if FALSE, table data is taken in R and operated in-memory. Not yet supported in_database = TRUE.
collect_size	a integer. The number of data samples from the DBMS to R. Applies only if in_database = FALSE.

Details

The scope of the diagnosis is the provide a outlier information. If the number of outliers is small and the difference between the averages including outliers and the averages not including them is large, it is necessary to eliminate or replace the outliers. You can use grouped_df as the group_by() function.

Value

an object of tbl_df.

Outlier Diagnostic information

The information derived from the numerical data diagnosis is as follows.

• variables : variable names

• outliers_cnt : number of outliers

• outliers_ratio : percent of outliers

• outliers_mean : arithmetic average of outliers

• with_mean : arithmetic average of with outliers

• without_mean : arithmetic average of without outliers

See vignette("diagonosis") for an introduction to these concepts.

See Also

diagnose_outlier.data.frame, diagnose.tbl_dbi, diagnose_category.tbl_dbi, diagnose_numeric.tbl_dbi.

Examples

# If you have the 'DBI' and 'RSQLite' packages installed, perform the code block:
if (FALSE) {
library(dplyr)

# connect DBMS
con_sqlite <- DBI::dbConnect(RSQLite::SQLite(), ":memory:")

# copy heartfailure to the DBMS with a table named TB_HEARTFAILURE
copy_to(con_sqlite, heartfailure, name = "TB_HEARTFAILURE", overwrite = TRUE)

# Using pipes ---------------------------------
# Diagnosis of all numerical variables
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier()
  
# Positive values select variables, and In-memory mode and collect size is 200
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier(platelets, sodium, collect_size = 200)
  
# Negative values to drop variables
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier(-platelets, -sodium)
  
# Positions values select variables
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier(5)
  
# Negative values to drop variables
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier(-1, -5)

# Using pipes & dplyr -------------------------
# outlier_ratio is more than 1%
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  diagnose_outlier()  %>%
  filter(outliers_ratio > 1)

# Using group_by() ----------------------------
con_sqlite %>% 
  tbl("TB_HEARTFAILURE") %>% 
  group_by(death_event) %>% 
  diagnose_outlier() 
  
# Disconnect DBMS   
DBI::dbDisconnect(con_sqlite)
}

dlookr documentation built on May 29, 2024, 2 a.m.