In JSzitas/categoryEncodings: Category Variable Encodings

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path = "man/figures/README-",
  out.width = "100%"
)

categoryEncodings

categoryEncodings intends to provide a fast way to encode 'factor' or qualitative variables through various methods. The packages uses data.table as the backend for speed, with as few other dependencies as possible. Most of the methods are based on the paper of Johannemann et al.(2019) - Sufficient Representations for Categorical Variables (arXiv:1908.09874).

The current version features automatic inference of factors and uses a very simple heuristic for encoding, as well as allowing manual controls.

Installation

You can install the latest version of categoryEncodings from github using the devtools package

``` {r, eval = FALSE} devtools::install_github("JSzitas/categoryEncodings")

**NOTE:** The latest stable version available from **CRAN** contains features deprecated in the current development version - I hope to resolve this soon, and publish the development version. 

## Example 

Here we want to encode all of the factors in a given data.frame.

````r
library(categoryEncodings)
# create some example data
data_fm <- cbind( data.frame( 
   matrix( rnorm(5*100),ncol = 5)),
           sample(sample(letters, 10), 100, replace = TRUE),
           sample(sample(letters, 20), 100, replace = TRUE),
           sample(sample(1:10, 5), 100, replace = TRUE),
           sample(sample(1:50, 35), 100, replace = TRUE ),
           sample(1:2, 100, replace = TRUE ))
colnames(data_fm)[6:10] <- c( "few_letters",  "many_letters",
                              "some_numbers", "many_numbers",
                              "binary" ) 
# it does not matter how many factor variables there are, whether they are encoded as factors
# and whether you supply a method to encode them by - some simple inference of factors is done
# based on the number of distinct values in every variable - over a certain threshold 
# a variable is deemed as essentially a factor, and treated as such for conversion 
# you will be notified of which variables are being converted via a warning
result <- encoder(data_fm)
# note that due to the data.table back-end, the result has to be saved to an object to be 
# visible: otherwise printing is suppressed.   
head(result$encoded)
````

We also recover a **function closure** which we can reuse to fit new data, as long as it conforms to the same format: 
```r
# to fit to any dataset you can either call it directly - it is a single argument function
data_fm_encoded <- result$fitted_encoder(data_fm)

# or rename it, and stash it away for later use
encoding_function <- result$fitted_encoder

You also get a "de-encoding" function -

deencoder <- result$fitted_deencoder

This undoes the "encoding", effectively returning the original data. This can be quite useful for interpretability methods, where the interpretation becomes easier for un-encoded data. Note that this sadly does not maintain the order of the data from the original - and some attributes may be lost. Nonetheless, the recovered data is almost the same, and equivalent for all practical purposes:

original <- data.table::data.table(data_fm)
deencoded <- deencoder( result$encoded ) 


all.equal( data.table::setorder(original), 
           data.table::setorder(deencoded), 
           check.attributes = FALSE )

Contributing

Please do contribute to the projects, all contributions are welcome, as long as people keep things civil - there is no need for negativity, hatred, and rudeness. Also, please do refrain from adding unnecessary dependencies (Ex: pipe) to the package (such pull requests as would add an unnecessary dependency will be denied/ suspended until the code can be made dependency free). This package wants to be as lightweight as possible - even if this means the code is a bit harder to write and maintain.

JSzitas/categoryEncodings documentation built on Sept. 29, 2021, 9:54 a.m.