In hrbrmstr/gzmem: Better gz in memory compression/decompression

gzmem : Better in memory compression/decompression

A partial resurrection of Rcompression for this.

The following functions are implemented:

• mem_compress: Compress a raw vector
• mem_inflate: Inflate a raw vector compressed

TODO

• [ ] Rewrite in Rcpp
• [ ] Make inflation dynamic vs relying on guessed bufffer size

Installation

devtools::install_git("https://gitlab.com/hrbrmstr/gzmem.git")

options(width=120)

library(gzmem)
library(magrittr)

# current verison
packageVersion("gzmem")

Usage

This package contains functions to compress and decompress raw vectors in R using the zlib library.

It supports 3 different formats: gzip, zlib and raw. These formats all use the deflate algorithm to compress the data, but differ in the headers and footers they wrap around the deflate compressed data:

• gzip has a 10 byte header and an 8 byte footer around the compressed data.
• zlib has a 2 byte header and a 4 byte footer around the compressed data.
• raw has no header or footer, and so is just the deflate compressed data.

For example:

text <- "Hey, the room is getting smaller. No, it's not. *He's* getting *bigger*!"

Shows the following differences between all 3 formats:

gzip <- text %>% 
  charToRaw() %>% 
  mem_compress(format = "gzip")

gzip

zlib <- text %>% 
  charToRaw() %>% 
  mem_compress(format = "zlib")

zlib

raw <- text %>% 
  charToRaw() %>% 
  mem_compress(format = "raw")

raw

These can then be decompressed by selecting the appropriate format:

gzip %>%
  mem_inflate(format = "gzip", 1024) %>% 
  rawToChar()

Test Results

library(gzmem)
library(testthat)

date()

test_dir("tests/")

hrbrmstr/gzmem documentation built on May 17, 2019, 5:08 p.m.