bom : Tools to Identify and Work with Byte Order Marks

Byte order marks (BOM) appear at the beginning of a file or buffer and provide information about the encoding of the contents. R provides facilities to work with files and connections with BOMs but there are situatons where these facilities are not sufficient. Tools are provided to identify the presence and type of byte order marks in files and R objects as well as remove them.

The following functions are implemented:

TODO

Installation

devtools::install_git("https://gitlab.com/hrbrmstr/bom.git")
options(width=120)

There are some basic examples in the Usage section, but this may be a better illustration. Say you have a CSV file:

fil <- system.file("examples", "stop_times.txt", package="bom")

And, say you want to read it in with a more modern CSV reader:

library(readr)

df <- read_csv(fil)

Let's look at that file:

print(df, n=1)

Hrm…why are those backticks around trip_id? Isn't it just a regular string?

print(colnames(df)[1])

It sure looks that way, but looks can be deceiving:

print(charToRaw(colnames(df)[1]))

Those strange characters at the beginning are a byte order mark (BOM). We can test for it being there and work around it:

library(bom)

if (file_has_bom(fil)) {
  n <- switch(file_bom_type(fil), `UTF-8`=3, 2)
  df <- read_csv(readBin(fil, "raw", file.size(fil))[-(1:n)])
}

print(df, n=1)

charToRaw(colnames(df)[1])

Note that the built-in read.csv() can be used with encoding="UTF-8-BOM" and you can even use that encoding on non-binary connections, but you end up having to type convert and tibble convert that object so you're basically rewriting (badly) readr::read_csv().

Usage

library(bom)

# current verison
packageVersion("bom")

file_has_bom(system.file("examples", "stops.txt", package="bom"))

file_has_bom(system.file("examples", "stop_times.txt", package="bom"))

raw_has_bom(readBin(system.file("examples", "stop_times.txt", package="bom"), "raw", 4))

file_bom_type(system.file("examples", "stops.txt", package="bom"))

file_bom_type(system.file("examples", "stop_times.txt", package="bom"))

raw_bom_type(readBin(system.file("examples", "stop_times.txt", package="bom"), "raw", 4))

Test Results

library(bom)
library(testthat)

date()

test_dir("tests/")


hrbrmstr/bom documentation built on May 17, 2019, 4:55 p.m.