bom
: Tools to Identify and Work with Byte Order Marks
Byte order marks (BOM) appear at the beginning of a file or buffer and provide information about the encoding of the contents. R provides facilities to work with files and connections with BOMs but there are situatons where these facilities are not sufficient. Tools are provided to identify the presence and type of byte order marks in files and R objects as well as remove them.
The following functions are implemented:
file_bom_type
: Get BOM type (file)file_has_bom
: Test if a file has a BOMraw_bom_type
: Get BOM type (raw vector)raw_has_bom
: Test if a raw vector has a BOMconnection
sans BOM or identify there is a BOMdevtools::install_git("https://gitlab.com/hrbrmstr/bom.git")
options(width=120)
There are some basic examples in the Usage section, but this may be a better illustration. Say you have a CSV file:
fil <- system.file("examples", "stop_times.txt", package="bom")
And, say you want to read it in with a more modern CSV reader:
library(readr) df <- read_csv(fil)
Let's look at that file:
print(df, n=1)
Hrm…why are those backticks around trip_id
? Isn't it just a regular string?
print(colnames(df)[1])
It sure looks that way, but looks can be deceiving:
print(charToRaw(colnames(df)[1]))
Those strange characters at the beginning are a byte order mark (BOM). We can test for it being there and work around it:
library(bom) if (file_has_bom(fil)) { n <- switch(file_bom_type(fil), `UTF-8`=3, 2) df <- read_csv(readBin(fil, "raw", file.size(fil))[-(1:n)]) } print(df, n=1) charToRaw(colnames(df)[1])
Note that the built-in read.csv()
can be used with encoding="UTF-8-BOM"
and you can even use that encoding on non-binary connections, but you end up having to type convert and tibble convert that object so you're basically rewriting (badly) readr::read_csv()
.
library(bom) # current verison packageVersion("bom") file_has_bom(system.file("examples", "stops.txt", package="bom")) file_has_bom(system.file("examples", "stop_times.txt", package="bom")) raw_has_bom(readBin(system.file("examples", "stop_times.txt", package="bom"), "raw", 4)) file_bom_type(system.file("examples", "stops.txt", package="bom")) file_bom_type(system.file("examples", "stop_times.txt", package="bom")) raw_bom_type(readBin(system.file("examples", "stop_times.txt", package="bom"), "raw", 4))
library(bom) library(testthat) date() test_dir("tests/")
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.