In bautheac/storethat: Store Bloomberg financial data for later use

library(storethat)
knitr::opts_chunk$set(collapse = T, eval = T, comment = "#>")

storethat together with pullit are the two workhorses for data ETL (extract, transform, load) work in the finRes suite. storethat provides an off-Bloomberg storage facility for financial data retrieved from Bloomberg using the pullit package. The package uses a SQLite database designed for easy and fast I/O (input/output) operations involving pullit data objects.
Install the development version with devtools::install_github("bautheac/storethat").

create

Start by creating a storethat database wherever suits (defaults to home folder):

library(storethat)

db_create(path = "~/data/", n = 1L, verbose = F)

Using the fields dataset from the BBGsymbols package that gathers common Bloomberg data field symbols, the function creates a SQLite database bespoke to host financial data retrieved from Bloomberg using the pullit package. n is a space/speed trade-off parameter; it refers to the number of tables created for hosting historical data for each financial instrument category. Historical data tables are indexed by unique combinations of ticker, field and date ids. Writing new data to a table involves reconstructing its index, a process that continuously increases in strain as the volume of data grows and the index gets bigger; spreading the data across multiple tables helps contain the strain expansion and eventually improves the writing time performance of the database at the expense of disc space. Should disc space be cheaper than time, which is often the case, select a high value for n.

store

storethat provides store methods for all objects returned by the pullit package. Retrieve data from Bloomberg using pullit and store it in an existing storethat database using:

library(pullit)

storethat <- "~/data/storethat.sqlite"; end <- "2018-09-30"; start <- "2016-01-01"
tickers <- c("BP/ LN Equity", "WEIR LN Equity", "AAPL US Equity", "RNO FP Equity")
equity_market <- pull_equity_market(source = "Bloomberg", tickers, start, end, verbose = F, file = storethat)

db_store(equity_market, file = storethat)

library(pullit)

storethat <- here::here("data-raw", "storethat.sqlite"); end <- "2018-09-30"; start <- "2016-01-01"
tickers <- c("BP/ LN Equity", "WEIR LN Equity", "AAPL US Equity", "RNO FP Equity")
equity_market <- pull_equity_market(source = "storethat", tickers, start, end, verbose = F, file = storethat)

db_store(equity_market, file = storethat)

what's in there

Check the content of an existing storethat database at anytime with:

snapshot <- db_snapshot(file = storethat, instrument = "equity", book = "market")
head(snapshot)

update

Update content in an existing storethat database using pullit:

storethat_update(
  instrument = "equity", book = "market", name = "ADM US Equity", file = storethat, verbose = F
  )

Higher order updates are also allowed:

storethat_update(instrument = "equity", file = storethat, verbose = F)

delete

Delete any content from an existing storethat database with:

db_delete(file = storethat, instrument = "equity", book = "market", name = "ADM US Equity")

Or delete everything with:

db_delete(file = storethat)

bautheac/storethat documentation built on June 7, 2021, 11:23 p.m.