megautils: Collection of Utility Functions

Utility functions for reading remote tabular data into tibbles and caching them into disk for future use. If your data is stored in Google Cloud Storage, see gcs_data.

materialize(reference, ...)

size(reference, ...)

import(
  reference,
  expr = .x,
  ignore_cache = FALSE,
  global = TRUE,
  overwrite = TRUE
)

cached_table(name)

db_table(conn, name)

query_table(conn, query, name)

pquery_table(conn, query, name, query_parameters)

import_all(...)

table_references(ref_type, ref_parameters = l(), ...)

`reference`	a lazy table reference.
`expr`	a post-processing expression to be run after the tibble has been loaded. The tibble can be referenced by '.x'.
`ignore_cache`	if set to 'TRUE', forces import to download data from the remote source again, even if it's already cached. Defaults to 'FALSE'.
`global`	if set to 'FALSE', causes import to bind the table to a reference in the calling environment instead of the global environment. Defaults to 'TRUE'.
`overwrite`	if set to 'TRUE', causes import to act as a noop if there is already a binding with name 'name' in the target environment. Useful for when the table is large and has already been loaded with .RData.
`name`	a string containing the name of the lazy table reference. For 'db_table', this has to match the name of the table in the database.
`conn`	a database connection created with 'DBI::dbConnect'. Can be set to 'NULL' if the user is sure there is a cached version on disk and the reference is being used with 'import'.
`query`	a string containing a (possibly parametric) SQL query.
`query_parameters`	a list of named parameters (e.g. ‘list(par1 = ’value1', par2 = 'value2')' to be substituted into the query.
`ref_type`	a lazy reference type. Either 'db_table', 'cached_table', or, less commonly, 'parametric_table' or 'cached_table'.
`ref_parameters`	a list of named parameters to be passed to 'ref_type' with each call.

'db_table()': lazy reference to a database table which can be read into memory as a tibble and cached into disk.
'query_table()': lazy reference to a query result which can be read into memory as a tibble and cached into disk.
'pquery_table()': same as 'query_table()', but taking a parametric query and a list of named query parameters.
'cached_table()': lazy reference to a table which has been previously cached. This exists mainly so that cached tables can be read without setting up a database connection, which is required by all other reference types.
'gcs_table()': lazy reference to tabular data stored into Google Cloud Storage.
'materialize()': materializes a lazy reference into an actual tibble by accessing the remote database. This function never reads or writes to the local cache.
'size()': returns the size, in megabytes, of a lazy table reference without materializing it. This may be useful for probing the size of a database table before deciding to download it.
'import()': materializes a lazy reference into an actual tibble by either _i)_ reading from the local cache, or _ii)_ calling 'materialize()' and then caching results if nothing is cached yet. The resulting tibble will be automatically bound to a variable with the table's 'name' in the global environment.
'import_all()': utility function. Equivalent to looping through a set of lazy references and calling import in each.
'cached_tables()': utility function for creating several cached table references from their names alone.