csv_to_sql_db: FCC file to SQLite Database via chunking

In kdmulligan/processFCC: processFCC: cleaning Fixed Broadband Data from FCC Form 477

FCC file to SQLite Database via chunking

Description

Reads FCC csv file in chunks while writing each chunk to a SQL database before reading in the next chunk. DBI connection and FCC column names from db_colname function must be provided.

Usage

csv_to_sql_db(
  csv_file,
  con,
  pre_process_size = 1000,
  chunk_size = 50000,
  show_progress_bar = TRUE,
  db_colname
)

Arguments

csv_file	Name of raw FCC csv file to covert to SQL database. This is relative to the working directory unless an absolute file path is provided.
con	A DBIConnection object, as returned by dbConnect.
pre_process_size	Number of rows with which to initialize SQL db.
chunk_size	Number of rows to include for each chunk
show_progress_bar	Display progress bar? (default TRUE)
db_colname	Column names for FCC dataset, as returned by processFCC::get_colname

Value

Returns a SQLite database named "table_FCC" inside provided DBI connection

Examples

## Not run: 
# set month and year of FCC data set to use
month = "December"
year = 2020

# download desired dataset
download_FCC(year, month)

# get proper column names
use_colnam <- get_colname(year, month)

# establish database connection
con <- DBI::dbConnect(SQLite(), dbname = "fcc.sqlite")

# filenames are constructed in the following format within `download_FCC`
filename <- paste0("FCC_fixed_brdbd_", month, "_", year, ".csv")
csv_to_sql_db(filename, con, db_colname = use_colnam)

# the SQL database created with `csv_to_sql_db` can be accessed in the
# following manner:
table <- tbl(con, "table_fcc")

# close the connection
dbDisconnect(con)

## End(Not run)

kdmulligan/processFCC documentation built on Oct. 30, 2024, 7:43 p.m.