process_files: Process importing files in stock database

In chriszheng2016/zstmodelr: Stock Factor Models for China Stock Market Investment

Description

Generic function to process importing files in stock database.

Usage

process_files(
  stock_db,
  data_source = get_datasource(stock_db),
  retry_log = NULL,
  log_file_prefix = "process_files_log",
  log_dir = "./log",
  ...
)

## S4 method for signature 'gta_db'
process_files(
  stock_db,
  data_source = get_datasource(stock_db),
  retry_log = NULL,
  log_file_prefix = "process_files_log",
  log_dir = "./log",
  ...
)

Arguments

stock_db	A stock database object to operate.
data_source	Data source info produced by get_datasource(stock_db)
retry_log	Log file for re-processing tables with recording failure in log file. If NULL, it will process all input_files in data_source, otherwise only process on these input_files with recording failure in log file. By default NULL.
log_file_prefix	A character of log file prefix to name log file. Log file is named as format of "log_file_prefix_XXXX_(current).csv" Default is "update_db_log".
log_dir	Path to save process log file. NULL means to use "./" as log dir. Default "./log".
...	Extra arguments to be passed to methods.

Details

By combining information from data_source and re-try log file , it process input_files of datasource in following steps:

1. Build input_files info from all input_files marked with process in datasource or some input_files with recording failure in retry log file;

2. Read input_files into original dataframe, process original dataframe by using process_fun specified by datasource, and save processed dataframe into files;

3. Save update log, etc.

Process log is saved in log dir, like "process_files_log_XXXX(current).csv"

Value

NULL invisibly. Raise error if anything goes wrong.

Functions

• process_files,gta_db-method: process importing files in a database of gta_db class

See Also

Other data management: clear_tables(), convert_import_file(), get_datasource(), import_table(), read_import_file(), ttm_financial_report(), update_db(), write_import_file()

chriszheng2016/zstmodelr documentation built on June 13, 2021, 8:59 p.m.