data_assurance_process: Data Assurance Process

In austl001/MOSLR: MOSL Reports and Utility Functions

Data Assurance Process

Description

GET TEMPLATE PATHS FROM ONEDRIVE Get possible synced OneDrive locations for templates in Data Assurance folder Check folder exists, if not show warning and skip Check if relevant files in folder, if not show message and skip Check if CSV

Usage

data_assurance_process(
  onedrive.folder = "Data Assurance/New",
  conf.loc = NULL,
  config.name = "data-assurance",
  upload.blob = TRUE,
  clean.sharepoint = TRUE,
  upload.log = TRUE,
  ...
)

Arguments

onedrive.folder	Folder or directory within Sharepoint Org site that should contain the new templates to be processed
conf.loc	The path to a configuration file containing Azure storage connection information and keys
upload.blob	Logical specifying whether to upload the new, processed and rejected templates to blob storage
clean.sharepoint	Logical specifying whether to clean-up Sharepoint by moving the processed file
upload.log	Logical specifying whether to upload the log file to blob storage
...	Optional arguments for underlying functions

Details

UPLOAD TEMPLATES TO BLOB NEW Use config file to store container endpoint and SAS key If relevant files present: Clean columns to upper_camel Upload to blob container under New, saving file as "ORG-ID_FileCreatedDate_OriginalFileName"

VERIFY TEMPLATES ARE VALID AND VALIDATE COLUMNS Check columns in file match expected columns for either Premises or Meters Verify columns are valid: Measure Action Explanation Create Status columns based on validations: ValidationStatus: Pass / Fail Reason: NA / Validation failure reasons

UPLOAD PROCESSED FILES TO BLOB PROCESSED OR REJECTED Include relevant Blob metadata: ORG-ID, FileCreatedDate, OriginalFileName Copy file to OneDrive Save file as "ORG-ID_FileCreatedDate_OriginalFileName_MOSL-Reviewed_ReviewDate.csv" For valid rows: Copy to Blob Processed Save file as "ORG-ID_FileCreatedDate_OriginalFileName.csv" For invalid rows: Copy to Blob Rejected Save file as "ORG-ID_FileCreatedDate_OriginalFileName.csv"

austl001/MOSLR documentation built on Aug. 17, 2022, 12:07 a.m.