inst/README-validation.md

In FPRgroup/fpemdata: fpemdata

FPEM validation

Usage

An FPEM data set is a folder containing a number of data files (e.g. CSV files). An FPEM data set can be validated using

python ./data_set_validator/data_set_validator.py path/to/data/set/folder path/to/schema/folder

This will ensure that

• The required files are in the data set
• Only the required files are in the data set
• Each CSV file in the data set adheres to its table schema

To test on the included data and schema run

python data_set_validator/data_set_validator.py unpd_2018_data_set/data_set/ table_schemas/

Prerequisites

This validator only works with Python3.

To restore the Python virtual environment associated with the Python app, run

pip install -r ./data_set_validator/requirements.txt

On some systems Python is called by writing python3 rather than just python. When the Python binary is called python3, the pip binary might be called pip3.

Using Docker

Alternatively, use the fpcounts/fpem:validator Docker container.

Start an interactive container session with the local data set mounted.

docker run --name=fpem_validator --volume  <full path to data set folder>:/data_set --interactive --tty fpcounts/fpem:validator /bin/sh

For example, if your data set is stored in C:\fpem\unpd2018 the command will be

docker run --name=fpem_validator --volume  C:\fpem\unpd2018:/data_set --interactive --tty fpcounts/fpem:validator /bin/sh

When the shell starts, run

/app/validate.sh

FPRgroup/fpemdata documentation built on April 15, 2020, 11:42 p.m.