README.md

In thomascrines/labourmarketreports: Fetch data from APIs and generate markdown reports

shsannualreport

:construction: :construction: :construction:

Package is in development and includes functions, data files, and scripts for testing. When functionality is finalised, the following functions can be deleted from the R folder:

• lm_data_country_stats.R
• lm_data_key_stats.R
• lm_data_nomis_example.R
• lm_data_ons_example.R
• lm_data_region_stats.R
• lm_data_time_stats.R

The examples folder can also be deleted when other functions are finalised.

TODO The process of making a function to pull and format a dataset for publication should be described in this readme.

NOMIS and ONS guides could be moved into separate files.

:construction: :construction: :construction:

labourmarketreports is an R package to produce markdown reports with data from the following web APIs:

• ONS
• NOMIS

Installation

You can install the development version of shsannualreport from GitHub with:

# install.packages("devtools")
devtools::install_github("thomascrines/labourmarketreports")

If you are working on SCOTS, or if the above does not work, you can install manually:

1. Go to the shsannualreport repository master branch on GitHub
2. Click Clone or download then Download ZIP
3. Save the file locally and unzip
4. Install with install.packages():

install.packages("C:/DownloadDirectory/labourmarketreports-master/labourmarketreports-master", repos = NULL,
                 type="source", lib = "C:/YourLibraryPath")

Library path can be seen by running .libPaths()

Usage - ONS API

If using the ONS API, so it’s useful to read through the documentation before starting to use the package. The main functions are:

• lm_ons_request_dataset - Requests a downloadable dataset be created by the ONS API, based on a dataset_id, a dataset_edition, and a dataset_version, as well as an optional dataset_dimension and dataset_dimension_option to filter on. (These can either be hard-coded or obtained using other functions described later.) A filter output ID (a unique reference to the newly created dataset) is returned by the function. This can then be passed to lm_ons_download_dataset.
• lm_ons_download_dataset - Downloads a dataset that has been created in the API using lm_ons_request_dataset, based on the filter_output_id that was returned by that function.

Examples

To return an entire dataset from the ONS API, without filtering:

filter_output_id <- lm_ons_request_dataset(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                                           dataset_id = "labour-market", 
                                           dataset_edition = "time-series", 
                                           dataset_version = "2")

unfiltered_dataset <- lm_ons_download_dataset(base_uri = "https://api.beta.ons.gov.uk/v1/",
                                              filter_output_id = filter_output_id)

To return a filtered dataset from the ONS API (in this case to only return the option “16+” for the dimension “agegroups”):

filter_output_id <- lm_ons_request_dataset(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                                           dataset_id = "labour-market", 
                                           dataset_edition = "time-series", 
                                           dataset_version = "2",
                                           dataset_dimension = "agegroups",
                                           dataset_dimension_option = "16+")

filtered_dataset <- lm_ons_download_dataset(base_uri = "https://api.beta.ons.gov.uk/v1/",
                                            filter_output_id = filter_output_id)

Helper functions

The above examples only work as the arguments passed in are already known. The package contains a number of helper functions to get the necessary options from the API. (The base URI used above is not taken directly from the API, but from the documentation.)

The three required arguments (dataset_id, dataset_edition, and dataset_version) can be obtained with the functions lm_ons_datasets, lm_ons_dataset_latest_version, and lm_ons_dataset_editions.

The two optional arguments used for filtering (dataset_dimension and dataset_dimension_option) can be obtained with the functions lm_ons_dataset_dimensions and lm_ons_dataset_dimension_options.

Helper function examples

lm_ons_datasets

To get a dataset ID, use lm_ons_datasets:

datasets <- lm_ons_datasets(base_uri = "https://api.beta.ons.gov.uk/v1/")

Any value in the id column can be used as a dataset_id in the other functions. The returned dataframe can be filtered to only return desired columns with the optional variables argument:

datasets <- lm_ons_datasets(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                            variables = c("id", "description", "title"))

lm_ons_dataset_latest_version

To get the latest available version of a dataset, use lm_ons_dataset_latest_version:

lm_ons_dataset_latest_version(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                              dataset_id = "labour-market")

lm_ons_dataset_editions

To get the available editions of a dataset, use lm_ons_editions:

lm_ons_dataset_editions(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                        dataset_id = "labour-market")

lm_ons_dataset_dimensions (optional, for filtering only)

To get the available dimensions (or column names) of a dataset, use lm_ons_dimensions:

lm_ons_dataset_dimensions(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                          dataset_id = "labour-market", 
                          dataset_edition = "time-series", 
                          dataset_version = "2")

lm_ons_dataset_dimension_options (optional, for filtering only)

To get the available options of a dimension, use lm_ons_dimension_options:

lm_ons_dataset_dimension_options(base_uri = "https://api.beta.ons.gov.uk/v1/", 
                          dataset_id = "labour-market", 
                          dataset_edition = "time-series", 
                          dataset_version = "2",
                          dataset_dimension = "agegroups")

Full example

The following is an example of using the helper functions to get metadata of a dataset, and using this metadata to download a filtered dataset:

base_uri <- "https://api.beta.ons.gov.uk/v1/"
datasets_variables <- c("id", "description", "title")

datasets <- lm_ons_datasets(base_uri = base_uri,
                            variables = datasets_variables)

dataset_id <- datasets$id[datasets$title == "UK Labour Market"]

dataset_edition <- lm_ons_dataset_editions(base_uri = base_uri,
                                           dataset_id = dataset_id)

dataset_version <- lm_ons_dataset_latest_version(base_uri = base_uri,
                                                 dataset_id = dataset_id)


dataset_dimensions <- lm_ons_dataset_dimensions(base_uri = base_uri,
                                                dataset_id = dataset_id,
                                                dataset_edition = dataset_edition,
                                                dataset_version = dataset_version)

example_dimension <- dataset_dimensions[3]

dataset_options <- lm_ons_dataset_dimension_options(base_uri = base_uri,
                                                    dataset_id = dataset_id,
                                                    dataset_edition = dataset_edition,
                                                    dataset_version = dataset_version,
                                                    dataset_dimension = example_dimension)

example_option <- dataset_options[1]

filter_output_id <- lm_ons_request_dataset(base_uri = base_uri,
                                           dataset_id = dataset_id,
                                           dataset_edition = dataset_edition,
                                           dataset_version = dataset_version,
                                           dataset_dimension = example_dimension,
                                           dataset_dimension_option = example_option)

filtered_dataset <- lm_ons_download_dataset(base_uri = base_uri,
                                            filter_output_id = filter_output_id)

Usage - NOMIS API

Again, it’s useful to read through the NOMIS documentation before starting to use the package. The main function is:

• lm_nomis_download_dataset - Downloads a dataset from the NOMIS API, based on a base_uri, and a dataset_id. Also accepts and optional filter_string and row_limit.

Examples

To return an entire dataset from the NOMIS API, without filtering:

  base_uri <- "http://www.nomisweb.co.uk/"
  dataset_id <- "NM_1_1"

  lm_nomis_download_dataset(base_uri, dataset_id)

To return 1000 rows of a filtered dataset from the NOMIS API (in this case to only return the option “2092957697” for the dimension “geography”):

  base_uri <- "http://www.nomisweb.co.uk/"
  dataset_id <- "NM_1_1"
  filter_string = "?geography=2092957697"
  row_limit = 10000

  dataset <- lm_nomis_download_dataset(base_uri, dataset_id, filter_string = filter_string, row_limit = row_limit)

Helper functions

As with the ONS examples, these examples only work if the arguments are already known. The package contains a number of helper functions to get the necessary options from the API. (The base URI used above is not taken directly from the API, but from the NOMIS documentation .)

The only required argument is dataset_id, which can be obtained with the function lm_nomis_datasets.

The filter_stringcan be created with values obtained with the functions lm_nomis_dataset_dimensions and lm_nomis_dataset_dimension_options.

Helper function examples

lm_nomis_datasets

To get a dataset ID, use lm_nomis_datasets:

datasets <- lm_nomis_datasets(base_uri = "http://www.nomisweb.co.uk/")

Returned columns can be specified with the optional variables argument:

datasets <- lm_nomis_datasets(base_uri = "http://www.nomisweb.co.uk/", 
                              variables = c("id", "uri", "description.value", "name.value", "components.dimension"))

Any value in the id column can be used as a dataset_id in the other functions.

The returned object from lm_nomis_datasets is required for the lm_nomis_dimensions function.

lm_nomis_dataset_dimensions (optional, for filtering only)

To get the available dimensions (or column names) of a dataset, use lm_nomis_dimensions:

datasets <- lm_nomis_datasets(base_uri = "http://www.nomisweb.co.uk/", 
                              variables = c("id", "uri", "description.value", "name.value", "components.dimension"))
dataset_id <- "NM_1_1"

dimensions <- lm_nomis_dataset_dimensions(dataset_id = dataset_id, 
                                          datasets_list = datasets)

lm_nomis_dataset_dimension_options (optional, for filtering only)

To get the available options of a dimension, use lm_nomis_dimension_options:

options <- lm_nomis_dataset_dimension_options(base_uri = "http://www.nomisweb.co.uk/", 
                                              dataset_id = "NM_1_1", 
                                              dataset_dimension = "GEOGRAPHY")

To filter a dataset currently involves manually creating a filter string to pass to lm_nomis_download_dataset. In the example above, using "geography" as the dimension and an available option of "2092957697", the filter string can be manually written: "?geography=2092957697". This is a possible issue to be improved in future if necessary.

thomascrines/labourmarketreports documentation built on Jan. 28, 2021, 9:47 a.m.