generate: Generate a report for a GCAM experiment

In JGCRI/gcamrpt: Convert GCAM results to the format required by various IAM experiment databases

Description

Read the user control files (see details below) and run the report generation. For each requested scenario the system will pull the queries needed to compute the variables requested by the user. The necessary calculations will be run, and the result will be written in the requested format.

Usage

generate(scenctl, varctl, dbloc, outputdir = getwd(), model = "GCAM",
  template = NULL, fileformat = getOption("gcamrpt.fileformat", "CSV"),
  scenmerge = getOption("gcamrpt.scenmerge", TRUE),
  dataformat = getOption("gcamrpt.dataformat", "tabs"),
  wideformat = getOption("gcamrpt.wideformat", TRUE))

Arguments

scenctl	Name of the scenario control file.
varctl	Name of the variable control file.
dbloc	Directory holding the GCAM databases
outputdir	Directory to write output to. Default is the current working directory.
model	Name of the model (e.g., 'GCAM'). This is required for the IIASA data format. It is ignored for all other formats.
template	Name of a csv file containing an IIASA template. This is only useful in the IIASA format. See complete_iiasa_template for instructions on how to use a template
fileformat	Desired format for output files.
scenmerge	Flag: if true, merge scenarios; otherwise, leave scenarios as separate tables.
dataformat	Indicates desired data format. Supported formats are 'tabs', 'merged', or 'IIASA'
wideformat	Flag: if true, convert data to wide format before output; otherwise, leave in long format.

Value

A list of the tables produced. Depending on the options chosen, this list could be nested up to two layers deep. Additionally, the report will be written to output files as described in the Output section. If file output was requested, then the results will be returned invisibly.

Control files

The report generator requires two control files. The first lists the scenarios to be run. It should be a CSV file with the following columns:

GCAM scenario

The name that GCAM used for the scenario.

output scenario

The name that will be used for the scenario in the final report.

scenario db

The name of the GCAM database the scenario was recorded in.

Each row in this table will cause a scenario to be generated in the report.

The second control file lists the variables that should be written into the report. It should have the following columns:

GCAM variable

The canonical GCAM name for the variable. The listVariables function lists the variables known to the report generation system.

output variable

The name that will be used for the variable in the output.

aggregation keys

A comma-separated list of columns to group by when aggregating the raw GCAM output. This column can be left blank if no aggregation is desired for this variable.

aggregation function

The function to use in the aggregation. Supported functions are sum, mean, max, min, and median. If none is specified, sum will be used.

years

List of years to include in the output. You can list individual years, ranges in the form start:end, or stepped ranges in the form start:end:step. Ranges are inclusive, so 2000:2010:5 is the same as 2000, 2005, 2010. If the year list is omitted, all years in the data will be included.

filters

Arbitrary filters to apply to the table, before aggregating. These should be in the modified s-expression format described below. If no filters are to be applied this column can be left blank.

output units

The desired output units for the variable. The report generation system will attempt to convert the units and will throw an error if it fails. If output in GCAM native units is desired, this column can be left blank.

Output

The system has several options for formatting output. These can be passed as arugments to the system, or set as R options. The names of the options and their functions are:

gcamrpt.fileformat

File format for output. Options are "R", "rgcam", "CSV" and "XLSX". If "R", do not output a file at all. In all cases the tables produced are returned as a (possibly nested) list. If file output was produced, then the results are returned invisibly, so they won't print to the terminal unless you explicitly call print on them.

gcamrpt.scenmerge

If TRUE, for each variable merge the results for all scenarios into a single table (distinguished by the value of the scenario column). Otherwise, create a separate table for each combination of scenario and variable.

gcamrpt.dataformat

Specify the data format; that is, how the data is organized in the output files. Three options are available:

"tabs"

Each table generated goes into a separate tab (if XLS output is selected) or file (if CSV output is selected). The tab or file will be named with the output of the table.

"merged"

The tables will be output sequentially into a single tab or file. Each table will be preceded by its name. This is similar to the format used by GCAM to output batch queries.

"IIASA"

The database format used by IIASA. In this format each table is spread into a row in a merged table, with a column to identify the variable that each row comes from.

gcamrpt.wideformat

If TRUE, reshape the tables into wide format (years as columns) before output. Otherwise, leave them in long format. If the IIASA data format is selected, then this option is ignored, since the IIASA format requires wide data.

Output filenames will be chosen automatically. For an XLSX file the filename will be 'gcamrpt.xlsx'. For CSV output with tabs == FALSE the result will be 'gcamrpt.csv'. For CSV output with tabs == TRUE the output files will be named with the scenario (if scenario tables are not merged) and variable output names, for example 'Reference-GDP.csv'. If the specified output is "rgcam", the output will be an rgcam project data file named 'gcamrpt.dat'. In any of these cases, if a file already exists with the automatically chosen filename, the first unused number will be appended, e.g., gcamrpt001.xlsx, gcamrpt002.xlsx, etc.

The system produces a variety of diagnostic messages as it runs. These can be suppressed with suppressMessages. More serious problems will be indicated with warnings.

Filters

Filters are specified using three-element modified s-expressions. For example:

(notmatches; technology; CCS)

This would describe a filter that would select only those rows for which the technology column does not match the regular expression "CCS". Multiple filters can be applied by putting them in a comma-separated list.

(!=; sector; beef), (!=; sector; sheepgoat)

The s-expressions are "modified" by separating the elements with semicolons instead of whitespace. This allows us to have operands with internal whitespace. However, leading and trailing whitespace will always be trimmed.

The filter functions currently recognized by the system are

==

String equality

!=

String inequality

<

Numeric less-than

>

Numeric greater-than

<=

Numeric less-than-or-equals

>=

Numeric greater-than-or-equals

matches

Regular expression match. Note that because of the way we parse these strings you can't have a ',', ';', '(', or ')' in your regular expressions for this function or any of the ones below.

matchesi

Case-insensitive regular expression match.

notmatches

Regular expression inverted match. That is, select the rows that do not match the given regular expression.

notmatchesi

Case-insensitive regular expression inverted match.

JGCRI/gcamrpt documentation built on June 30, 2019, 8:41 a.m.