In SWS-Methodology/faoswsTrade: FAO Trade Module

knitr::opts_chunk$set(echo = TRUE)
suppressPackageStartupMessages(library(data.table))
library(knitr)
library(stringr)
library(magrittr)
library(scales)
library(tidyr, warn.conflicts = FALSE)
library(futile.logger)
suppressPackageStartupMessages(library(dplyr, warn.conflicts = FALSE))
library(faosws)
library(faoswsUtil)
library(faoswsTrade)
library(faoswsFlag)
library(pander)
# files = dir("C:/Users/caetano/Documents/Github/faoswsTrade/R", full.names = TRUE)
# sapply(files, source)

About this document

This vignette is a detailed guide of the various data sets created for the Pre-Processing Report of the Trade Module. In order to have access to those data sets the user must be logged in SWS - Q.A.

1 SWS - Datatables

The pre-processing reports is available on SWS in Datatables under the domain trade-reports as shown in the figure below.

SWS

2 Tables

2.1 Table 1: Country list by year of which reporters are contained in the file

This table allows you to quickly identify data gaps and facilitate comparisons. It contains the following columns:

rep_code: country code of the reporter
rep_name: country name of the reporter
years: years on which the data availability is assessed

If the table contains a 1 it means that there is information for the country in that particular year. If the country did not report for that year, the correspondig cell will be blank. Its name on SWS is Reporters by years.

# ts_all_reports(
#   collection_path = "//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/for_reports", 
#   prefix = "complete_tf_cpc", complete = FALSE
#   )
# \\hqlprsws1.hq.un.fao.org\sws_r_share\mongeau\pre_processing_report\ts_reports
table1 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_1_ts_reporters.csv")
# kable(head(table1), format = "markdown", padding = 0)
set.caption("Country list by year of which reporters are contained in the file")
pander(rbind(head(table1), tail(table1)), split.table = 120, style = 'rmarkdown')

2.2 Table 2: List of non-reporting countries

It contains the complementary list of countries not reporting for those years by flow.

The variables in the table are:

non_rep_area: country code
non_rep_name: country name
flow: the flow, where 1 means imports and 2 exports
name: country name
years: years on which the data availability is assessed

Please note that in creating this table re-imports and re-exports are treated as imports and exports, respectively.

In the table the meaning of the cells is:

9: the country did not report both imports and exports
1: the country did not report imports
2: the country did not report exports
blank cell: the country reported all the flows

Only countries that for which at least in one year did not report at least one flow are reported. Its name on SWS is Non-reporting countries.

table2 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_2_ts_non_reporters.csv")
set.caption("List of non-reporting countries")
pander(rbind(head(table2), tail(table2)), split.table = 120, style = 'rmarkdown')
# kable(head(table2), format = "markdown", padding = 0)

2.3 Table 3: Number of records by reporter

It contains the number of records by flow and year per country. It also has the percentage difference from the current year to the previous one and a variable (hs_diff) saying if the hslenght (the maximum length of the HS code used a specific year) has changed or not comparing to the previous year (in which case it is TRUE).

Note: the records_diff variable is reported as $variable_t/variable_{t-1} - 1$ (so, e.g., 0.01 means 1%).

Its name on SWS is Number records by reporter/year.

table3 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_3_ts_preproc_long_format.csv")
# kable(head(table3), format = "markdown", padding = 0)
set.caption("Number of records by reporter")
pander(rbind(head(table3), tail(table3)), split.table = 120, style = 'rmarkdown')

\newpage

2.4 Table 4: Import/export content check

For every reporter for the year in question, a check that the file contains both import data and export data (as well as re-exports and re-imports). If the cell contains a 1, the country reported all the flows. If the country did not report for that year, the corresponding cell will be blank. Its name on SWS is Import and export content check.

table4 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_4_ts_check_content.csv")
set.caption("Import/export content check")
pander(rbind(head(table4), tail(table4)), split.table = 120, style = 'rmarkdown')

\newpage

2.5 Table 5: Check that quantities and values are included, by country/flow/year

For every reporter for the year in question, a check if that file contains both quantities and values. A 0 indicates that no information lacks, while 1 indicates that the information is missing. Its name on SWS is Check qty and value included.

table5 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_5_ts_flows_check.csv")
set.caption("Quantities and values are included, by country/flow/year")
pander(rbind(head(table5), tail(table5)), split.table = 120, style = 'rmarkdown')

2.6 Table 6: Report of records with missing quantities, for each year

For every reporter for the year in question, a check if that file contains both quantities and values for export, import, reexport, reimport.

Note: missing quantity means that both the quantity variable and the weight variable are not reported. This choice is due to the fact that generally the weight variable can be obtained from the quantity variable by using standard conversion factors, so that if the weight variable is missing it does not mean that no information on its weight can be obtained.

The table contains the following variables:

rep_code: country code
rep_name: country name
year: year
noqty_export: number of exports with missing quantities
noqty_import: number of imports with missing quantities
noqty_reexport: number of re-exports with missing quantities
noqty_reimport: number of re-imports with missing quantities
noqty_prop_export: proportion of exports with missing quantities
noqty_prop_import: proportion of imports with missing quantities
noqty_prop_reexport: proportion of re-exports with missing quantities
noqty_prop_reimport: proportion of re-imports with missing quantities

Note: the noqty_prop_* variables are reported as shares (so, e.g., 0.01 means 1%). Its name on SWS is Missing data by report.

table6 = fread("//hqlprsws1.hq.un.fao.org/sws_r_share/mongeau/pre_processing_report/ts_reports/Table_6_ts_missing_records.csv")
set.caption("Report of records with missing quantities, for each year")
pander(rbind(head(table6), tail(table6)), split.table = 120, style = 'rmarkdown')