*.T2
) indicators from PEPFAR Target Setting Tool output.District + Metro
level.
Datasets are saved in extdata/demo-district28
.read_dp_art_dec31()
with new .DP file flags to ensure ART adjustment factor and ART patient reallocation counts are applied to number on ART extracted from Spectrum.anc_already_art
, anc_status
, anc_art_among_known
and anc_total_pos
indicators from ANC input time series data."pepfar_datapack_indicators_2025.csv"
.PMTCT_STAT*.T
and PMTCT_ART*.T
anc_already_art
from Spectrum PJNZ file (was errantly
reading number of women initiated ART <4 weeks before delivery).extdata/demo_mwi2024_v6.36.PJNZ
.
File created with Spectrum v6.36. Created new example Northern / Central / Southern region Spectrum files for testing and demonstration of subnational Spectrum PJNZ files.
data-raw/demo-subnational-pjnz/README.md
for more detailsUpdate vignettes with new datasets and default options.
Update Navigator checklist calendar quarter for 2025 estimates to 2024Q4 and 2025Q3 for current period and projection.
aggregate_art()
and aggregate_anc()
.spdep
v1.3.6 see https://r-spatial.github.io/spdep/news/index.html#version-13-6-development. We expect this warning for some countries and it will make tests and output noisy to leave on.vmmc_path
to Spectrum download function for path to DMMPT2 output Excel file.spdep v1.2-8
warnings from mat2listw()
(issue #421).pepfar_datapack_indicators_2024.csv
.December 2025
to December 2027
aggregate_art()
and aggregate_anc()
: Data aggregated retaining missing values at lowest admin level and summed totals at higher admin levels.prepare_input_time_series_art()
and prepare_input_time_series_anc()
: New column containing a list of area_ids corresponding to missing districts included in aggreagated totals. hintr_prepare_agyw_download
for creating AGYW tool.<NeedARTDec31 MV>
, and so it was fine to extract the total
value. Now child ART is recorded, and so need to sum the adult age groups only.aware_plhiv_attend
and unaware_plhiv_attend
for consistent facility attendance cascades.Add projection to T4 and T5 for PEPFAR COP projections. Only outputs plhiv, plhiv_attend, and infections, and, for T4, incidence.
Update indicators and age groups for PEPFAR Data Pack export to 2023 specification.
Change PEPFAR Data Pack file name to pepfar_datapack_indicators_2023.csv
.
Add higher area levels to PEPFAR Datim ID mapping table to enable alternate PSNU level selection.
Add advanced model option to select PEPFAR PSNU level to produce Data Pack output. Defaults set to current choices.
summary_report.Rmd
and comparison_report.Rmd
.output_aware_plhiv
= "No".art_spectrum_warning()
and anc_spectrum_warning()
where district totals in naomi were compared to national totals in spectrum.hintr_run_model
data can include a resource_url
to include a static link to the original resource.No user-visible changes, only internal.
extract_shiny90_age_sex()
to extract awareness of status estimates saved with Spectrum internal Shiny90 implementation (Spectrum v6.21).extdata/demo_mwi2019_v6.21-shiny90.PJNZ
anchor_home_district = TRUE
if option is not set.options$anchor_home_district = TRUE
`options$anchor_home_district = FALSE
aggregate_art()
to aggregate ART separately when provide at different admin levels. Previous behaviour was to aggregate from lowest level provided.hintr_run_model
and hintr_calibrate
as qs files for faster reading. See https://github.com/traversc/qs.quarter_id_to_calendar_quarter
conversion functions.get_metadata
into memory on first retrieval for quicker repeated access from other packages.Update for Spectrum v6.2 with calendar year projection period instead of mid-year projection.
eppasm >= 0.7.0
and first90 >= 1.6.1
which incorporate code updates
for calendar year projection.For Spectrum inputs parsed by function extract_pjnz_naomi()
, add a column
specifying to which calendar quarter the outputs correspond. This enables
Naomi model compatibility with either mid-year projection (Spectrum version <=6.19)
or calendar year projections (Spectrum version >=6.2).
Update read_dp_anc_testing()
for new ANC testing data input tag in .DP file "<ANCTestingValues MV4>"
.
generate_comparison_report()
that requires a naomi output file saved as an .Rds or output zip file created by hintr_run_model()
inputs_outputs
dataset in the naomi outputs including bar_plotly()
, age_bar_plotly()
and scatter_plotly
.art_adult_child_ratio
indicator. This is ratio of adults to children not paediatric sex ratio.get_calibration_option_labels()
, get_model_calibration_options()
and get_model_options_template()
. Model options specification has been refactored into naomi.options package.naomi_fit
object do not contain advanced options. Updates read_output_package()
function to read in full model options when available in the model output zipextract_pjnz_one()
.plhiv_attending
reporting the estimated denominator for art_current
based on assuming same ART attendance probabilities for untreated population as the treated population. This give estimates for the predicted number of PLHIV ‘in the ART catchment’ for a district and the number of untreated PLHIV in the catchment. This is an effort to address the recurring issue that the TX_CURR value that is provided to Data Pack does not correspond to the district PLHIV denominator, and to provide an output more directly useful for setting a district treatment targets.hintr_comparison_plot()
will error if comparison plot data does not exist.
This will avoid a class of obscure errors being shown to end user.Previously, new infections by age were calculated as:
(1 - exp(-lambda_t1 * duration)) * hiv_negative_t1
.
This implicitly assumed that incidence rate and HIV negative population size
were constant over the period from T1 to T2. This was reasonably okay for
short periods, but less acceptable when time from T1 to T2 becomes many years.
In settings with declining infections, this approach will tend to over-estimate
the number of PLHIV at T2.
New approach calculates infections in one year by age and sex based on lambda_t1
and then extrapolates to the number of infections between T1 and T2 based on
the ratio of the number of infections occurring in Spectrum between T1 and T2
to the number of infections in the year preceding T1 (Spectrum definition for
infections). This calculation also accounts for the number progressing from the
age group at infection to the age group at T2 based on Spectrum infections by
cohort (same as previous implementation).
The model accounts for impact of different ART coverage levels at T1 on district-level transmission rate and incidence, but does not account for differential change in ART coverage between T1 and T2. That is, a larger scale-up in ART in district A than district B between T1 and T2 would not result in a larger relative decline in incidence in A than B during this period.
The model does not explicitly account for mortality among infections between T1 and T2, but this is implicitly handled how new infections are subtracted away from the cohort survival probabilities for PLHIV at baseline.
Better handling of incidence changes over time and mortality probably need a more complete simulation model rather than the single time jump approximation.
spec_incid_t1
, etc. in the result
of naomi_model_frame()
) uses current year HIV negative population as
denominator instead of previous year HIV negative population as denominator
(used by Spectrum). This is for consistency with current time population
denominator used in internal incidence simulation.Infections are gradauted to quarters over periods rather than by cohort such that all infections remain within the same age as they are reported in single-age Spectrum outputs. Graduation is done via monotonic (Hyman) smoothing on cumulative infections.
Handle adjacency matrix construction for case with single area.
indicators.csv
and meta_*.csv
are in the language selected by the user at the time of generation.aggregate_anc()
and aggregate_art()
: select required columns when joining to prevent inadvertent column name clash due to extra columns (e.g. area_level
).select_naomi_data()
: Provide an annotated version of all input data provided with data tagged as raw_included
, raw_excluded
and interpolated_inlcuded
. Individual data frames for survey, ART and ANC inputs are added to naomi-data
in a list named full_data
.output_package()
: Create a long dataframe containing matched data inputs and model outputs with align_inputs_output()
. This adds a new object to the naomi output package that may used to compare model estimates and data inputs for T1 and T2.births_facility
in ANC data is missing it will be replaced by NA
when reading input so input time series aggregation passesUpdates for 2023 UNAIDS estimates (Dec 2022 - Mar 2023).
anc_known_neg
: Number of women who were not tested for HIV at antenatal visit because they had a recent documented HIV negative test.NA
in data input, it will be replaced by values 0
in reading input.anc_known_neg
is added to denominator for calcuating anc_prevalence
.births_facility
: The number of live births recorded at health facilities. This is added for triangulation with trend in number of ANC visits for data review purposes. Currently not used in modelling.eppasm_v0.6.2
.hintr_prepare_spectrum_download
now takes arg notes
for arbitrary notes which will be added into the output ziphintr_prepare_comparison_report_download
function to generate comparison summary reportoutputs
objectAdd option for district-sex-time interaction for ART coverage. This is implemented by argument naomi_model_frame(..., alpha_xst_term = TRUE, ...)
.
The default is currently alpha_xst_term = FALSE
for backwards compatibility. If set to TRUE
, the model also checks that sex-stratified ART data
are included at both T1 and T2.
Add term to PLHIV projection from T1 to T2 and T2 to T3 to account for district-level net migration. The term is the ratio of the change in the cohort population at district level divided by the national level cohort change. The national level cohort change from Spectrum accounts for mortality and international net migration.
Applying the ratio is controlled by the argument adjust_area_growth = TRUE
to naomi_model_frame()
.
This is currently set as FALSE
by default. Pending further testing in problematic cases.
Limitations of this approach: (1) It does not explicitly account for migration of PLHIV between districts. Therefore, it may 'create' or 'dissolve' infections if net population growth is greater in high prevalence districts (and vice versa). This is consistent with Spectrum and EPP handling of migration, but implications could be larger for smaller subnational areas. (2) In some cases, these net migration ratios mask unrealistic demographic assumptions in subnational population data (rather than true net migration patterns).
Internal changes
create_Lproj()
to avoid replicating same code for T1 -> T2 and T2 -> T3 projection.sf
to sp
object for spdep::poly2nb()
, which now supports sf
objects from v1.0.The model uses offsets from Spectrum to determine HIV prevalence in age groups outside the age range for which survey HIV prevalence are available. Stratify determination of offset age ranges by sex. This addresses situation for DHS surveys where maximum female age is 49 but maximum male age is 54 or 59.
Set default to model paediatric prevalence relative to adult female age 15-49 prevalence instead of using offsets relative to age 15-19 prevalence. This will (1) ensure same prevalence for male and female children, and (2) reduce distortions to age 15-19 prevalence arising from paediatric ART data.
Expose advanced option to allow paediatric to adult prevalence ratio to vary by district (option rho_paed_x_term
).
Throw validation error if time-varying ART attendance is selected, but ART data are not included at both Time 1 and Time 2.
area_id
in inst/datapack/datapack_psnu_area_id_map.csv
to new area hierarchy with separate Maputo provinces at admin-2.eppasm
v0.5.12 to avoid divide-by-zero in model computations (troubleshooting #2022-51).meta_indicator
table in the output package.anc_tested
and anc_tested_pos
to anc time series plot.Cal_Population
for population calibration in UNAIDS Navigator checklist file.= "sex_age_coarse"
for any indicators. Add this to check.area_sort_order
order.output_naomi_warning()
to report specific area/age/sex disaggregates that are triggering warnings in simulated outputs.readr::col_double()
with readr::col_number()
Adds output age group 10-19 and 25-49 years.
Update TZA area_id
in inst/datapack/datapack_psnu_area_id_map.csv
to new area hierarchy with Zone level added.
In construction of survivorship projection matrices, the method for interpolating annual population totals to quarter-year age group totals was revised to use Hyman monotonic interpolation of population counts for smoother population distributions.
Paediatric new infections (MTCT) are distributed to districts based on the ratio of paediatric incidence rate to adult 15-49 female HIV prevalence. This implicitly assumes the same mother-to-child transmission rate for all districts; results should be interpreted accordingly.
Spectrum inputs for number on ART at end of year (Dec 31) for age 0-14, male 15+, and female 15+ are read from the PJNZ and used for model calibration. As a result, if calibration to Spectrum outputs is applied, Q4 outputs for art_current
will match exactly to Spectrum. ART coverage will not be exactly the same because Spectrum uses mid-year PLHIV for the denominator and Naomi uses end-year PLHIV for the denominator.
Internal changes
naomi_objective_function_r()
] for stepping through model line by line.REPORT()
values match C++ code exactly.summary_report.Rmd
to avoid spherical geometry aggregation error (troubleshooting #2022-43).naomi_warning
helper to raise warnings which can be displayed in front endpermissive
option from model options, warning now returned instead of errorget_plot_type_label_and_description
with get_plot_type_column_metadata
and include formatting info in responseCY2021Q4
(December 2021) and short-term projection quarter to CY2022Q3
(September 2022).vl_tested_12mos
and vl_suppressed_12mos
to specify that indicators are reported for the previous 12 months even if reporting period is quarterly.prepare_input_time_series_art()
to accept data without art_new and vls indicators providedThrow an error from calibrate_outputs()
if user tries to calibrate an ouptut package that has
already been calibrated. This is determined by whether the output$spectrum_calibration
table
exists in the output object. In future, it would be nicer to allow user to re-calibrate an output
package. This will require saving additional information to un-calibrate and re-calibrate.
Save calibrated count outputs at Spectrum region level in the spectrum_calibration.csv
outputs.
Add function hintr_calibrate_plot
to return data for plotting calibrate barchart.
na.rm=
to output_package()
to allow calculation of quantiles if there are missing values in the simulation. Default is na.rm = FALSE
and na.rm = TRUE
is to be used for debugging purposes only. Cases where missing values occur will usually indicate very poor model fits and issues that need to be addressed.Updates to PEPFAR Data Pack outputs.
dataelement_uid
.=""<>""
around age group values.pepfar_datapack_indicators_2022.csv
.time_step
and add year
, quarter
and calendar_quarter
in input
time series function outputsparent_area_id
and area_sort_order
in input time series function outputsget_plot_type_label_and_description
to get label and description for each input time series plot types.prepare_input_time_series_art
and prepare_input_time_series_anc
for returning ART and ANC data formatted for plotting input time series graphs.hintr_prepare_spectrum_download
, hintr_prepare_coarse_age_group_download
and hintr_prepare_summary_report_download
for generating downloads from hintr_output
objecthintr_run_model
and hintr_calibrate
to save out minimal datawrite_datapack_csv()
when running hintr_run_model()
with zone-level demo data. This occurs because the demo data have ISO3 = MWI
for which the PSNU level is 3, but demo zone-level model fits are only applied to levels 0:2. A better solution would be to change the ISO3 for the demo data to something artificial, but this requires relaxing some validation in the user interface first.validate_calibration_options()
typo.option$calibrate_method
back into the summary report.pmax(..., 0)
in function log_lin_approx()
. (NAOMI TROUBLESHOOTING #100)artnum_mf()
, if exact ART quarter is found in the
data, filter those data instead of interpolating. This is more efficient and
avoids interpolation error if only a single quarter of data are uploaded.option$calibrate_method
from summary report (naomi troubleshooting #76).calibrate_method
model option in user interface via JSON metadata.calibrate_outputs()
]; handle uncertainty ranges for proportion adjustments.calibrate_outputs()
] such that estimates are
adjusted on logistic scale by fine district/sex/age group to ensure proportions do
not go above 100%.Implement survey data likelihood as flexible aggregation over areas / sex / age groups, similar to likelihood for number on ART and ANC testing.
Adjust flexibility of random effects specification based on granularity of available survey data when using aggregate survey data.
If only both sexes data are available, do not estimate district x sex interaction.
Add advanced model option use_survey_aggregate
to select use of aggregate uploaded
survey dataset.
Reparameterise random effects to be scaled to the linear predictor, as implemented by INLA parameterisation. That is now: mu = beta0 + u_i; u_i ~ N(0, sigma) instead of the previous parameterisation: mu = beta0 + u_i * sigma; u_i ~ N(0, 1).
This does not change the model at all, reduces the fitting time noticeably because the starting values for the linear predictor for the 'inner optimisation' step do not change resulting from steps in the hyperparameters.
Implementation for the BYM2 model follows the sparsity preserving conditional parameterisation described by Riebler et al. in Section 3.4.
art_current
for current estimates.art_current
at current estimates as indicator_code TX_CURR_SUBNAT.R
.Return number aware of status instead of proportion aware of status (DIAGNOSED_SUBNAT.T_1).
Update CIV Datim UID mapping for new 113 health districts.
anc_clients_year2
passed to model.To estimate the proportion aware of status at the district, this proportion is applied to the estimated untreated population in each district. This is added to the ART coverage to estimate the total proportion aware of status at district level by age and sex.
New output indicators for awarness of HIV status:
untreated_plhiv_num
: Number of untreated PLHIV (the 'treatment gap').aware_plhiv_prop
: Proportion of PLHIV aware of HIV positive status ('first 90' indicator).unaware_plhiv_num
: Number of PLHIV who are not aware of their HIV positive status.
Add model option output_aware_plhiv
in 'Advanced options'. If no .shiny90
file is present
in the PJNZ, this must be set to FALSE
. An error is raised by validate_model_options()
prompting this if no .shiny90
file is found inside the PJNZ.
demo_
and
affix _demo
to the end of the area IDs.quantile(..., names = FALSE)
in the funciton add_stats()
for computation
of posterior quantiles. This gives a decent speedup in producing the outptus package.This release revises the modelling of ANC prevalence and ART coverage and produces outputs for district-level ANC testing cascade.
Add ANC bias parameters to age-specific prevalence and ART coverage regression equations instead of to aggregate ANC prevalence and ART coverage.
ANC prevalence and ANC ART coverage outputs are produced by five-year age group, 15-24, 25-34, 35-49, and all ages.
New output indicators for ANC testing cascade:
anc_clients
: Number of ANC1 clients.anc_plhiv
: Number of HIV positive ANC attendees.anc_already_art
: Number of ANC clients already on ART prior to first ANC visit.anc_art_new
: Number of HIV positive ANC attendees initiating ART.anc_known_pos
: Number of HIV positive ANC attendees aware of HIV status prior to first ANC.anc_tested_pos
: Number of HIV positive ANC attendees tested HIV positive during ANC.anc_known_pos
: Number of HIV negative ANC attendees.Currently these indicators are calculated assuming that the number of 'known positive' ANC attendees are the same as the number already on ART.
Currently only a single time point is used for ANC clients estimates and projections. The future projection is closely linked to the accuracy of current year data; uncertainty around the estimate and projection are not appropriately quantified.
Age-specific fertility rate ratios (FRR) for HIV positive relative to HIV negative pregnant women and women already on ART relative to untreated HIV positive women are calculated from Spectrum model outputs (using EPP-ASM simulation) and included in regression equations for ANC prevalence and ANC ART coverage, respectively.
Aggregation of ANC testing data observations generalized using same approach as aggregation of number currently on ART observations. This means that:
Age-stratified ANC testing do not yet inform national or district level estimates for age-specific fertility or HIV/ART fertility rate ratios.
Additional changes:
Use Kish effective sample size in likelihood for household survey data.
Update summary report with additional styling, number formatting, and methods overview.
Streamline function extract_pjnz_naomi()
to reduce the number of calls to unzip and
read .DP file by getting all outputs from [eppasm::read_hivproj_output()
] which was
utilised; remove dependency on specio
.
area_name
field to example datasets, conforming to ADR schemas.naomi.utils
package.survey_mid_calendar_quarter
as survey run option.check_boundaries()
, compare_boundaries()
, and gather_areas()
.area_name
field to survey indicators dataset prepared by calc_survey_indicators()
."none"
calibration options in hintr_run_model()
if calibration options are not specified in the options list.Version 2.0 established for 2021 UNAIDS estimates. Changes are not guaranteed to be backward compatible with 2020 version of Naomi model.
YXXX_XXX
.age_group_label
from 00-04 to 0-4 and 05-09 to 5-9.age_group_id
, indicator_id
, and quarter_id
column from output package.ancrt_*
changed to anc_*
.current_art
changed to art_current
and art_new
column added.sum(weights) ^ 2 / sum(weights ^ 2)
)
to survey indicators dataset in field n_eff_kish
.prevalence
, art_coverage
).anc_prevalence
and anc_art_coverage
.art_num_attend
becomes art_current
.art_num_residents
becomes art_current_residents
.Internal changes:
inst/metadata/
rather than
scripted functions.meta_age_group.csv
accessed by get_age_groups()
.meta_indicator.csv
accessed by get_meta_indicator()
.datapack_indicator_mapping.csv
,
datapack_agegroup_mapping.csv
, datapack_sex_mapping.csv
.hintr_run_model()
for reproducing web app workflow.summary_report.Rmd
to remove age_group_id
.sample_tmb()
when redoing the sampling with different TMB
configuration (different computer/number of set threads) by recreating the
pointers if they are missing using obj$retape()
.get_metadata()
and get_plotting_metadata()
%*%
from Matrix
package.hintr_calibrate()
which takes output and calibration options and runs
calibration and returns in hintr output formatget_calibration_options()
to retrieve options for calibrationsample_tmb()
for R 4.0 changes.create_adj_matrix()
to create an adjacency matrix from a shape file.scale_gmrf_precision()
to scale precision matrix such that geometric
mean of marginal variance is one. This mirrors the behaviour of INLA::inla.scale.model()
.fit_inla()
is
called and the package is not found.rdhs::download_boundaries()
in script data-raw/survey.R
. (No change to model or datasets.)subset_output_package()
to subset and re-save a zipped Naomi output package.naomi_run_model
either as list of file paths or list of lists each containing a path, hash and filenamerho_paed_15to49f_ratio
to calculate paediatric prevalence as ratio of female 15-49 adult prevalence.rho_paed_x_term
specified by argument naomi_model_frame(..., rho_paed_x_term = TRUE, ...)
. Defaualt value is FALSE
.Formatting updates requested for Data Pack 2019:
survey_year
with survey_mid_calendar_quarter
in the survey indicators dataset.restype
for urban/rural to survey indicators datasets and option to stratify survey indicators calculation by urban/rural.artattend_t2
can be set to TRUE if there is evidence that patterns of cross-district ART attendance changed for the period before the survey and the inter-survey period.export_datapack()
to export an output package to datapack CSV format.read_*()
functions.This is for development and comparison purposes, not for production use.
fit/spectrum_calibration.csv
in the ouput zip download.Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.