ratetest.calibrate: Calibrating 'ratetest' for ABC
In olli0601/abc.star: abc.star

Description Usage Arguments Value Note References See Also Examples

Calibrate the one-sample equivalence test for testing if simulated and observed summary values occur at similar rates. This test is applicable when the observed and simulated summary values are Exponentially distributed, or when Exponentiality cannot be rejected. The testing problem is described in terms of the scale parameter of the Exponential distribution, the reciprocal of the rate parameter.

Different types of calibrations are available, see Notes for details:

(what=ALPHA) compute the ABC false positive rate for given critical region,
(what=CR) calibrate the critical region for given ABC false positive rate,
(what=MXPW) calibrate the critical region and the equivalence region for given ABC false positive rate and maximum power,
(what=KL) calibrate the critical region, the equivalence region and the number of simulated summary values for given ABC false positive rate, maximum power and sample standard deviation of the observed data.

The calibration KL should be used. Typically, the KL calibration requires multiple i. i. d. instances of observed summary statistics at each ABC iteration. Here, the maximum likelihood estimate of the scale parameter is the sample mean, and can be computed from a single summary value. However, based on a single summary value, it is not possible to determine if the Exponential distribution is appropriate. It is recommended to use this test with multiple summary values, and to evaluate on the fly if the Exponential distribution is appropriate.

Depending on the type of calibration, some of the following inputs must be specified (see Examples).

ratetest.calibrate(n.of.x = NA, mean.x = NA, n.of.y = NA, what = "MXPW",
  mx.pw = 0.9, alpha = 0.01, tau.l = NA, tau.u = NA, tau.u.ub = NA,
  c.l = NA, c.u = NA, max.it = 100, tol = 1e-05, pow_scale = 1.5,
  debug = FALSE, plot = FALSE, verbose = FALSE)

`n.of.x`	Number of observed summary values
`mean.x`	Mean of observed summary values
`n.of.y`	Number of simulated summary values
`what`	Character string to indicate the type of calibration to be performed
`mx.pw`	Maximum power at the point of equality
`alpha`	Level of the equivalence test
`tau.l`	Lower boundary point of the equivalence region
`tau.u`	Upper boundary point of the equivalence region
`tau.u.ub`	Guess on the upper boundary point of the equivalence region
`c.l`	Lower boundary point of the critical region
`c.u`	Upper boundary point of the critical region
`max.it`	Maximum number of optimization steps at each calibration hierarchy
`tol`	Required error tolerance in calibrating the actual maximum power to the requested maximum power
`pow_scale`	Scale for the support of the standardized power. The power is truncated to `[tau.l/pow_scale,tau.u*pow_scale]` and then standardized
`debug`	Flag to switch off C implementation
`plot`	Flag to plot calibrations
`verbose`	Flag to run in verbose mode
`plot_debug`	Flag to plot at each calibration iteration

vector

(what=ALPHA) This calibration requires the inputs n.of.y, c.l, c.u, tau.l, tau.u with c.l>tau.l, c.u<tau.u, tau.u>1, tau.l<1. The output contains the corresponding ABC false positive rate alpha. This option does not specify any of the free ABC parameters, but may be useful to determine the ABC false positive rate for uncalibrated ABC routines.
(what=CR) This calibration requires the inputs tau.l, tau.u, alpha with tau.l<1, tau.u>1 and default alpha=0.01. The output contains the corresponding critical region [c.l, c.u], which corresponds to the ABC tolerance region typically denoted by [-epsilon, epsilon]. This is an intermediate calibration step and may result in unsuitable power properties (see Examples).
(what=MXPW) This calibration requires the inputs alpha and mx.pw, with default values 0.01 and 0.9 respectively. The output contains the corresponding critical region [c.l, c.u] (to be used in ABC, see Notes on (2)), and the corresponding equivalence region [tau.l, tau.u] that gives a suitable ABC accept/reject probability if the simulated summary values are close to the observed summary values. As a check to the numerical calibrations, the actual power at the point of equality is returned (pw.cmx).
(what=KL) This calibration can be used when a set of observed summary values is available. It is desirable because it specifies the number of simulated summary values so that the power is very close to the desired summary likelihood in terms of the KL divergence. The inputs are alpha (default is 0.01), mx.pw (default is 0.9), n.of.x, mean.x, n.of.y. The output consists of the corresponding critical region [c.l, c.u] (to be used in ABC, see Notes on (2)), the equivalence region [tau.l, tau.u], and the number of simulated summary values needed (n.of.y). As a check to the numerical calibrations, the KL divergence is returned (KL). It is desirable to compare the power to the summary likelihood in terms of the KL divergence, see References.

Note that the underlying test statistic only depends on n.of.x, n.of.y, mean.x, which are all known before ABC is run. Consequently, the free ABC parameters are calibrated once, before ABC is started.

The lower boundary point of the critical region c.l is calibrated numerically, so that the power is maximized at the point of equality rho=1. The calibrated c.l does not equal 1/c.u.

http://arxiv.org/abs/1305.4283

mutost.calibrate, ztest.calibrate, vartest.calibrate

# Example 1: calculate ABC false positive rate (alpha) for given ABC tolerance
# this requires to specify c.l, c.u, tau.l, tau.u (ad-hoc ABC parameters), n.of.y, what='ALPHA'
# note: can be useful to compute the ABC false positive rate for uncalibrated ABC routines

ratetest.calibrate(n.of.y=40, c.l=0.8, c.u=1.2, tau.l=0.5, tau.u=1.5, what='ALPHA', plot=TRUE)
ratetest.calibrate(n.of.y=40, c.l=0.8, c.u=1.2, tau.l=0.8, tau.u=1.2, what='ALPHA', plot=TRUE)

# Example 2: calibrate critical region for given ABC false positive rate and equivalence region
# this requires to specify alpha (default is 0.01), tau.l, tau.u, n.of.y, what='CR'
# note: this is just an intermediate calibration and may result in unsuitable power properties

ratetest.calibrate(n.of.y=40, tau.l=1/1.4, tau.u=1.4, alpha=0.01, what='CR', plot=TRUE)


# Example 3: calibrate critical region and power of ABC accept/reject step (default method when not specifying 'what')
# this requires to specify alpha (default is 0.01), mx.pw (desired maximum power), n.of.y, tau.u.ub (for numerical optimization)
# note: this is the default calibration because it specifies all ABC parameters
# for sensible calibration parameters, and only requires minimal information on 
# the observed summary values. If a set of observed summary values is available,
# use the calibrations in Example 4.

ratetest.calibrate(n.of.y=40, mx.pw=0.9, tau.u.ub=1.4, what='MXPW', plot=TRUE)


# Example 4: calibrate critical region, power of ABC accept/reject step, and number of simulated data points
# this requires to specify alpha (default is 0.01), mx.pw (desired maximum power, default is 0.9), and n.of.x, mean.x
# and a starting value for n.of.y, which must be equal to n.of.x
# note: the advantage here is that the KL divergence is also minimised, but we also
# need to have a set of observed summary values

ratetest.calibrate(n.of.x=30, mean.x=2, n.of.y=30, what='KL', plot=TRUE)