qleTest: Monte Carlo testing

Description Usage Arguments Details Value Author(s)

View source: R/qleTest.R


Monte Carlo hypothesis testing


qleTest(est, par0 = NULL, obs0 = NULL, ..., sim, criterion = NULL,
  nsim = 100, obs = NULL, alpha = 0.05, multi.start = 0L,
  na.rm = TRUE, cores = 1L, cl = NULL, iseed = NULL,
  verbose = FALSE)



object of class qle, the estimation results from function qle


optional, vector of parameter for the null hypothesis


optional, vector of observed statistics corresponding to 'par0'


arguments passed to the simulation function 'sim', searchMinimizer and multiSearch


user supplied simulation function, see qle


optional, NULL (default), name of the test statistic, either "qle" or "mahal" which overwrites the function criterion used for estimation of the model parameter


number of bootstrap replications of the statistical model to generate (simulated) statistics


optional, NULL (default), simulated statistics at the hypothesised parameter, if not given, these are generated at 'par0' or at 'est$par'


significance level for testing the hypothesis


integer, =0,1,2, level of multi start root finding (see details)


logical, TRUE (default), whether to remove 'NA' values from the matrix of re-estimated parameters


number of cores for multistart searches for each given/generated observation, only if multi.start>0 enabled and ignored otherwise


cluster object, NULL (default), of class MPIcluster, SOCKcluster, cluster


integer, the seed for initializing the cluster workers for parallel computations


logical, TRUE for intermediate output


The function tests the null hypothesis H_0:\,\hat{θ}=θ_0, that is, whether the statistical model w.r.t. to the estimated parameter explains the observed statistics, against the alternative H_1:\,\hat{θ}\neqθ_0 based on a Monte Carlo approach (see vignette). Due to the approximate nature of the assumed statistical model for the observed statistics the exact distribution of the test statistics, that is, the Mahalanobis distance or quasi-deviance, is generally unknown and therefore its asymptotic distribution might be an unrealistic assumption for the null hypothesis. For this reason, and in order to retrieve an empirical p-value for testing, we generate bootstrap observations and re-estimate the model parameter for each observation in the same way as done before when estimating the model parameter. This includes all possible types of variance approximations available (by kriging or average approximations) and types of prediction variances (by kriging or cross-validation).

The function expects an estimation result 'est' as returned from the main estimation function qle. If any simulated statistics are available at the final parameter estimate or at 'par0', then these can be passed by 'obs' and used as bootstrapped observations of the summary statistics. Otherwise the function first generates those using 'nsim' model replications. The criterion function approximations are used as specified in the object 'qsd' and will not be further augmented by additional samples or simulations during the test procedure. The value of the test statistic is either chosen as the current criterion function value at the estimated parameter or it is re-computed at the given parameter 'par0' using, if given, the 'real' observed statistics 'obs0'. The user can also select a different criterion function as a test statistic compared to the estimation before which can be set by 'criterion'. Apart from the quasi-deviance as a test statistic, in principle, any supported type of a least squares criterion, more generally, the Mahalanobis distance, can be used which only depends on the prefered type of variance matrix approximation, see covarTx.

In order to efficiently find the roots of the quasi-score vector we implement a multi start concept for minimizing the criterion function. If 'multi.start=0' no single root finding is initiated from the estimated parameter (as a starting point) for each newly generated observation. Using 'multi.start=1' starts a multi start root finding only in case the local optimization gets stuck into a local minimum or does not converge and setting 'multi.start=2' always triggers a multi start local search for each simulated observation. Practically, the re-estimations of the parameters might still fail to converge. However, the user can control the convergence conditions of the local solvers (including the quasi-scoring iteration) by the corresponding control parameters (see searchMinimizer). Any failed re-estimation is excluded from the test results and stored in the attribute 'info'. In addition, as part of the returned data frame 'param' the empirical standard error, predicted standard error (based on the average inverse quasi-information matrix), the root mean square error, the bias and sample mean value of the re-estimated parameters are also available. For a full example we refer the reader to the package vignette.


An object of class qleTest as a list of:


data frame of estimated parameters and error measures


the test result


name of the test

with attributes:


mean square error matrix of re-estimated parameters


average inverse quasi-information matrix over all re-estimated parameters


inverse quasi-information matrix at the parameter to be tested 'est$par'


relative difference of the empirial and predicted standard error of the parameter to be tested


list of simulated statistics either at the estimated parameter or at the optional parameter 'par0'


results from re-estimating the model parameters for each simulated observation 'obs'


average quasi-score, respectively, average gradient of the MD at the re-estimated parameters


name of criterion function used as a test statistic: "qle" or "mahal"


list of the following elements: indices of re-estimation results where the inversion of the quasi-information matrix failed, the re-estimated parameters have 'NA's, criterion function minimizations failed or did not converge numerically, the integer seed value 'iseed'


M. Baaske

qle documentation built on May 2, 2019, 9:55 a.m.