Description Usage Arguments Details Value Use cases See Also Examples
This function implements parametric bootstrapping for LNRE models, i.e. it draws a specified number of random samples from the population described by a given lnre
object. For each sample, two callback functions are applied to perform transformations and/or extract statistics. In an important application (bootstrapped confidence intervals for model parameters), the first callback estimates a new LNRE model and the second callback extracts the relevant parameters from this model. See ‘Use Cases’ and ‘Examples’ below for other use cases.
1 2 3 
model 
a trained LNRE model, i.e. an object belonging to a subclass of 
N 
a single positive integer, specifying the size N (i.e. token count) of the individual bootstrap samples 
ESTIMATOR 
a callback function, normally used for estimating LNRE models in the bootstrap procedure. It is called once for each bootstrap sample with the sample as first argument (in the form determined by 
STATISTIC 
a callback function, normally used to extract model parameters and other relevant statistics from the bootstrapped LNRE models. It is called once for each bootstrap sample, with the value returned by 
replicates 
a single positive integer, specifying the number of bootstrap samples to be generated 
sample 
the form in which each sample is passed to Alternatively, a callback function that will be invoked with arguments 
simplify 
if 
verbose 
if 
parallel 
whether to enable parallel processing. Either an integer specifying the number of worker processes to be forked, or a preinitialised snow cluster created with 
seed 
a single integer value used to initialize the RNG in order to generate reproducible results 
... 
any further arguments are passed through to the 
The parametric bootstrapping procedure works as follows:
replicates
random samples of N
tokens each are drawn from the population described by the LNRE model model
(possibly using a callback function provided in argument sample
)
Each sample is passed to the callback function ESTIMATOR
in the form determined by sample
(a frequency spectrum, typefrequency list, or factor vector of tokens). If ESTIMATOR
fails, it is rerun with a different sample, otherwise the return value is passed on to STATISTIC
. Use ESTIMATOR=identity
to pass the original sample through to STATISTIC
.
The callback function STATISTIC
is used to extract relevant information for each sample. If STATISTIC
fails, the procedure is repeated from step 2 with a different sample. The callback will typically return a vector of fixed length or a singlerow data frame, and the results for all bootstrap samples are combined into a matrix or data frame if simplify=TRUE
.
Warning: Keep in mind that sampling a token vector can be slow and consume large amounts of memory for very large N
(several million tokens). If possible, use sample="spc"
or sample="tfl"
, which can be generated more efficiently.
Parallelisation
Since bootstrapping is a computationally expensive procedure, it is usually desirable to use parallel processing. lnre.bootstrap
supports two types of parallelisation, based on the parallel package:
On Unix platforms, you can set parallel
to an integer number in order to fork the specified number of worker processes, utilising multiple cores on the same machine. The detectCores
function shows how many cores are available, but due to hyperthreading and memory contention, it is often better to set parallel
to a smaller value. Note that forking may be unstable especially in a GUI environment, as explained on the mcfork
manpage.
On all platforms, you can pass a preinitialised snow cluster in the argument
, which consists of worker processes on the same machine or on different machines. A suitable cluster can be created with makeCluster
; see the parallel package documentation for further information. It is your responsibility to set up the cluster so that all required data sets, packages and custom functions are available on the worker processes; lnre.bootstrap
will only ensure that the zipfR package itself is loaded.
Note that parallel processing is not enabled by default and will only be used if parallel
is set accordingly.
If simplify=FALSE
, a list of length replicates
containing the statistics obtained from each individual bootstrap sample. In addition, the following attributes are set:
N
= sample size of the bootstrap replicates
model
= the LNRE model from which samples were generated
errors
= number of samples for which either the ESTIMATOR
or the STATISTIC
callback produced an error
If simplify=TRUE
, the statistics are combined with rbind()
. This is performed unconditionally, so make sure that STATISTIC
returns a suitable value for all samples, typically vectors of the same length or singlerow data frames with the same columns.
The return value is usually a matrix or data frame with replicates
rows. No additional attributes are set.
The confint
method for LNRE models uses bootstrapping to estimate confidence intervals for the model parameters.
For this application, ESTIMATOR=lnre
reestimates the LNRE model from each bootstrap sample. Configuration options such as the model type, cost function, etc. are passed as additional arguments in ...
, and the sample must be provided in the form of a frequency spectrum. The return values are successfully estimated LNRE models.
STATISTIC
extracts the model parameters and other coefficients of interest (such as the population diversity S
) from each model and returns them as a named vector or singlerow data frame. The results are combined with simplify=TRUE
, then empirical confidence intervals are determined for each column.
For some of the more complex measures of productivity and lexical richness (see productivity.measures
), it is difficult to estimate the sampling distribution mathematically. In these cases, an empirical approximation can be obtained by parametric bootstrapping.
The most convenient approach is to set ESTIMATOR=productivity.measures
, so the desired measures can be passed as an additional argument measures=
to lnre.bootstrap
. The default sample="spc"
is appropriate for most measures and is efficient enough to carry out the procedure for multiple sample sizes.
Since the estimator already returns the required statistics for each sample in a suitable format, set STATISTIC=identity
and simplify=TRUE
.
Vocabulary growth curves can only be generated from token vectors, so set sample="tokens"
and keep N
reasonably small.
ESTIMATOR=vec2vgc
compiles vgc
objects for the samples. Pass steps
or stepsize
as desired and set m.max
if growth curves for V_1, V_2, … are desired.
Either use STATISTIC=identity
and simplify=FALSE
to return a list of vgc
objects, which can be plotted or processed further with sapply()
. This strategy is particulary useful if one or more V_m are desired in addition to V.
Or use STATISTIC=function (x) x$V
to extract ycoordinates for the growth curve and combine them into a matrix with simplify=TRUE
, so that prediction intervals can be computed directly. Note that the corresponding xcoordinates are not returned and have to be inferred from N
and stepsize
.
More complex populations and nonrandom samples can be simulated by providing a user callback function in the sample
argument. This callback is invoked with parameters model
and n
and has to return a sample of size n
in the format expected by ESTIMATOR
.
For simulating nonrandomness, the callback will typically use rlnre
to generate a random sample and then apply some transformation.
For simulating mixture distributions, it will typically generate multiple samples from different populations and merge them; the proportion of tokens from each population should be determined by a multinomial random variable. Individual populations might consist of LNRE models, or a finite number of “lexicalised” types. Note that only a single LNRE model will be passed to the callback; any other parameters have to be injected as bound variables in a local function definition.
lnre
for more information about LNRE models. The highlevel estimator function lnre
uses lnre.bootstrap
to collect data for approximate confidence intervals; lnre.productivity.measures
uses it to approximate the sampling distributions of productivity measures.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41  ## parametric bootstrapping from realistic LNRE model
model < lnre("zm", spc=ItaRi.spc) # has quite a good fit
## estimate distribution of V, V1, V2 for sample size N=1000
res < lnre.bootstrap(model, N=1000, replicates=200,
ESTIMATOR=identity,
STATISTIC=function (x) c(V=V(x), V1=Vm(x,1), V2=Vm(x,2)))
bootstrap.confint(res, method="normal")
## compare with theoretical expectations (EV/EVm = center, VV/VVm = spread^2)
lnre.spc(model, 1000, m.max=2, variances=TRUE)
## lnre.bootstrap() also captures and ignores occasional failures
res < lnre.bootstrap(model, N=1000, replicates=200,
ESTIMATOR=function (x) if (runif(1) < .2) stop() else x,
STATISTIC=function (x) c(V=V(x), V1=Vm(x,1), V2=Vm(x,2)))
## empirical confidence intervals for vocabulary growth curve
## (this may become expensive because tokenlevel samples have to be generated)
res < lnre.bootstrap(model, N=1000, replicates=200, sample="tokens",
ESTIMATOR=vec2vgc, stepsize=100, # extra args passed to ESTIMATOR
STATISTIC=V) # extract vocabulary sizes at equidistant N
bootstrap.confint(res, method="normal")
## parallel processing is highly recommended for expensive bootstrapping
library(parallel)
## adjust number of processes according to available cores on your machine
cl < makeCluster(2) # PSOCK cluster, should work on all platforms
res < lnre.bootstrap(model, N=1e4, replicates=200, sample="tokens",
ESTIMATOR=vec2vgc, stepsize=1000, STATISTIC=V,
parallel=cl) # use cluster for parallelisation
bootstrap.confint(res, method="normal")
stopCluster(cl)
## on MacOS / Linux, simpler forkbased parallelisation also works well
## Not run:
res < lnre.bootstrap(model, N=1e5, replicates=400, sample="tokens",
ESTIMATOR=vec2vgc, stepsize=1e4, STATISTIC=V,
parallel=8) # if you have enough cores ...
bootstrap.confint(res, method="normal")
## End(Not run)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.