calc.stError  R Documentation 
Calculate point estimates as well as standard errors of variables in surveys. Standard errors are estimated using bootstrap weights (see draw.bootstrap and recalib). In addition the standard error of an estimate can be calcualted using the survey data for 3 or more consecutive periods, which results in a reduction of the standard error.
calc.stError( dat, weights = attr(dat, "weights"), b.weights = attr(dat, "b.rep"), period = attr(dat, "period"), var, fun = weightedRatio, national = FALSE, group = NULL, fun.adjust.var = NULL, adjust.var = NULL, period.diff = NULL, period.mean = NULL, bias = FALSE, size.limit = 20, cv.limit = 10, p = NULL, add.arg = NULL )
dat 
either data.frame or data.table containing the survey data. Surveys can be a panel survey or rotating panel survey, but does not need to be. For rotating panel survey bootstrap weights can be created using draw.bootstrap and recalib. 
weights 
character specifying the name of the column in 
b.weights 
character vector specifying the names of the columns in

period 
character specifying the name of the column in 
var 
character vector containing variable names in 
fun 
function which will be applied on 
national 
boolean, if TRUE point estimates resulting from fun will be divided by the point estimate at the national level. 
group 
character vectors or list of character vectors containig
variables in 
fun.adjust.var 
can be either 
adjust.var 
can be either 
period.diff 
character vectors, defining periods for which the
differences in the point estimate as well it's standard error is
calculated. Each entry must have the form of 
period.mean 
odd integer, defining the range of periods over which the sample mean of point estimates is additionally calcualted. 
bias 
boolean, if 
size.limit 
integer defining a lower bound on the number of
observations on 
cv.limit 
nonnegativ value defining a upper bound for the standard
error in relation to the point estimate. If this relation exceed

p 
numeric vector containing values between 0 and 1. Defines which
quantiles for the distribution of 
add.arg 
additional arguments which will be passed to fun. Can be either a named list or vector. The names of the object correspond to the function arguments and the values to column names in dat, see also examples. 
calc.stError
takes survey data (dat
) and returns point estimates
as well as their standard Errors defined by fun
and var
for each sample
period in dat
. dat
must be household data where household members
correspond to multiple rows with the same household identifier. The data
should at least contain the following columns:
Column indicating the sample period;
Column indicating the household ID;
Column containing the household sample weights;
Columns which contain the bootstrap weights (see output of recalib);
Columns listed in var
as well as in group
For each variable in var
as well as sample period the function fun
is
applied using the original as well as the bootstrap sample weights.
The point estimate is then selected as the result of fun
when using the
original sample weights and it's standard error is estimated with the result
of fun
using the bootstrap sample weights.
fun
can be any function which returns a double or integer and uses sample
weights as it's second argument. The predifined options are weightedRatio
and weightedSum
.
For the option weightedRatio
a weighted ratio (in \
calculated for var
equal to 1, e.g
sum(weight[var==1])/sum(weight[!is.na(var)])*100
.
Additionally using the option national=TRUE
the weighted ratio (in \
divided by the weighted ratio at the national level for each period
.
If group
is not NULL
but a vector of variables from dat
then fun
is
applied on each subset of dat
defined by all combinations of values in
group
.
For instance if group = "sex"
with "sex" having the values "Male" and
"Female" in dat
the point estimate and standard error is calculated on the
subsets of dat
with only "Male" or "Female" value for "sex". This is done
for each value of period
. For variables in group
which have NA
s in
dat
the rows containing the missings will be discarded.
When group
is a list of character vectors, subsets of dat
and the
following estimation of the point estimate, including the estimate for the
standard error, are calculated for each list entry.
The optional parameters fun.adjust.var
and adjust.var
can be used if the
values in var
are dependent on the weights
. As is for instance the case
for the poverty thershhold calculated from EUSILC.
In such a case an additional function can be supplied using fun.adjust.var
as well as its first argument adjust.var
, which needs to be part of the
data set dat
. Then, before applying fun
on variable var
for all period
and groups, the function fun.adjust.var
is applied to
adjust.var
using each of the bootstrap weights seperately (NOTE: weight is
used as the second argument of fun.adjust.var
).
Thus creating i=1,...,length(b.weights)
additional variables.
For applying fun
on var
the estimates for the bootstrap replicate will
now use each of the corresponding new additional variables. So instead of
fun(var,weights,...),fun(var,b.weights[1],...), fun(var,b.weights[2],...),...
the function fun
will be applied in the way
fun(var,weights,...),fun(var.1,b.weights[1],...),fun(var.2, b.weights[2],...),...
where var.1
, var.2
, ...
correspond to the estimates resulting from
fun.adjust.var
and adjust.var
.
NOTE: This procedure is especially usefull if the var
is dependent on
weights
and fun
is applied on subgroups of the data set. Then it is not
possible to capture this procedure with fun
and var
, see examples for a
more hands on explanation.
When defining period.diff
the difference of point estimates between periods
as well their standard errors are calculated.
The entries in period.diff
must have the form of "period1  period2"
which means that the results of the point estimates for period2
will be
substracted from the results of the point estimates for period1
.
Specifying period.mean
leads to an improvement in standard error by
averaging the results for the point estimates, using the bootstrap weights,
over period.mean
periods.
Setting, for instance, period.mean = 3
the results in averaging these
results over each consecutive set of 3 periods.
Estimating the standard error over these averages gives an improved estimate
of the standard error for the central period, which was used for
averaging.
The averaging of the results is also applied in differences of point
estimates. For instance defining period.diff = "20152009"
and
period.mean = 3
the differences in point estimates of 2015 and 2009, 2016 and 2010 as well as
2014 and 2008 are calcualated and finally the average over these 3
differences is calculated.
The periods set in period.diff
are always used as the middle periods around
which the mean over period.mean
years is build.
Setting bias
to TRUE
returns the calculation of a mean over the results
from the bootstrap replicates. In the output the corresponding columns is
labeled _mean at the end.
If fun
needs more arguments they can be supplied in add.arg
. This can
either be a named list or vector.
The parameter size.limit
indicates a lower bound of the sample size for
subsets in dat
created by group
. If the sample size of a subset falls
below size.limit
a warning will be displayed.
In addition all subsets for which this is the case can be selected from the
output of calc.stError
with $smallGroups
.
With the parameter cv.limit
one can set an upper bound on the coefficient
of variantion. Estimates which exceed this bound are flagged with TRUE
and
are available in the function output with $cvHigh
.
cv.limit
must be a positive integer and is treated internally as \
for cv.limit=1
the estimate will be flagged if the coefficient of
variantion exceeds 1\
When specifying period.mean
, the decrease in standard error for choosing
this method is internally calcualted and a rough estimate for an implied
increase in sample size is available in the output with $stEDecrease
.
The rough estimate for the increase in sample size uses the fact that for a
sample of size n the sample estimate for the standard error of most
point estimates converges with a factor 1/√{n} against the true
standard error σ.
Returns a list containing:
Estimates
: data.table containing period differences and/or k period
averages for estimates of
fun
applied to var
as well as the corresponding standard errors, which
are calculated using the bootstrap weights. In addition the sample size,
n
, and poplutaion size for each group is added to the output.
smallGroups
: data.table containing groups for which the number of
observation falls below size.limit
.
cvHigh
: data.table containing a boolean variable which indicates for each
estimate if the estimated standard error exceeds cv.limit
.
stEDecrease
: data.table indicating for each estimate the theoretical
increase in sample size which is gained when averaging over k periods. Only
returned if period.mean
is not NULL
.
Johannes Gussenbauer, Alexander Kowarik, Statistics Austria
draw.bootstrap
recalib
# Import data and calibrate set.seed(1234) eusilc < demo.eusilc(n = 4,prettyNames = TRUE) dat_boot < draw.bootstrap(eusilc, REP = 3, hid = "hid", weights = "pWeight", strata = "region", period = "year") dat_boot_calib < recalib(dat_boot, conP.var = "gender", conH.var = "region") # estimate weightedRatio for povertyRisk per period err.est < calc.stError(dat_boot_calib, var = "povertyRisk", fun = weightedRatio) err.est$Estimates # calculate weightedRatio for povertyRisk and fraction of oneperson # households per period dat_boot_calib[, onePerson := .N == 1, by = .(year, hid)] err.est < calc.stError(dat_boot_calib, var = c("povertyRisk", "onePerson"), fun = weightedRatio) err.est$Estimates ## Not run: # estimate weightedRatio for povertyRisk per period and gender and # period x region x gender group < list("gender", c("gender", "region")) err.est < calc.stError(dat_boot_calib, var = "povertyRisk", fun = weightedRatio, group = group) err.est$Estimates # use average over 3 periods for standard error estimation # and calculate estimate for difference of # period 2011 and 2012 inclulding standard errors period.diff < c("20122011") err.est < calc.stError( dat_boot_calib, var = "povertyRisk", fun = weightedRatio, period.diff = period.diff, # < take difference of periods 2012 and 2011 period.mean = 3) # < average over 3 periods err.est$Estimates ## End(Not run) # for more examples see https://statistikat.github.io/surveysd/articles/error_estimation.html
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.