FHDI_Driver: Main Driver of Fractional Hot Deck Imputation

In FHDI: Fractional Hot Deck and Fully Efficient Fractional Imputation

Description

Fully efficient fractional imputation (FEFI) or fractional hot deck imputation (FHDI) is implemented to fill in missing values in incomplete data. This package is partially supported by the NSF grant CSSI 1931380.

Usage

FHDI_Driver(daty, datr=NULL, datz=NULL, s_op_imputation="FEFI", 
 i_op_SIS=0, s_op_SIS="global", s_op_cellmake="knn", top_corr_var=100, 
 i_op_variance=1, M=5, k=5, w=NULL, id=NULL, 
 s_op_merge="fixed", categorical=NULL)

Arguments

daty	raw data matrix (nrow_y, ncol_y) containing missing values. Each row must have at least one observed value, and no completely missing (blank) rows are allowed.
datr	response indicator matrix with the same dimensions as daty. Each response is recorded with 0 for missing value and 1 for observed value. If NULL, automatically filled with 1 or 0 according to daty.
datz	imputation cell matrix. If daty is a set of continuous data, datz can be obtained using FHDI_CellMake.
s_op_imputation	"FEFI" for fully efficient fractional imputation or "FHDI" for fractional hot deck imputation.
i_op_SIS	(FHDI Version >= 1.4) the desired number of reduced variables after the sure independence screening per each missing pattern. Default = 0 means no variable reduction and uses all variables. Range must be <= ncol_y.
s_op_SIS	(FHDI Version >= 1.4) "intersection" for sure independence screening with an intersection of simple correlation, "union" for sure independence screening with a union of simple correlation, or default = "global" for sure independence screening with a global ranking of simple correlation.
s_op_cellmake	(FHDI Version >= 1.4) option for different methods of cell construction with deficient donors. "merging" for adopting cell collapsing and merging or default = "knn" for adopting the k-nearest neighbors in terms of the Euclidean distance.
top_corr_var	(FHDI Version >= 1.4) the number of top-ranking variables based on simple correlation with default 100.
i_op_variance	1: perform Jackknife variance estimation; 0: no variance estimation.
M	the number of donors for FHDI with default 5.
k	the number of total categories per variable. Default = 5. The maximum is 35 since 9 integers (1-9) and 26 alphabet letters (a-z) are used. When a scalar value is given, all variables will have the same number of categories, while when a vector is given, i.e., k(ncol_y), each variable may have a different number of categories.
w	sampling weight for each row of daty. Default = 1.0 if NULL. When a scalar value is given, all rows will have the same weight, while when a vector is given, i.e., w(nrow_y), each row may have a different sampling weight.
id	index for each row. Default = 1:nrow_y if NULL.
s_op_merge	option for random cell make. Default = "fixed" using the same seed number; "rand" using a purely random seed number.
categorical	(FHDI Version >1.3) index vector indicating non-collapsible categorical variables. Default = zero vector of size ncol_y. For instance, when categorical=c(1,0), the first variable (i.e., 1st column) is considered strictly to be non-collapsible and categorical, thus no automatic cell-collapse will take place while the second variable (i.e., 2nd column) is considered as a continuous or collapsible categorical variable.

Details

In the FEFI method, all possible donors are assigned to each missing unit with the FEFI fractional weights. In the FHDI method, M (>1) donors are selected with the probability proportional to the FEFI fractional weights. Thus, the imputed values have equal fractional weights in general.

The jackknife replicated weights are produced as the default output. The replicated weights are presented by the product of replicated sampling weights and replicated fractional weights. Thus, the replicated weights can be directly used to compute the variance estimate of the estimators. From version >= 1.4, the sure independence screening method (Fan and Lv 2008) has been incorporated to perform variable reduction for each missing pattern, which is useful for high dimensional (i.e., big-p) datasets. Besides, we provide an alternative method using k-nearest neighbors to speed up the convergence of cell construction with deficient donors, which is useful for big-p datasets.

Value

fimp.data	imputation results with fractional weights in the form of a matrix consisting of ID, donor id (FID), weight (WGT), fractional weight (FWGT), and fractionally imputed data.
simp.data	imputed data in the format of single imputation. The same shape as daty.
imp.mean	the mean estimates of each variable (first row) and the estimated standard error of each variable (second row). If input argument "i_op_variance=0" then this output is not produced.
rep.weight	replication fractional weights for variance estimation. If input argument "i_op_variance=0" then this output is not produced.
M	reprint of the number of donors M for FHDI defined by the user.
s_op_imputation	reprint of the option "s_op_imputation" initially defined by the user.
i_op_merge	reprint of the option "i_op_merge" initially defined by the user.
i_op_SIS	reprint of the option "i_op_SIS" initially defined by the user.
s_op_SIS	reprint of the option "s_op_SIS" initially defined by the user.
s_op_cellmake	reprint of the option "s_op_cellmake" initially defined by the user.
top_corr_var	reprint of the option "top_corr_var" initially defined by the user.

Author(s)

Dr. Cho, In Ho (maintainer) [e-mail] Dr. Kim, Jae Kwang [e-mail] Dr. Im, Jong Ho [e-mail] Yicheng Yang, Graduate Research Assistant

References

Im, J., Cho, I.H. and Kim, J.K. (2018). FHDI: An R Package for Fractional Hot-Deck Imputation. The R Journal. 10(1), pp. 140-154; Im, J., Kim, J.K. and Fuller, W.A. (2015). Two-phase sampling approach to fractional hot deck imputation, Proceeding of the Survey Research Methods Section, Americal Statistical Association, Seattle, WA.

Examples

### Toy Example ### 
# y : multi-variate vector
# r : indicator corresponding to missingness in y

set.seed(1345) 
n=100 
rho=0.5 
e1=rnorm(n,0,1) 
e2=rnorm(n,0,1) 
e3=rgamma(n,1,1) 
e4=rnorm(n,0,sd=sqrt(3/2))

y1=1+e1 
y2=2+rho*e1+sqrt(1-rho^2)*e2 
y3=y1+e3 
y4=-1+0.5*y3+e4

r1=rbinom(n,1,prob=0.6) 
r2=rbinom(n,1,prob=0.7) 
r3=rbinom(n,1,prob=0.8) 
r4=rbinom(n,1,prob=0.9)

y1[r1==0]=NA 
y2[r2==0]=NA 
y3[r3==0]=NA 
y4[r4==0]=NA

daty=cbind(y1,y2,y3,y4)

result_FEFI=FHDI_Driver(daty, s_op_imputation="FEFI", k=3)
result_FHDI=FHDI_Driver(daty, s_op_imputation="FHDI", M=5, k=3)
result_FHDI_merging=FHDI_Driver(daty, s_op_imputation="FHDI", s_op_cellmake="merging", M=5, k=3)
FEFI_SIS=FHDI_Driver(daty, i_op_SIS=2, s_op_SIS="intersection", k=3)

names(result_FEFI)
names(result_FHDI)
names(result_FHDI_merging)
names(FEFI_SIS)

FHDI documentation built on Oct. 23, 2020, 7:12 p.m.