Description Usage Arguments Details Value Note Author(s) References Examples
View source: R/simClustDesign.R
Generating data sets via a factorial design, which has factors: degree of separation, number of clusters, number of nonnoisy variables, number of noisy variables. The separation between any cluster and its nearest neighboring clusters can be set to a specified value. The covariance matrices of clusters can have arbitrary diameters, shapes and orientations.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30  simClustDesign(numClust = c(3,6,9),
sepVal = c(0.01, 0.21, 0.342),
sepLabels = c("L", "M", "H"),
numNonNoisy = c(4,8,20),
numNoisy = NULL,
numOutlier = 0,
numReplicate = 3,
fileName = "test",
clustszind = 2,
clustSizeEq = 50,
rangeN = c(50,200),
clustSizes = NULL,
covMethod = c("eigen", "onion", "cvine", "unifcorrmat"),
eigenvalue = NULL,
rangeVar = c(1, 10),
lambdaLow = 1,
ratioLambda = 10,
alphad = 1,
eta = 1,
rotateind = TRUE,
iniProjDirMethod = c("SL", "naive"),
projDirMethod = c("newton", "fixedpoint"),
alpha = 0.05,
ITMAX = 20,
eps = 1.0e10,
quiet = TRUE,
outputDatFlag = TRUE,
outputLogFlag = TRUE,
outputEmpirical = TRUE,
outputInfo = TRUE)

numClust 
Vector of the number of clusters for data sets in the design. 
sepVal 
Vector of desired values of the separation index between clusters
and their nearest neighboring clusters. Each element of 
sepLabels 
Labels for "close", "separated", and "wellseparated" cluster structures. By default, "L" (low) means "close", "M" (medium) means "separated", "H" (high) means "wellseparated". 
numNonNoisy 
Vector of the number of nonnoisy variables. 
numNoisy 
Vectors of the number of noisy variables. The default value of 
numOutlier 
The number or ratio of outliers. If 
numReplicate 
Number of data sets to be generated for the same cluster structure specified
by the other arguments of the function 
fileName 
The first part of the names of data files that record the generated data sets
and associated information, such as cluster membership of data points, labels
of noisy variables, separation index matrix, projection directions, etc.
(see details). The default value of 
clustszind 
Cluster size indicator.

clustSizeEq 
Cluster size.
If the argument 
rangeN 
The range of cluster sizes.
If 
clustSizes 
The sizes of clusters.
If 
covMethod 
Method to generate covariance matrices for clusters (see details). The default method is 'eigen' so that the user can directly specify the range of the diameters of clusters. 
eigenvalue 
numeric. userspecified eigenvalues when 
rangeVar 
Range for variances of a covariance matrix (see details). The default range is [1, 10] which can generate reasonable variability of variances. 
lambdaLow 
Lower bound of the eigenvalues of cluster covariance matrices.
If the argument 
ratioLambda 
The ratio of the upper bound of the eigenvalues to the lower bound of the
eigenvalues of cluster covariance matrices.
If the argument 
alphad 
parameter for unifcorrmat method to generate random correlation matrix

eta 
parameter for “cvine” and “onion” methods to generate random correlation matrix

rotateind 
Rotation indicator.

iniProjDirMethod 
Indicating the method to get initial projection direction when calculating
the separation index between a pair of clusters (c.f. Qiu and Joe,
2006a, 2006b). 
projDirMethod 
Indicating the method to get the optimal projection direction when calculating
the separation index between a pair of clusters (c.f. Qiu and Joe,
2006a, 2006b). 
alpha 
Tuning parameter reflecting the percentage in the two
tails of a projected cluster that might be outlying.
We set 
ITMAX 
Maximum iteration allowed when to iteratively calculating the optimal projection direction. The actual number of iterations is usually much less than the default value 20. 
eps 
Convergence threshold. A small positive number to check if a quantitiy q
is equal to zero. If q< 
quiet 
A flag to switch on/off the outputs of intermediate results and/or possible warning messages. The default value is 
outputDatFlag 
Indicates if data set should be output to file. 
outputLogFlag 
Indicates if log info should be output to file. 
outputEmpirical 
Indicates if empirical separation indices and projection directions should be
calculated. This option is useful when generating clusters with sizes which
are not large enough so that the sample covariance matrices may be singular.
Hence, by default, 
outputInfo 
Indicates if theoretical and empirical separation information data frames should be output to a file with format ‘[fileName]\_info.log’. 
The function simClustDesign
is an implementation of the design for
generating random clusters proposed in Qiu and Joe (2006a). In the design,
the degree of separation between any cluster and its nearest neighboring
cluster could be set to a specified value while the cluster covariance
matrices can be arbitrary positive definite matrices, and so that clusters
generated might not be visualized by pairwise scatterplots of variables.
The separation between a pair of clusters is measured by the separation index
proposed in Qiu and Joe (2006b).
The current version of the function simClustDesign
implements two
methods to generate covariance matrices for clusters. The first method,
denoted by eigen
, first randomly generates eigenvalues
(λ_1,…>λ_p) for the covariance matrix
(\boldsymbol{Σ}), then uses columns of a randomly generated
orthogonal matrix
(\boldsymbol{Q}=(\boldsymbol{α}_1,…,\boldsymbol{α}_p))
as eigenvectors. The covariance matrix
\boldsymbol{Σ} is then contructed as
\boldsymbol{Q}*diag(λ_1,…,λ_p)*\boldsymbol{Q}^T.
The second method, denoted as unifcorrmat
, first generates a random
correlation matrix (\boldsymbol{R}) via the method proposed in Joe (2006),
then randomly generates variances (σ_1^2,…, σ_p^2) from
an interval specified by the argument rangeVar
. The covariance matrix
\boldsymbol{Σ} is then constructed as
diag(σ_1,…,σ_p)*\boldsymbol{R}*diag(σ_1,…,σ_p).
For each data set generated, the function simClustDesign
outputs
four files: data file, log file, membership file, and noisy set file.
All four files have the same format:
‘[fileName]J[j]G[g]v[p1]nv[p2]out[numOutlier]\_[numReplicate].[extension]’
where ‘extension’ can be ‘dat’, ‘log’, ‘mem’, or
‘noisy’. ‘J’ indicates separation index, with ‘j’
indicating the level of the factor ‘separation index’;
‘G’ indicates number of clusters, with ‘g’ indicating the
level of the factor ‘number of clusters’; ‘v’ indicates
the number of nonnoisy variables, with ‘p1’ indicating the level
of the factor ‘number of nonnoisy variables’; ‘nv’ indicates
the number of noisy variables, with ‘p2’ indicating the level of
the factor ‘number of noisy variables’; ‘out’ indicates
number of outliers, with ‘numOutlier’ indicating the value of the
argument numOutlier
of the function simClustDesign
;
‘numReplicate’ indicates the value of the argument numReplicate
of the function simClustDesign
.
The data file with file extension ‘dat’ contains n+1 rows and p columns, where n is the number of data points and p is the number of variables. The first row is the variable names. The log file with file extension ‘log’ contains information such as cluster sizes, mean vectors, covariance matrices, projection directions, separation index matrices, etc. The membership file with file extension ‘mem’ contains n rows and one column of cluster memberships for data points. The noisy set file with file extension ‘noisy’ contains a row of labels of noisy variables.
When generating clusters, population covariance matrices are all
positivedefinite. However sample covariance matrices might be
semipositivedefinite due to small cluster sizes. In this case, the
function genRandomClust
will automatically use the
“fixedpoint” method to search the optimal projection direction.
The function outputs four data files for each data set (see details).
This function also returns separation information data frames
infoFrameTheory
and infoFrameData
based on population
and empirical mean vectors and covariance matrices of clusters for all
the data sets generated. Both infoFrameTheory
and infoFrameData
contain the following seven columns:
Column 1: 
Labels of clusters (1, 2, …, numClust), where numClust is the number of clusters for the data set. 
Column 2: 
Labels of the corresponding nearest neighbors. 
Column 3: 
Separation indices of the clusters to their nearest neighboring clusters. 
Column 4: 
Labels of the corresponding farthest neighboring clusters. 
Column 5: 
Separation indices of the clusters to their farthest neighbors. 
Column 6: 
Median separation indices of the clusters to their neighbors. 
Column 7: 
Data file names with format ‘[fileName]J[j]G[g]v[p1]nv[p2]out[numOutlier]\_[numReplicate]’ (see details). 
The function also returns three lists: datList
, memList
, and noisyList
.
datList: 
a list of lists of data matrices for generated data sets. 
memList: 
a list of lists of cluster memberships for data points for generated data sets. 
noisyList: 
a list of lists of sets of noisy variables for generated data sets. 
The speed of this function might be slow.
Weiliang Qiu weiliang.qiu@gmail.com
Harry Joe harry@stat.ubc.ca
Joe, H. (2006) Generating Random Correlation Matrices Based on Partial Correlations. Journal of Multivariate Analysis, 97, 2177–2189.
Milligan G. W. (1985) An Algorithm for Generating Artificial Test Clusters. Psychometrika 50, 123–127.
Qiu, W.L. and Joe, H. (2006a) Generation of Random Clusters with Specified Degree of Separaion. Journal of Classification, 23(2), 315334.
Qiu, W.L. and Joe, H. (2006b) Separation Index and Partial Membership for Clustering. Computational Statistics and Data Analysis, 50, 585–603.
Su, J. Q. and Liu, J. S. (1993) Linear Combinations of Multiple Diagnostic Markers. Journal of the American Statistical Association, 88, 1350–1355
1 2 3 4 5 6 7 8 9 10  ## Not run:
tmp < simClustDesign(
numClust = 3,
sepVal = c(0.01, 0.21),
sepLabels = c("L", "M"),
numNonNoisy = 4,
numOutlier = 0,
numReplicate = 2,
clustszind = 2)
## End(Not run)

Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.