HierarchicalSparseCluster.permute: Choose tuning parameter for sparse hierarchical clustering
In sparcl: Perform Sparse Hierarchical Clustering and Sparse K-Means Clustering

Description Usage Arguments Details Value Author(s) References See Also Examples

The tuning parameter controls the L1 bound on w, the feature weights. A permutation approach is used to select the tuning parameter.

HierarchicalSparseCluster.permute(x, nperms = 10, wbounds = NULL,
dissimilarity=c("squared.distance",
"absolute.value"),standardize.arrays=FALSE)
## S3 method for class 'HierarchicalSparseCluster.permute'
plot(x,...) 
## S3 method for class 'HierarchicalSparseCluster.permute'
print(x,...)

`x`	A nxp data matrix, with n observations and p feaures.
`nperms`	The number of permutations to perform.
`wbounds`	The sequence of tuning parameters to consider. The tuning parameters are the L1 bound on w, the feature weights. If NULL, then a default sequence will be used. If non-null, should be greater than 1.
`dissimilarity`	How should dissimilarity be computed? Default is squared.distance.
`standardize.arrays`	Should the arrays first be standardized? Default is FALSE.
`...`	not used.

Let $d_ii'j$ denote the dissimilarity between observations i and i' along feature j.

Sparse hierarchical clustering seeks a p-vector of weights w (one per feature) and a nxn matrix U that optimize $maximize_U,w sum_j w_j sum_ii' d_ii'j U_ii'$ subject to $||w||_2 <= 1, ||w||_1 <= s, w_j >= 0, sum_ii' U_ii'^2 <= 1$, where s is a value for the L1 bound on w. Let O(s) denote the objective function with tuning parameter s: i.e. $O(s)=sum_j w_j sum_ii' d_ii'j U_ii'$.

We permute the data as follows: within each feature, we permute the observations. Using the permuted data, we can run sparse hierarchical clustering with tuning parameter s, yielding the objective function O*(s). If we do this repeatedly we can get a number of O*(s) values.

Then, the Gap statistic is given by $Gap(s)=log(O(s))-mean(log(O*(s)))$. The optimal s is that which results in the highest Gap statistic. Or, we can choose the smallest s such that its Gap statistic is within $sd(log(O*(s)))$ of the largest Gap statistic.

`gaps`	The gap statistics obtained (one for each of the tuning parameters tried). If O(s) is the objective function evaluated at the tuning parameter s, and O(s) is the same quantity but for the permuted data, then Gap(s)=log(O(s))-mean(log(O(s))).
`sdgaps`	The standard deviation of log(O*(s)), for each value of the tuning parameter s.
`nnonzerows`	The number of features with non-zero weights, for each value of the tuning parameter.
`wbounds`	The tuning parameters considered.
`bestw`	The value of the tuning parameter corresponding to the highest gap statistic.

Daniela M. Witten and Robert Tibshirani

Witten and Tibshirani (2009) A framework for feature selection in clustering.

HierarchicalSparseCluster, KMeansSparseCluster, KMeansSparseCluster.permute

  # Generate 2-class data
  set.seed(1)
  x <- matrix(rnorm(100*50),ncol=50)
  y <- c(rep(1,50),rep(2,50))
  x[y==1,1:25] <- x[y==1,1:25]+2
  # Do tuning parameter selection for sparse hierarchical clustering
  perm.out <- HierarchicalSparseCluster.permute(x, wbounds=c(1.5,2:6),
nperms=5)
  print(perm.out)
  plot(perm.out)
  # Perform sparse hierarchical clustering
  sparsehc <- HierarchicalSparseCluster(dists=perm.out$dists, wbound=perm.out$bestw, 
method="complete")
  par(mfrow=c(1,2))
  plot(sparsehc)
  plot(sparsehc$hc, labels=rep("", length(y)))
  print(sparsehc)
  # Plot using knowledge of class labels in order to compare true class
  #   labels to clustering obtained
  par(mfrow=c(1,1))
  ColorDendrogram(sparsehc$hc,y=y,main="My Simulated
Data",branchlength=.007)

Running sparse hierarchical clustering on unpermuted data
123456
Running sparse hierarchical clustering on permuted data
Permutation  1  of  5
123456
Permutation  2  of  5
123456
Permutation  3  of  5
123456
Permutation  4  of  5
123456
Permutation  5  of  5
123456

Tuning parameter selection results for Sparse Hierarchical Clustering:
  Wbound # Non-Zero W's Gap Statistic Standard Deviation
1    1.5              5        0.0594             0.0012
2    2.0              9        0.0801             0.0012
3    3.0             16        0.0953             0.0011
4    4.0             23        0.1000             0.0004
5    5.0             37        0.0960             0.0003
6    6.0             50        0.0790             0.0003
Tuning parameter that leads to largest Gap statistic:  4
1234567
Wbound is  4 :
Number of non-zero weights:  23
Sum of weights:  3.999973