gap: Gap statistic algorithm.

In MetabolSSMF: Simplex-Structured Matrix Factorisation for Metabolomics Analysis

Gap statistic algorithm.

Description

Estimating the number of prototypes/clusters in a data set using the gap statistic.

Usage

gap(
  data,
  rss,
  meth = c("kmeans", "uniform", "dirichlet", "nmf"),
  itr = 50,
  lr = 0.01,
  ncore = 2
)

Arguments

data	Data matrix or data frame.
rss	Numeric vector, residual sum of squares from ssmf model using the number of clusters 1,2, \ldots, k.
meth	Character, specification of method to initialise the W and H matrix, see 'method' in init( ).
itr	Integer, number of Monte Carlo samples.
lr	Optimisation learning rate in ssmf().
ncore	The number of cores to use for parallel execution.

Details

This gap statistic selects the biggest difference between the original residual sum of squares (RSS) and the RSS under an appropriate null reference distribution of the data, which is defined to be

\mathrm{Gap}(k) = \frac{1}{B} \sum_{b=1}^{B} \log(\mathrm{RSS}^*_{kb}) - \log(\mathrm{RSS}_{k}),

where B is the number of samples from the reference distribution; \mathrm{RSS}^*_{kb} is the residual sum of squares of the b^th sample from the reference distribution fitted in the SSMF model model using k clusters; RSS_{k} is the residual sum of squares for the original data X fitted the model using the same k. The estimated gap suggests the number of prototypes/clusters (\hat{k}) using

\hat{k} = \mathrm{smallest} \ k \ \mathrm{such \ that} \ \mathrm{Gap}(k) \geq \mathrm{Gap}(k+1) - s_{k+1},

where s_{k+1} is standard error that is defined as

s_{k+1}=sd_k \sqrt{1+\frac{1}{B}},

and sd_k is the standard deviation:

sd_k=\sqrt{ \frac{1}{B} \sum_{b} [\log(\mathrm{RSS}^*_{kb})-\frac{1}{B} \sum_{b} \log(\mathrm{RSS}^*_{kb})]^2}.

Value

gap Gap value vector.

optimal.k The optimal number of prototypes/clusters.

standard.error Standard error vector.

Author(s)

Wenxuan Liu

References

Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the Number of Clusters in a Data Set via the Gap Statistic. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 63(2), 411–423. <doi:10.1111/1467-9868.00293>

Examples

# example code

data <- SimulatedDataset

k <- 6

rss <- rep(NA, k)
for(i in 1:k){
  rss[i] <- ssmf(data = data, k = i)$SSE
}

gap(data = data, rss = rss)

MetabolSSMF documentation built on April 3, 2025, 5:44 p.m.