trace_ratio: Trace Ratio Optimization Returning a Discriminant Projector

In bbuchsbaum/discursive: What the Package Does (One Line, Title Case)

Trace Ratio Optimization Returning a Discriminant Projector

Description

This function performs trace ratio optimization, a technique commonly used in dimensionality reduction and feature extraction problems such as those found in discriminant analysis. The trace ratio criterion seeks a subspace that maximizes the ratio of the trace of one scatter matrix (A) to another (B).

Usage

trace_ratio(A, B, X = NULL, y = NULL, ncomp = 2, eps = 1e-06, maxiter = 100)

Arguments

A	A symmetric numeric matrix representing the "numerator" scatter matrix.
B	A symmetric numeric matrix representing the "denominator" scatter matrix.
X	(optional) A numeric matrix (n \times d) of the original data. If provided, we compute the training scores X \times V in the final projector.
y	(optional) A vector of length n of class labels. If provided, stored in the projector's labels field.
ncomp	An integer specifying the number of components (dimension of the subspace) to extract. Default is 2.
eps	A numeric tolerance for convergence. The iterative procedure stops if the change in the trace ratio (\rho) is less than eps. Default is 1e-6.
maxiter	An integer specifying the maximum number of iterations for the optimization. Default is 100.

Details

Formally, given two symmetric matrices A and B, we want to find a projection matrix V (with orthonormal columns) that maximizes:

\mathrm{trace}(V^T A V) / \mathrm{trace}(V^T B V).

This is solved iteratively using eigen decomposition methods.

What if you want a discriminant_projector for typical usage? If you provide X (the original data) and optional y (class labels), this function will build a discriminant_projector object, storing:

• v ~ the n x p projection vectors (the subspace),

• s ~ the training scores (X \times v),

• sdev ~ standard deviations of each dimension in s,

• labels ~ set to y if provided,

• classes ~ set to "trace_ratio".

If X is NULL, we just return a projector with v but no training scores.

This function solves a generalized eigenvalue-like problem iteratively, updating the projection V until convergence. It uses PRIMME::eigs_sym for eigen decompositions at each step. The approach is inspired by the trace ratio criterion found in linear discriminant analysis and related dimension reduction techniques.

Algorithm:

1. Initialize V randomly (size n \times ncomp).

2. At iteration t:

   • Compute \rho_t = trace(V^T A V) / trace(V^T B V).

   • Form M = A - \rho_t B.

   • Eigen-decompose M to get top ncomp eigenvectors => new V.

3. Repeat until |\rho_{t+1} - \rho_t| < eps or maxiter is reached.

Value

A discriminant_projector object, with:

• v: A matrix whose columns are the ncomp vectors corresponding to the subspace that maximizes the trace ratio criterion.

• s: If X is given, the training scores (X \times v).

• sdev: The standard deviations of columns in s.

• labels: set to y if provided, otherwise NULL.

• classes: A character string "trace_ratio".

See Also

eigs_sym for the eigen decomposition solver, and between_class_scatter, within_class_scatter for common scatter matrices used in discriminant analysis.

Examples

## Not run: 
data(iris)
X <- scale(iris[,1:4])
y <- iris[,5]

# Build scatter matrices (a typical case in LDA)
A <- between_class_scatter(X, y)
B <- within_class_scatter(X, y)

# Solve trace ratio with 3 components, storing training scores
proj <- trace_ratio(A, B, X = X, y = y, ncomp = 3)
print(proj)

# You can now project new data with project(proj, newdata), etc.

## End(Not run)

bbuchsbaum/discursive documentation built on April 14, 2025, 4:57 p.m.