find_param: Solves a system of nonlinear equations

In zq00/glmhd: Estimates the inflation and variance of the MLE from a high-dimensional GLM

Solves a system of nonlinear equations

Description

This function solves a system of equations, whose solution characterizes the asymptotic bias and variance of the M-estimator (in case of the MLE, it is the negative log-likelihood).

Usage

find_param(
  rho_prime = rho_prime_logistic,
  f_prime1 = f_prime1_logistic,
  f_prime0 = f_prime0_logistic,
  kappa,
  gamma,
  beta0 = 0,
  intercept = TRUE,
  verbose = FALSE,
  x_init = NULL
)

Arguments

rho_prime	A function that computes the success probability \rho'(t) = \mathrm{P}(Y=1 | X^\top \beta = t), here \beta is the coefficient. The default is logistic model.
f_prime1	A function. Derivative of the loss function when Y = 1. The default is the derivative of the negative log-likelihood of logistic regression when Y = 1.
f_prime0	A function. Derivative of the loss function when Y = -1. The default is the derivative of the negative log-likelihood of logistic regression when Y = -1.
kappa	Numeric. The problem dimension \kappa = p/n.
gamma	Numeric. Signal strength \gamma = \sqrt{\mathrm{Var}(X^\top \beta)}.
beta0	Numeric. Intercept.
intercept	If TRUE, the glm contains an intercept term. intercept = TRUE by default.
verbose	If TRUE, print progress at each step.
x_init	Initial values for the parameters.

Value

A vector solution to the system. When gamma != 0 and b !=0, returns (\alpha_\star, \lambda_\star, \sqrt{\kappa}\sigma_\star, b_\star). When signal strength is zero (gamma = 0), returns the solution to the system with three equations (\alpha_\star = 0, \lambda_\star, \sqrt{\kappa}\sigma_\star, b_\star). When gamma = 0 and b = 0, returns (\alpha_\star = 0, \lambda_\star, \sqrt{\kappa}\sigma_\star).

References

The Impact of Regularization on High-dimensional Logistic Regression, Fariborz Salehi, Ehsan Abbasi and Babak Hassibi, Proceedings of NeurIPS 2019.

Examples

# Compute parameters for a logistic model
param <- find_param(kappa = 0.1, gamma = sqrt(5))
# Asymptotic bias
param[1]
# Standard deviation
param[3] / sqrt(0.1)
# Another example
param <- find_param(kappa = 0.1, gamma = 0, intercept = FALSE)
# Asymptotic standard deviation
param[2] / sqrt(0.1)

zq00/glmhd documentation built on April 7, 2023, 7:45 a.m.