Introduction to LiblineaR
In LiblineaR: Linear Predictive Models Based on the LIBLINEAR C/C++ Library

knitr::opts_chunk$set(collapse = TRUE, comment = "#>")

LiblineaR wraps the LIBLINEAR C/C++ library for large-scale regularized linear classification and regression. This vignette is a practical guide to the parts of the API that are easy to get wrong: which type to pick, what bias/epsilon/svr_eps actually default to, how sparse input is handled, how class weighting (wi) works, and the two ways to search for a good cost.

library(LiblineaR)
data(iris)

Choosing a `type`

type selects both the loss function and the regularization. Two families:

Classification (type 0-7):

| type | Regularization | Loss | Solver | |---|---|---|---| | 0 | L2 | logistic | primal (Newton) | | 1 | L2 | L2-loss SVM (hinge²) | dual (coordinate descent) | | 2 | L2 | L2-loss SVM | primal (Newton) | | 3 | L2 | L1-loss SVM (hinge) | dual | | 4 | L2 | Crammer & Singer multi-class SVM | dual | | 5 | L1 | L2-loss SVM | dual | | 6 | L1 | logistic | dual | | 7 | L2 | logistic | dual |

Regression (type 11-13), all L2-regularized support vector regression:

| type | Loss | Solver | |---|---|---| | 11 | L2-loss (epsilon-insensitive²) | primal | | 12 | L2-loss | dual | | 13 | L1-loss | dual |

Rules of thumb:

For an ordinary classification problem, type=0 (logistic regression) or type=2 (L2-loss SVM) are the usual defaults; both give one weight vector per class for multi-class problems via one-vs-rest, except type=4, which always fits one weight vector per class simultaneously (relevant if you read $W's shape — see below).
type=1/2 and type=0/7 are the dual/primal formulations of the same problem respectively, and converge to (numerically close to) the same model — a useful sanity check if you're unsure which to trust.
L1-regularized types (5, 6) push weights toward exact zero — useful for feature selection on high-dimensional data.
For regression, type=11 (primal) is usually fastest; type=12/13 differ in whether large errors are penalized quadratically or linearly.

x <- iris[, 1:4]
y <- iris[, 5]

m_lr  <- LiblineaR(x, y, type = 0)   # logistic regression
m_svm <- LiblineaR(x, y, type = 2)   # L2-loss SVM

dim(m_lr$W)   # one row per class (3 classes, multi-class problem)

`bias`, `epsilon`, `svr_eps`: what the defaults actually mean

bias (default 1): if bias > 0, every row gets an extra constant feature appended with that value ([data; bias]) — this is what lets the model fit an intercept. If bias <= 0, no bias term is added at all (the decision boundary is forced through the origin). For backward compatibility, bias=TRUE/FALSE are also accepted (TRUE behaves like 1, FALSE like 0, i.e. no bias).

epsilon (default NULL): the solver's stopping tolerance. Leave it at the default — NULL lets LIBLINEAR apply its own per-solver default (0.01 for primal solvers, 0.1 for dual solvers; these differ because primal and dual solvers measure convergence on different quantities). Passing a numeric value overrides that for every solver uniformly, which is rarely what you want unless you're deliberately trading convergence tightness for speed.

svr_eps (regression only, default 0.1 if left NULL): the width of the epsilon-insensitive tube — errors smaller than this aren't penalized at all. There's no universally good default; it depends on the scale of your target variable, so it's worth setting explicitly for regression:

xr <- as.matrix(iris[, 1:3])
yr <- iris[, 4]
m_svr <- LiblineaR(xr, yr, type = 11, svr_eps = 0.05)

Sparse input

data (and predict()'s newx) accept dense matrices/data frames, or sparse matrices of class matrix.csr/matrix.csc/matrix.coo (package SparseM) or dgCMatrix/dgRMatrix/dgTMatrix (package Matrix). The type is detected automatically — no separate argument needed. All six sparse classes and dense input give identical coefficients and predictions on the same data; pick whichever integrates better with the rest of your pipeline.

if (requireNamespace("Matrix", quietly = TRUE)) {
  x_sparse <- Matrix::Matrix(as.matrix(x), sparse = TRUE)
  m_sparse <- LiblineaR(x_sparse, y, type = 0)
  identical(dim(m_sparse$W), dim(m_lr$W))
}

Class weighting with `wi`

wi reweights each class's effective regularization constant (C_class = cost * wi[class], default weight 1 for every class not named). This is the tool for imbalanced data: naming only the minority class with a higher weight pushes the solver to trade some overall accuracy for better recall on that class — a deliberate trade-off, not a bug, and one you should expect to see reflected in a lower raw accuracy alongside a better balanced accuracy.

# Not all classes need to be named -- only the one(s) you want to reweight.
m_weighted <- LiblineaR(x, y, type = 0, wi = c(setosa = 5))

Finding a good `cost`: `heuristicC()`, `cross`, and `findC`

Three complementary tools:

heuristicC(data): a fast, closed-form heuristic (Joachims' SVM-light heuristic) giving a reasonable starting point for cost, computed directly from the data with no training involved.
cross=k: runs k-fold cross-validation at the given cost and returns the CV accuracy (classification) or MSE (regression) as a single number — no model object. Useful for evaluating one specific cost.
findC=TRUE: automatic search for a good cost, using repeated cross-validation internally. Only supported for type=0 and type=2 (the primal L2-regularized solvers); any other type raises an error. Returns the best cost found, not a model — retrain with that value to get the actual model.

co <- heuristicC(x)
co

acc <- LiblineaR(x, y, type = 0, cost = co, cross = 5)
acc

best_cost <- LiblineaR(x, y, type = 0, findC = TRUE, cross = 5)
best_cost

m_final <- LiblineaR(x, y, type = 0, cost = best_cost)

Predicting

predict() accepts a vector (single feature) or a matrix/data frame with the same columns as training — reordered and with any extra columns dropped automatically, matched by column name if newx has names.

p <- predict(m_final, x)
mean(as.character(p$predictions) == as.character(y))

# Probabilities are only available for logistic regression (type 0, 6, 7).
p_proba <- predict(m_final, x, proba = TRUE)
head(p_proba$probabilities)