Three vignettes ship with the package (quickstart, inference modes, spatial models). The other ten (priors, temporal models, model comparison, calibration, reliability, data formatting, random slopes, EM + Laplace, checkpointing, custom latent processes) are articles on the package website, https://gillescolling.com/tulpa/articles/.
energy_bfmi() is exported: E-BFMI from a vector of per-iteration
Hamiltonian energies and their chain ids, diffed within each chain and
reported for the worst chain, so a model package's energy diagnostic uses
the same computation as tulpa's.
print(summary(fit)) leaves out the Reliability section when the fit's
record has nothing to report under it, as on a plain Laplace fit, instead of
ending on a bare header (#961).
The Polya-Gamma Gibbs areal kernels (binomial ICAR, BYM2 and restricted ICAR, negative-binomial ICAR) sample the ICAR prior the other backends sample on a disconnected graph. Each component's level, an island's whole value, now carries the field's own precision, where the kernels gave it a flat prior and an island a near-flat N(0, 1000) one. On a chain with a separated island the Gibbs island read 14.5 against the sampler's 0.32 and pulled the mainland and the intercept with it; it now reads 0.30. A component with no observation is drawn from its prior instead of refused. On a connected graph nothing changes (#963).
field_effects() and posterior_predict() on a Gibbs BYM2 fit use the
combined field sigma * (sqrt(rho) * s * phi + sqrt(1 - rho) * theta)
that enters the linear predictor, where they summed the two unit-scale
components (#964).
An isolated node is reported with what the field does with it: an ICAR field gives it its own N(0, 1 / tau) level and a BYM2 field unit variance on its structured part, both as a message; a proper CAR field leaves it without a prior and warns. The constructors had warned that every field is improper there (#962).
latent_factors() is a generic and spatiotemporal_effects() dispatches to
a default method: both read the fields a fit stores (the factor[i,k] draw
columns and $.internal$latent_info; the $spatiotemporal block and its
draws) on a fit of any class, so a model package re-exports them instead of
defining its own copies that mask tulpa's on attach (gcol33/tulpaRatio#100).summary() on a tulpa_fit returns a summary.tulpa_fit object: the call,
how the fit was produced ($inference: backend, tier, mode, selection
reason), the fixed-effect table with intervals, variance components,
integrated hyperparameters, smoothers, dropped axes, the fixed-effect prior
and a reliability headline read through diagnostics(). print() renders
it; coef(summary(fit)) is the fixed-effect table summary() returned
before. print(fit) is now the short view: backend, call, observation count,
coefficient estimates and a pointer to summary() (gcol33/tulpa#953).
tulpa_fit_summary(object, coefficients) is the constructor summary()
returns through, so a model package whose fit stores its coefficients in
its own layout returns the same object.tulpa_outer_grid() returns a nested-Laplace fit's outer hyperparameter
grid: nodes, normalized weights, log marginals, the integrated support and
the per-cell latent modes. diagnostics() now also carries the outer
k-hat's bootstrap interval and band verdict, the outcome of a
control$k_quality request and the per-index inner skewness, so the
vignettes read none of these off the fit with $ (gcol33/tulpa#958).methods(class = )
lists: the nested-Laplace results (tulpa_nested_laplace_joint,
_joint_multi, tulpa_st_nested) print their own header, and
tulpa_family, tulpa_hyper_axis_spec, tulpa_temporal_latent_block, the
SVC / TVC / temporal / space-time summaries and tulpa_variogram print. The
subclass tags that carried no behaviour are gone: tulpa_agq_fit,
tulpa_imh_fit, tulpa_mala_fit, tulpa_pathfinder_fit,
tulpa_sample_fit, tulpa_ep, tulpa_tgmrf and tulpa_hyper_grid fits are
plain tulpa_fits (the backend is on the fit and in its header), and
spatiotemporal_effects() returns a plain array or data frame
(gcol33/tulpa#954).\dontrun{} now run on a small simulated fixture
(latent_factors(), spatiotemporal_effects(), tgmrf_cpp(),
tulpa_em_mc(), tulpa_nuts_spde(), tulpa_joint_inner_vcov_blocks(),
tulpa_joint_inner_vcov_mixture(); tulpa_cache_clear() runs
interactively only) (gcol33/tulpa#956)..spd_inverse(), in place of solve(A) at 26
sites. An indefinite matrix now takes each site's existing fallback (an NA
covariance with a warning, a declined cell, a NULL) instead of returning a
"covariance" with negative variances; the joint CCD mode-find declines such
a Hessian as hessian_not_pd (gcol33/tulpa#957).field_effects() returns a spatial field's posterior per unit (an areal
region or an SPDE mesh node): estimate, SD, 95% bounds and the source of
each row, as ranef() reports a group. It reads ICAR, BYM2 and proper-CAR
fields off sampler draws (through the engine's own linear-predictor
assembly), Polya-Gamma Gibbs draws, a nested-Laplace grid's per-cell
Gaussian mixture and a conditional-Laplace mode, and an SPDE field's
mesh-node draws or mode. The field is located from the fit's recorded block
sizes; posterior_predict() reads the Laplace and Gibbs fields through the
same helpers (gcol33/tulpa#960).
A NUTS or fixed-trajectory HMC step that reaches a non-finite energy now
counts as acceptance 0 in step-size adaptation. std::min(1.0, exp(NaN))
is 1, so such a step read as a perfect one, dual averaging raised the step
size until every step diverged, and the chain stood still for the whole
sampling phase. On a 16-node BYM2 field with an island, 2 of 9 seeds had a
chain adapted to a step size of 7.4 (the other chain 0.23), 500 of 500
post-warmup draws divergent, Rhat 2.2 and a slope mean of 0.82 or 0.53
against 0.64; across 20 seeds the slope mean now ranges 0.628 to 0.651 with
Rhat at most 1.01 (gcol33/tulpa#959). One helper,
metropolis_accept_prob(), carries the acceptance for both samplers.
The joint driver's outer-grid placement now places every scale axis of a
multi-block prior, not only a copied field's SD: an iid random effect's SD
and an arm-specific field's precision are laid at the outer mode like the
copied field's, and both fire on an unresolved posterior as well as on a
rail (gcol33/tulpa#951). The placement mode-find searches each axis over the
nodes it was declared on, so an axis that the adaptive lattice, a pilot or
the prune screen kept at one node is no longer frozen out of it, and its
trust region shortens the Newton step as a whole, which keeps it uphill on a
correlated outer posterior. On the full 25 km occu_cover fits these axes had
been integrated at a single grid node with posterior SD 0. An axis still
holding 0.99 of its marginal on one node is recorded
(outer_grid_single_node_axes) and warned about when the engine was free
to place it.
A cell whose inner Hessian is not positive definite at the point the inner
Newton returned is re-solved from a converged neighbour and, if it stays
indefinite, left out of the integration and recorded (indefinite_cells,
indefinite_mass) with a warning (gcol33/tulpa#952). Under a flat outer
hyperprior such a cell had carried weight and made every posterior draw
fail its Cholesky.
mu clamp the fit applies before evaluating them, are
exported as inst/include/tulpa/family_density.h (gcol33/tulpa#950). The
Laplace kernels read them from there, and a package linking to tulpa can
score a fit's pointwise log-likelihood (WAIC, PSIS-LOO, CPO) with the
density the fit maximised instead of a restatement of it. Fitted values are
unchanged.diagnostics() reports the exact moments of the outer-grid mixture a
fit's draws were sampled from, in place of the Monte-Carlo mean and SD of
those draws (gcol33/tulpa#949). A sampler that draws i.i.d. from a known
mixture stamps it on the draws as attr(., "grid_mixture") (weights,
per-cell modes, per-cell variances or covariances);
tulpa_posterior_draws() on a single-block fit and the node mixture of
tulpa_re_cov_nested() now do, as tulpaObs's grid-mixture draws already
did. Those rows carry mcse_mean = 0 and moment_source =
"mixture_moments", columns past the stamped block join the fit's
$means / $sds by name, and every other row keeps the draw moments with
moment_source = "draws". Draws the subspace debias corrected are a
different posterior and are not stamped. On a 25 km Calluna occu_cover
fit the 1,000-draw table differed from the exact means by an rms of 0.032
posterior SD and from the exact SDs by a ratio of 0.935 to 1.066, and
changed with the RNG state; the estimate and SE summary() reads off the
same draws now come from the same exact moments.
tulpa_joint_inner_vcov_mixture() returns the outer grid's posterior
covariance of a joint fit's latent coordinates by the law of total
covariance, accumulated in the engine without one length(idx) square block
per grid cell (gcol33/tulpa#947). tulpa_joint_inner_vcov_blocks() holds
every cell's block, so with a field in idx its memory is the grid size
times the latent dimension squared: 878 cells at 4479 latents is about
141 GB, and the process died with std::bad_alloc inside the OpenMP region.
The mixture door keeps one result plus a batch of per-cell strips (the
fixed-effect columns and the field diagonal, which is all the
field-marginal recipe forms), skips cells of zero weight, adds the cells in
grid order so the result does not depend on n_threads, and returns each
cell's fixed-effect block for a mixture draw. Both doors read one per-cell
assembly.
The cells a refinement pass adds to the outer grid go through the cheap-pass
screen the declared grid goes through, at the fit's own prune_tol, on the
joint, multi-block and registry doors (gcol33/tulpa#948). A round's cells
are ranked against the mass of the grid they join, not against each other,
so a level laid across rows that hold no posterior, or an axis extended off
the mode, costs a truncated solve per cell instead of a full inner Newton.
The dropped-mass bound is read over the merged grid under its final measure.
On the 25 km Calluna occu_cover fit 605 of 878 solved cells carried weight
under 1e-6; on the test fixture the screen leaves 67 of 204 cells for a
full solve (208 of 208 unscreened), with the hyperparameter means within
0.01 posterior SD. A refinement kernel now takes
kernel_fn(new_cells, warm_start, store_extras, screen).
This release collects 0.6.11 and 0.6.12; their entries below carry the detail.
store_eta = TRUE wrote the predictor computed at the
top of the sweep, before beta, the random effects and the field moved, so
saved row s paired the parameters of draw s with the predictor of draw
s - 1. PgGibbsCommon::save() now rebuilds it from the saved state and
the same field contribution the recorded log density reads.tulpa_nested_laplace_joint() takes a model with no latent block
(gcol33/tulpa#939). prior = list() is a list of zero blocks: each arm's
predictor is its fixed effects, and the outer grid is the per-arm
dispersion axes of phi_grid alone, or one cell when there are none. It
used to be refused, so a fixed-effects ratio model had to carry a silent
iid block held at sigma = 0; the two give the same log marginals,
weights, coefficients and covariance. tulpa_nested_laplace(), whose outer
grid has no dispersion axis, refuses an empty prior and names the doors
that fit one.icar, bym2 or
car_proper block may carry projector, one N_k x n_units matrix per
arm (dense or sparse), in place of spatial_idx; arm k's predictor then
receives A_k z with z on the block's own prior. Restricted spatial
regression is A_k = P_k S_k, which used to be refused on the Laplace tier.
The field's level is placed from A_k 1: removed with no fold where it is
zero, folded into the intercept where it is the intercept column. A
projector equal to the unit incidence reproduces the gathered fit. On a
binomial 8 x 8 lattice (4 seeds) the restricted fit's fixed effects sit
within 0.31 posterior SDs of the exact Polya-Gamma Gibbs RSR sampler, with
SDs within 4%; the unrestricted ICAR shows the same offset (0.29-0.32 SDs)
against its own Gibbs run, so the offset comes from the Laplace
approximation and not from the projector. The SPDE block reads its mesh
projector through the same code.tulpa() fits a restricted areal field (spatial_rsr() on an icar, bym2
or car_proper field) by nested Laplace, under any family that path takes.
The single-arm multi-block driver behind tulpa_nested_laplace() reads a
block projector as the joint one does, through the same reader and the
same rule for the field's level, and a single projected block routes there;
tulpa() hands it A = S P_perp, the unit-level projector the Gibbs
sampler applies, row by observation. mode = "auto" keeps the Polya-Gamma
Gibbs sampler for a binomial response and takes nested Laplace otherwise,
where a non-binomial restricted field used to be refused; the restricted
field can now also share the nested stack with a temporal field, smoothers
or latent() blocks. On a declared grid the registry door and the joint
door give the same log marginals and coefficients for a restricted icar,
bym2 and car_proper field. The conditional mode = "laplace" refuses a
restricted field.spatial_rsr() on spatial_gp()) is fitted by
nested Laplace through tulpa() under any family (gcol33/tulpa#942); it
used to be binomial and Gibbs only. The single-arm multi-block driver takes
an nngp block, gathered through spatial_idx or read through a
projector, and tulpa() hands it A = S P_perp at the field's unique
locations. mode = "auto" keeps Gibbs for a binomial response. An NNGP
field held by its incidence projector, gathered on the multi-block driver
and fitted alone by the single-block kernel give the same log marginals and
coefficients. Against the Polya-Gamma Gibbs RSR sampler (binomial, n = 150,
4 seeds) the restricted slope sits 0.17-0.69 posterior SDs from the Gibbs
mean with SDs within 2%, where the unrestricted NNGP field sits 0.07-1.34
SDs from its own Gibbs run (dev_notes/issue942).tulpa_nested_laplace() solves an areal, temporal or NNGP field past 200
latents on the sparse joint Newton (gcol33/tulpa#944). The single-arm
multi-block driver assembled a dense n_x x n_x Hessian whenever a latent
block was present, so a 2000-region ICAR field factorized a 2000 x 2000
matrix per Newton step. Past the threshold, or with a block whose prior is
sparse-only, the fit now runs as a one-arm fit on the joint driver's sparse
path, a model-supplied likelihood included. The ICAR, BYM2, proper-CAR, RW1,
RW2 and AR1 blocks of all three drivers fill their dense and sparse prior
scatters from one setter each, and the NNGP block lost its dense twin. Below
the threshold the dense solve equals the joint door forced sparse; above it
the routed fit equals the joint door.tulpa_nested_laplace_joint() takes an nngp block on the list-of-blocks
path (gcol33/tulpa#943), gathered through a per-arm spatial_idx or read
through a per-arm projector, so an NNGP field, restricted or not, can be
shared across arms. On one declared grid a one-arm joint fit equals the
registry door's, gathered and restricted, and an incidence projector equals
the gathered field. The joint hyperprior read the block's b1.phi_gp
lengthscale axis as a per-arm dispersion; a phi_ axis is one only without
a block prefix (.joint_axis_is_dispersion()).(1 | g)
term, a temporal field, smoothers and latent() blocks through tulpa()
(gcol33/tulpa#794, gcol33/tulpa#945). mode = "auto" used to send a
continuous field with a random-intercept term to exact NUTS and
mode = "nested_laplace" refused it. The single-arm multi-block driver
takes an hsgp block, the same basis block the single-block kernel and the
joint driver build, and reads its per-row predictor and predictive variance
off the basis at each cell. An HSGP field alone on that driver gives the
single-block kernel's log marginals, coefficients, fitted predictor and its
variance.hsgp block at the (sigma2, lengthscale) it
declares (gcol33/tulpa#946). The block read its two grid columns as
logarithms, which cpp_nested_laplace_hsgp supplied and the joint driver
did not, so a joint cell labelled (0.3, 0.2) solved the field at
(exp(0.3), exp(0.2)) while its hyperprior and every reported quantity
read (0.3, 0.2): its inner modes equalled the registry door's at exp()
of the grid. The block now reads the grid on its declared scale at every
entry, and a one-arm joint fit equals the registry door's.<tulpa/walnuts.h> (with its tuning constants in
<tulpa/walnuts_config.h>), over a caller-supplied model: gradient, kinetic
energy, metric product, drift and momentum draw. The engine's chain driver
runs it through an adapter over DenseMassMatrix, and a package with its
own sampler (tulpaRatio) runs the same code. Seeded funnel chains are
bit-identical before and after the move.n_softabs_rescued. The chain
record layout changed with it, and the layout tag is shared by every
checkpoint file, so a sampler or outer-grid checkpoint written before this
change is refused and needs a fresh path.tulpa_re_cov_nested(), and so
mode = "structured" / "laplace" with a free random-effect scale) keeps
its exact gradient when an inner solve that did not settle sits at a
line-search trial the optimizer rejects. The settled-mode gate refuses the
gradient there, but the value is still the trial's log marginal, and the
refusal counts only where the optimizer reads the gradient. It used to
replace the value with the failure sentinel and send the whole fit to the
derivative-free optimizer. On a binomial (1 | g) fit (n 3000, 300 groups)
BFGS's third trial is sigma = 8.2e7, whose inner solve stops at
max_iter; the fit now finishes on BFGS in 17 inner solves where it took 46
through Brent, at the same estimate and without the warning.tulpa_joint_inner_vcov_blocks()
(the field marginal variances behind a joint fit's per-cell covariance),
the implicit-differentiation gradient and the ST GMRF mass matrix. On a
two-arm ICAR joint fixture (106 latent coordinates, 251 cells) the
inner-vcov pass takes 0.40 ms per cell against 4.34, with the same blocks.The within-cell log-quadratic read tabulates each grid row's density and
CDF in C++ (cpp_nl_lq_group_cdf()); the rows' quadratics are still built
in R. On a single-block ICAR joint fit (16 x 16 and 24 x 24 lattices, field
coefficient and dispersion axes) the R time per fit falls from 0.41 to
0.20 s and from 0.35 to 0.16 s, and every fit field is bit-identical.
tulpa_nested_laplace_joint() gains control$hessian = "auto", the new
default: each inner Newton step takes the observed Hessian where it
factors as it stands and the cell-coupled arms' complete-data expected
information where it does not, so an indefinite Hessian is no longer
conditioned by the escalating ridge (a factorization per rung, then a short
gradient step). "lm", "fisher" and "psd" keep their meaning, and a fit
with no coupled arm runs "auto" as "lm". On the full 25 km Calluna
occu_cover fit the expected-information step took a median of 48 inner
iterations per cell and left cells at max_iter; the observed one under
the ridge took 5, but its placement mode-find ran 20 evaluations at 79 s
each to the iteration cap from the pilot's warm starts. Under "auto", with
the two screen fixes below, the fit takes 21 min against 67.6 (32 threads,
peak 18 GB against 14), no cell stops at max_iter, and the
hyperparameter summaries agree with the unscreened dense grid within 0.06
posterior SD.
"auto" the peak falls from 13.4 to 5.1 GB and the
wall time from 14.6 to 12.3 min, with the same 799 cells and every cell's
log marginal unchanged. The grid carries a per-cell s2z_field_indefinite
flag and diagnostic_summary() reports s2z_field_indefinite_cells. Both
that flag and s2z_log_det_fallback ride the refinement passes, so they
cover every cell of a refined joint grid; the fallback flag had covered
the declared cells only, and a refined cell that fell back went
unreported. The inner Newton iterations read the pinned matrix the same
way, so "auto" steps on the observed Hessian wherever the pinned matrix
is positive definite, where it used to require the field block alone to
be. On the Calluna fit no iterate reaches such a point: run side by side on
one node, every cell's log marginal is identical and both fits take 592 s.-l''(eta), where it was the expected (Fisher) information, so the Laplace
log-determinant is the textbook one at the mode. Beta-arm fits change:
the mode is the same stationary point, but each cell's log marginal moves
(-845.9 to -852.0 on a 16 x 16 ICAR cover fixture), and the outer weights,
hyperparameter summaries and fixed-effect intervals move with it. Against an
exact (60-node adaptive Gauss-Hermite) marginal on a separable beta-logit
fixture, the observed form's per-group error is about 0.6 of the Fisher
form's and varies less across the hyperparameter grid; neither is
uniformly closer on the hyperparameter posterior. On the Fisher weight the
Newton step contracted at 0.80 per iteration (61 steps to converge on a
pinned cover cell, against 6 now), and cells that hit control$max_iter
kept their full weight; on the four-seed joint fixture the mean inner
iteration count went from 9.3-16.4 to 3.6-4.0 and no cell stalls. A joint
fit records each cell's final score residual as score_max, and
glmm_weights() for beta now needs y (gcol33/tulpa#936).control
(merged over the driver's; diagnose_k = TRUE restores it). Measured on an
areal occupancy fit of 25 EM iterations: 12.0 s to 1.4 s, results identical.tulpa_nested_laplace_joint() and tulpa_nested_laplace()
(control$prune = FALSE switches it off), held to a 1% bound on the
posterior mass the dropped cells may carry and repaired rather than refit
when it misses (gcol33/tulpa#925).occu_cover fit the copy scale went from 0.274 to the
joint mode's 0.227 and the positive arm's intercept from -2.378 to -2.312
(gcol33/tulpa#925).tulpa_nested_laplace_joint() runs the same outer-grid refinement on a
multi-block prior as on a single block: the var-of-means consistency pass,
the opt-in boundary / interior pass and control$axis_refine, through one
runner, .joint_refine_outer_grid(). A CCD design, the adaptive lattice and
a locally CCD-refined grid keep the nodes they were laid on. Where a
multi-block tensor fit refines, its reported intervals are read under the
cell measure log_quad, as the single-block driver's are
(gcol33/tulpa#926).sigma on a field-SD
grid, tau on a precision grid, and their b<k>. names on a multi-block
grid. A field SD narrower than its placed cell, which sat on one node and
reported the cell as its interval, has nodes laid in its modal cell's row
until its marginal resolves. Nodes a caller wrote down are densified inside
their span and an engine-placed axis may be followed past its ends, the rule
the copy scale and the dispersions already follow
(control$axis_refine = "none" holds an axis) (gcol33/tulpa#926).min_sd_u_joint = 0.05 against the registry's min_sd_u = 0.15. With the
floor at 0.15 the placed cell stayed wider than the posterior and the
consistency pass resolved the field SD in one row only (coverage at sigma 2
on 12 x 12: 0.855). On the two-arm ICAR fixture (200 seeds per row, eight
rows) the field SD's coverage of the true value at sigma 2 goes from 0.765 to
0.945 on 12 x 12 and from 0.860 to 0.915 on 16 x 16, and summed
|coverage - 0.95| over the rows from 0.500 to 0.175 (copy scale 0.300 to
0.260). On the ladder whose truth is drawn from the prior and scored against
the exact posterior (80 seeds, 12 rungs, no placement) the refinement alone
takes it from 0.325 to 0.238 and the CRPS falls on every rung that gains
nodes (gcol33/tulpa#928).tulpa_nested_laplace() runs the consistency pass (control$var_of_means_consistency,
on by default), the opt-in boundary / interior pass (control$adaptive_grid)
and control$axis_refine over its field SD axes: tau of an icar, rw1 or
rw2 block and sigma of an iid, bym2 or spde block, single and
multi-block. The cells a pass adds are solved by the dispatcher and carry the
same per-cell data as the rest (modes, precisions, fitted_eta), tagged in
refining_axis. A fit whose field SD had collapsed onto a few nodes gains
cells and its summaries move; tulpa_grid_log_quad() of such a fit needs
refining = fit$refining_axis. The subspace debias and the corrected
integrated Laplace re-solve the fit's own cells, refinement nodes included
(gcol33/tulpa#927).debug_gradient_check = TRUE) runs one
mass-matrix SD off the initial point in a seeded direction, reported as
gradient_check$displacement. At the pilot mode the z block of the gradient
is at roundoff, so the check could not resolve it and 20 of 83 entries read
exactly zero; off the mode the (beta, z) blocks agree with central
differences to 1.2e-08 and 7.5e-09 on the periodic AR1 example
(gcol33/tulpa#929).within_cell = "log_quadratic" by default: each row's node log densities
joined by overlapping quadratics and the end quadratic continued past the
outer node, on the rows that resolve their own conditional (h / sd <= 2),
boxes elsewhere. tulpa_hyper_draws() samples the same reconstruction, and
"box_uniform" / "chord" stay selectable (gcol33/tulpa#932).tulpa_nested_laplace() (it was 0.15), where a
1400-trial paired ladder gives the same 95% intervals at 0, 0.05 and 0.15. On
the two-arm ICAR fixture the summed error of the reported intervals against
dense references falls from 0.156 to 0.038, and fixed-truth 95% coverage of
the 16x16 field SD goes from 0.922 to 0.936 (1000 seeds; a converged grid on
the same seeds reads 0.935) and of the 24x24 copy scale from 0.922 to 0.965
(600 seeds). Fits refine into more cells: 170 against 106 at 16x16, 1147
against 580 on a four-axis cover-hurdle fixture. A refinement round on
tulpa_nested_laplace_joint() solves its cells on the fit's outer width
(n_threads_outer) instead of one after another, with the same cells and
intervals to 1e-12 and 2 to 2.5 times faster on four threads. refining_axis now names
the axis whose pass added a cell, and the declared cells fix the span and
any prior read off the nodes (gcol33/tulpa#932).converged on a refined joint fit has one entry per cell of the merged grid.
It covered only the declared cells, and a shorter vector reads as all
converged, so a refined cell whose inner Newton stopped at max_iter was
reported as converged (gcol33/tulpa#932).control$max_iter
without reaching a mode is solved again from the mode of its nearest
converged cell on the grid, before the grid is read. The refinement passes
start every cell of a new level from the one heaviest base cell, which for a
level laid across many rows is the right row for one of them. A cell that
still has no mode stays in the measure at the value it stopped at and the
fit says so: nonconverged_cells lists it, nonconverged_mass is the
posterior mass such cells carry, nonconverged_resolve counts what the
re-solve recovered, diagnostics() and diagnostic_summary() name them,
and past the screen's 1% mass gate the fit warns. On a beta arm the inner
Newton weight is the Fisher information, so the solve converges linearly
(a measured contraction of 0.80 per iteration, 61 iterations to the default
tolerance, where a Newton step on the observed curvature would take a
handful); the joint door's default max_iter = 50 can leave such cells
(gcol33/tulpa#936).tulpa_re_cov_nested() solves its importance draws in
one compiled batch, each warm-started from the latent mode at the proposal
centre, where it paid one tulpa_laplace() call per draw. The draws spread
over control$k_threads (new on this door, following n_threads by
default), and the values do not depend on the width. On the binomial random
intercept of #931 (10 seeds, serial) the diagnostic takes 0.041 s instead of
0.195 s at 20 groups x 3 and 0.143 s instead of 0.327 s at 60 groups x 3; at
600 groups it takes 1.75 s serial and 0.45 s on four threads against a
0.67 s fit. Its cost scales with the inner solve, so it is not confined to
small fits. The proposal sample is drawn from the stream the fit started on,
so a subspace-debiased fit reports the plain fit's k-hat rather than one
scored on a different sample, which had also bought it a second
moment-matching pass (gcol33/tulpa#934).A Newton step of the multi-term random-effect Laplace solve (tulpa_laplace()
and everything built on it, the nested-Laplace grid included) no longer makes
full passes over the dense Hessian that nothing reads: the scatter assembles
the lower triangle the factorizations read without mirroring it, the
Hessian is zeroed once per step instead of twice, and the CHOLMOD refill
reads it in one row-order pass instead of two column-order ones. At 602
latents (600 groups) the scatter takes 0.143 ms instead of 0.276 ms and the
factorization 0.142 ms instead of 0.251 ms; every mode, log-marginal and
Hessian is bit-identical (gcol33/tulpa#934).
At 50 or more latents, the same solve assembles its Hessian into a sparse
matrix on the model's structural pattern instead of a dense one, so a Newton
step no longer touches an n x n matrix at all; between 50 and 200 latents
this moves results at rounding level (at most 7e-13 relative on the
reference fixtures). The pattern is built once and
reused across the draws of the Pareto-k batch and the cells of a grid. At 602
latents the factorization takes 0.051 ms instead of 0.142 ms per step, and
the 500-draw diagnostic batch 0.92 s instead of 1.30 s; the serial
diagnostic of tulpa_re_cov_nested() costs 1.4x the fit there (from 1.9x).
Models with spatial or temporal latent blocks keep the dense assembly
(gcol33/tulpa#934).
The per-observation work of a Laplace Newton step is cheaper. The binomial
and Poisson score and Fisher weight share one exp (they computed it twice,
through separate calls), and the solve no longer recomputes the linear
predictor or the data log-likelihood at a point its line search has just
evaluated. All three are bit-identical, and the kernel change speeds up every
binomial and Poisson Laplace fit in the engine. The 500-draw Pareto-k batch
takes 0.70 s instead of 0.92 s at 600 groups and 0.080 s instead of 0.130 s
at 60 (gcol33/tulpa#934).
LikelihoodSpec gains an optional ll_eta_weights_fn: one call returning an
observation's log-likelihood and filling its eta-space score and negative
Hessian, with the values ll_double and eta_weights_fn would give. The
spec-driven Laplace solve evaluates it on every objective evaluation and
hands the weights at the next iterate to its scatter, which then calls
nothing; the built-in families fill it, sharing the binomial and Poisson
exp. Bit-identical. This changes the struct layout, so
TULPA_ABI_VERSION is 47 and packages building a LikelihoodSpec
(tulpaObs, tulpaRatio) rebuild against this version.
The binomial and Poisson likelihood kernels evaluate exp and log through
a port of the Arm Optimized Routines implementations (src/fastmath.h, the
routines glibc and musl ship; within 0.52 ulp of the true value). They agree
with the platform library to within one ulp (identical on 99.9% of inputs
against Windows' UCRT) and are the same code on every platform, so these
kernels no longer differ in the last bit between operating systems. Fits
move at rounding level: at most 2e-11 relative on any model quantity of the
reference fixtures. With the fused weights above, the 500-draw Pareto-k
batch takes 0.38 s instead of 0.70 s at 600 groups and 0.040 s instead of
0.080 s at 60 (gcol33/tulpa#934).
An intrinsic field's sum-to-zero augmentation is stored in the sparse
Hessian pattern only for components of up to 32 nodes (S2Z_DENSIFY_MAX,
was 256) and folded in at solve time above that. Storing it makes the
component's block dense: every scatter writes n(n+1)/2 slots and the factor
carries a dense n-block, so a 16x16 lattice ICAR under the sparse joint
driver cost 12 ms per outer cell against 1 ms folded, and a 24x24 one would
have cost 93 ms against 2 ms. Measured on the two-arm ICAR fixture the two
storages cost the same at 36 nodes and storing is ahead by a fifth at 16 to
25, where many small islands would otherwise each add a solve per Newton
step. The 16x16 fixture's fit takes 0.8 s instead of 4.9 s; its cells'
log-marginals move at most 2e-14 relative, the intervals 1e-12.
TULPA_S2Z_DENSIFY_MAX still overrides the cutoff.
On the folded path the final pass reads the Newton step and the
log-determinant off the one block-Schur factor of the pinned matrix, and no
longer runs the PD-enforced solve of the unpinned Hessian, whose ridge
escalation was the normal case there and whose step only reached the
reported decrement. One factorization fewer per cell (1304 -> 936 on the
24x24 fixture, inner time -7%); pd_conditioned no longer reports that
escalation on a cell whose pinned matrix factorized cleanly. A cell the
corrected integrated Laplace is requested on keeps the solve, since it reads
that factor.
A refinement round warm-starts each new cell from the solved cell in its own
row nearest along the refined axis (.hyper_row_warm_starts()), through the
kernels' per-cell start, instead of every cell from the heaviest cell of the
grid. On a round run across the outer team, where cells otherwise start from
one pilot mode, the refined cells take 10-15% fewer inner Newton steps on the
two-arm ICAR fixtures (3.6 -> 3.3 per cell at 16x16, 3.5 -> 3.0 at 24x24);
on a serial round the chained start already did most of this.
Log-marginals move at most 6e-14 relative.
On that Calluna fit (LiSC, 32 threads) the engine defaults take 25.3 min and 20 full solves in the final grid besides the 79-evaluation mode-find; the engine behind the 78 EVA fits took 198.85 min on a 22-cell grid for an Ambrosia fit of the same design. Every hyperparameter and fixed effect agrees with the unscreened grid to within 0.012 posterior SD.
The nested-Laplace grid is screened by default (gcol33/tulpa#925).
control$prune now defaults to TRUE on tulpa_nested_laplace_joint()
and tulpa_nested_laplace(); prune = FALSE solves every cell as before.
A grid placed around its mode crosses axes the posterior does not fill (a
pinned dispersion axis dozens of posterior SDs per node, the far cells of a
placed tensor), and solving each of those in full was most of a large fit's
cost. The SPDE fitter stays opt-in: its CCD mode-find reads the
log-marginal through the same screened entry, where a screen would drop
stencil points. The per-door defaults live in one table, .NL_SCREEN$prune.
A dropped cell carries log_marginal = -Inf, zero weight and no retained
mode or precision, so the evidence (logLik()) and every summary are read
over the cells the fit solved; the bound below caps what they leave out.
control$local_ccd solves every cell unless the screen is asked for.
Local CCD refinement reads a finite-difference stencil off each refined
cell's axis neighbours, and a dropped neighbour has no log-marginal to read,
so a fit that asks for it (directly, or through k_refine = "ccd") keeps
the full grid unless control$prune = TRUE is set.
tulpa_joint_grid_batch() refuses a screened grid on the first fit it
captures. The fused solve walks one grid for the whole batch and cannot
carry a screen that prunes each species to its own kept set, so a batch of
fits on the engine defaults stops with tulpa_grid_batch_ineligible before
a second fit is captured; pass control$prune = FALSE to batch.
prune_mask indexes the grid the fit reports. It is carried cell by
cell through the refinement and consistency passes like the other per-cell
fields, and a cell those passes add reads FALSE; it used to keep the
length of the base tensor.
The prune is held to a bound on the mass it may have dropped, and the
driver repairs it rather than solving the grid again. After the full
pass, each dropped cell is lifted by the worst screening error measured on a
kept cell, and the share of the posterior they may then carry is
prune_dropped_mass_bound. Past 1% (.NL_SCREEN$gate_mass, mirrored by
CHEAP_SCREEN_GATE_MASS) the driver solves the dropped cells that
contribute most to the bound, largest first, and reads it again
(prune_n_repaired). The safety gate reads the same bound and falls back to
the full grid only if it still fails. It replaces two triggers that read the
ranking: a disagreement between the screen's and the full solve's top cell,
and a kept posterior collapsed onto a cell the screen mis-read. On a grid
placed around its mode the neighbouring cells sit within a few nats of each
other, so which one the screen ranked first carries no information about
the dropped cells. A screen off by the same amount on every cell also ranks
them as the full pass does, however large that amount is. On the full 25 km
Calluna occu_cover fit both triggers fired on both screened grids, and
each fell back to the full grid (90 and 200 cells).
Every movable axis is placed from the one outer mode, with one refit, and
the copy scale is among them. The field SD, a copy block's field SD, the
per-arm dispersions and now the copy scale alpha are the families of one
placement pass (.joint_place_axes(), R/joint_placement_pass.R). Each
axis that fires, by railing or by a grid that does not resolve its own
posterior, is laid from the mode the placement mode-find reached, and the
grid is refit once with all of them. Before, each family refit the grid on
its own, and the copy scale was never placed, so the 25 km Calluna fit's
final grid crossed a placed field SD with the declared 10-level copy axis and
4-node dispersion axis: 200 cells. A copy scale's continuum is laid at the
mode and its "no coupling" atom at 0 kept. An axis whose posterior sits on
the atom is neither placed nor reported as railed: outer_grid_railed_axes
no longer lists such an axis as alpha:lower, since the atom is a model and
not an endpoint. The placement pilot thins a wrapper's auto_grid() copy
axis like any other default axis (the Calluna pilot goes from 3 x 10 x 3 to
3 x 4 x 3 cells).
A placement refit reads the mode it was laid from. Unless the placed
grid rails, the refit attaches the detecting fit's outer mode
(outer_mode_carried = TRUE) instead of running the mode-find again. An
axis placed at the SD floor, which a sharp full-data posterior always is,
reads as unresolved on its refit, so before this each such refit cost a
second mode-find: about fifty full grid solves on the Calluna fit.
The var-of-means consistency pass lays an axis's points at the found
mode. When the fit was placed from a mode, a collapsed axis's first round
lays five points one measured SD apart around it (.hyper_propose_at_mode()),
which resolves a Gaussian marginal (quadrature ESS 3.48 against the floor of
3) in one kernel call. Bisection without a mode can take up to four rounds and
may not resolve it within its eight-node cap at all: the Calluna fit's pinned
dispersion axis sat at 301 posterior SDs per node and ended its eight nodes
at ESS 1.0. A mode whose SD is past the placement ceiling is not used; the
pass bisects as before.
The points laid at the mode run out to where the box rule reads the
tails. A node's measure is the box to the midpoints with its neighbours in
its row, so the outermost of the five points owned half the gap to the next
declared node, and the reported interval and SD, which spread a node's mass
over its box, carried that mass across the gap. On the Calluna fit's pinned
dispersion axis the gap is 36 posterior SDs, and the point at 2 SDs ended
the pass holding 62% of the axis's weight with the mean 1.2 SDs above the
mode. The points now form a ladder one SD apart that runs, on each side of
the row it is laid in, to the first step whose box adds at most
.NL_DIAG$at_mode_gap_var (1%) to the axis's variance by reading into the
gap beyond it: 5 and 6 SDs on that axis. The bound is on the variance, since
0.7% of the posterior read across a 100-SD box still quadrupled the reported
SD. The ladder is sized by the posterior and does not count against
axis_refine_nodes. Each later round reads the solved row and gives a node
whose flat box misreads more than that bound into a gap, against the
log-linear run to its neighbour, a neighbour one SD out: that takes up a mode
the mode-find left a fraction of an SD off (1.1 SDs on the Calluna
dispersion axis).
The consistency pass lays its slices through the row of the outer mode.
A slice re-tiles one fibre, and the pass laid it through the heaviest base
cell. Where the placement could not move an axis and it sits tens of SDs
from its mode, that cell is the best of the other axes GIVEN the far level:
on the Calluna fit the dispersion slice went through the copy scale's best
level at a dispersion node 36 SDs off, 4 copy-scale SDs from the joint mode,
and every summary was read in that row. A slice is now laid through the base
row nearest the outer mode, in each axis's own SDs
(.hyper_consistency_anchor()). Axes with a found mode are taken farthest
from their nearest node first, and an axis whose row sits more than
sqrt(2 log(1 / consistency_row_mass)) mode-SDs from the mode, where under
the mode's Gaussian it can hold under 1% of what the row through the mode does,
is held and listed in var_of_means_consistency_info$held (on the Calluna
fit the copy scale, which took eight nodes carrying nothing).
A parabola is not read across another axis's slice. Once the posterior
sits in one axis's slice, the other axes' modal levels hold that slice while
their neighbouring levels integrate it on the base nodes alone, and the
three-point parabola at the modal node compares unlike sums. On the Calluna
fit it reported the field SD at 0.0155, a ninth of the 0.143 the placement
mode-find measured, beside an interval nine times its width. The parabola now
declines there (theta_sd_stencil_declined = "cross_slice",
.nl_axis_cross_slice()), and the SD is the within-cell one the interval is
read from.
n_threads. That covers the rounds of
the var-of-means consistency pass and of the adaptive refinement, the skew
probe, and the pilot every parallel grid solves before its team starts. A
fit run the recommended way (n_threads = 1, the cores on
n_threads_outer) therefore solved all of these on one thread while the
rest of the pool waited. On the full 25 km Calluna occu_cover fit the
consistency rounds ran at 17 to 35 min each with 1 of 24 threads busy.
The joint driver now resolves the fit's grant once (.tulpa_thread_grant():
the larger of n_threads and the realised n_threads_outer, under the
n_threads_scatter cap) and hands it to the kernel as the inner count. A cell
solved inside the outer team still takes its share of the pool; a cell solved
alone takes the whole grant (nl_cell_solve_threads()), including the
coupled-cell scatter's team. On a coupled occu_cover fixture (3,600 sites,
8 threads) a consistency round goes from 1.34 s to 0.38 s at the same
log-marginal. A fit is reproducible bit for bit at a given pair of
n_threads and n_threads_outer.recenter_pilot and the cheap screen both on, the
pilot's detecting grid solved every kept cell in full, though all it is read
for is where the posterior sits. That means the rail test, the heaviest cell
that seeds the placement mode-find, and the weights. Its cells are then
discarded whichever way the placement goes. The detecting grid now returns
the screen's own log-marginal for every cell, the pilot cell included (which
keeps its full-solve mode as the placement's warm start), and skips the full
pass (screen_only in run_nested_laplace_grid()). The placement probes,
the refinement solves and the reported grid remain full solves, and the
prune safety gate stands down on a screened read. outer_grid_pilot$screened
records whether the pilot was read this way. On the full 25 km Calluna fit
this targets the ~1.5 h 90-cell detecting pass; that saving has not been
measured yet.occu_cover fit, 45
inner solves (about 2.5 h) followed a round whose log posterior no longer
changed. The search now stops when the full Newton step predicted by the
round's stencil is shorter than stop_sd = 0.05 outer posterior SDs (in
.CCD_PLACEMENT). That stencil was taken at the final point, so it is used
as the design's curvature, and the same reuse applies when a line search
accepts no step. On an exact quadratic over four axes the placement costs 74
inner solves instead of 114.std::map lookup in
SparseHessianBuilder::add(), repeated on every Newton iteration of every
outer cell. A CoupledScatterPlan (src/coupled_scatter_plan.h) now
resolves those offsets once per fit, and the coupled rows share the
per-observation row body (scatter_row_indexed) with the uncoupled path.
The batched multi-species driver reads the same plan in place of its
per-cell lookups. The coupled cell loop is split into a fixed number of
chunks set by the cell count alone (coupled_chunk_count()): up to 64,
with one chunk per 64 cells. An entry owned by a single chunk is written in
place, in single-pass order. An entry shared by several chunks is summed
per chunk and then added in chunk order. The chunks run as OpenMP tasks
inside the outer grid, so grid threads that have run out of cells take a
straggler cell's chunks, and on a team of n_threads outside it (the
serial pilot and screen). The coupled log-likelihood is summed over the
same partition. At fewer than 128 cells (one chunk) a fit is bit-identical
to 0.6.7. Above that, the shared entries' summation order changes the last
bits (2e-12 in the log-marginal on a 600-cell fixture), and a fit is now
identical at every inner thread count. TULPA_GRID_WORKSTEAL and
TULPA_COUPLING_FORCE_PARALLEL are removed: the partition they switched no
longer depends on thread count. The scatter index cache is keyed on the
shared pattern and built once, instead of being copied per outer thread.no_usable_curvature whenever
another axis still carried spread. On the full 25 km Calluna occu_cover fit
the dense tensor kept the field SD on its 3.0 ceiling and the default
(piloted, screened) fit left it on the edge of a recentred axis at 0.21, the
two about 189,000 WAIC units apart. The placement now finds the mode: the
damped Newton mode-find the CCD integrator already uses, started at the
heaviest cell, run over every outer axis through full inner solves at the
fit's own tolerances, and laid out as mode +/- 2.5 sd from the curvature it
reaches (.joint_placement_mode()). The mode it used is on the fit
(outer_mode_u, outer_mode_cov_u, outer_mode_status,
outer_mode_rounds, outer_mode_evals) and the search reports on the
progress channel as [outer placement].tulpa_nested_laplace_joint() warns with each default axis whose mode stayed
on a grid edge, its side and the reason (outer_grid_axis_declined, which now
records attempts_exhausted on an axis a pass moved and that still rails). A
caller's own pin stays silent and on the fit.prune_screen_decrement..nl_diag("axis_sd_ess")) and the modal parabola declines, the reported SD
was the weighted read's zero beside an interval spanning the node's whole
cell (sd = 0 with a 95% interval of [2.03, 4.52]). It is now the SD of
the same within-cell read (theta_sd_source = "within_cell"). The two
multi-block moment paths now choose each axis's SD by its resolution as every
other nested path does, which they never had.d = 1 search stepped in k
dimensions. The CCD never runs at one axis; the placement does.M_cholmod_* stub looks its target up with
R_GetCCallable() on its first call, and that lookup pushes and pops the R
protect stack. In a fresh session the first call to some stubs came from
OpenMP workers (per-cell solvers in cpp_joint_inner_vcov_blocks and the
outer-grid loops), so concurrent lookups raced R_PPStackTop and a .Call
returned with the stack off by one: R then printed a cascade of
Warning: stack imbalance in '.Call', ... lines. The DLL's init now runs
one 1 x 1 factor-and-solve through every stub tulpa uses, so no worker
resolves one. It showed up intermittently, in about one run in four of
tulpaObs test-occu-joint-predict.R with a package build running
alongside, because it needs two first-lookups to overlap.
test-cholmod-stub-resolve.R fails if a stub used in src/ is missing
from the resolver.wbic() returns Watanabe's (2013) widely applicable Bayesian information
criterion: the expected negative log-likelihood under the posterior tempered
at 1 / log(n), an estimate of the negative log evidence that keeps its
asymptotics in singular models (mixtures, latent factors, a variance
component at zero) where BIC's parameter count does not. Fit with
mode = "smc" and control = list(wbic = TRUE): the SMC path then runs in
two legs, from its reference to p(theta) L(theta)^(1 / log n), where the
population is recorded as $tempered_draws, and on to the posterior, so
$draws are posterior draws as before. The log-likelihood is the same
normalized pointwise density waic() and loo() read. The default method
takes a pointwise log-likelihood matrix at tempered draws, and
compare_models(criterion = "wbic") ranks fits by it.tulpa_smc_fit C entry takes a bridge_end argument and returns the
population at that likelihood power in SMCShimResult::tempered_particles,
so a model package can compute WBIC from its own likelihood.
TULPA_ABI_VERSION is 46; packages calling the SMC entry rebuild.wbic = TRUE the SMC path is the one-leg path it was, and a fit
with a given seed is unchanged.vi_patience iterations: every coordinate's mean gradient within a
Bonferroni-corrected z of zero in units of its own standard error. On the
same fit the SD ratios are 1.12 / 0.90 (full-rank), 1.10 / 1.11
(low-rank) and 0.88 / 0.85 (mean-field). VI runs take more iterations.control$max_grid_cells stays as an opt-in ceiling and now defaults to none.
The warning, the refusal under a set ceiling and the post-solve timing
warning all name the axes that produced the count, dispersion axes included
(It crosses b1 (...) x phi_pos 4.).control$integration = "grid_adaptive" falls back to the dense tensor when
the grid is small or the posterior mass spreads over most of it, and that
fallback left no trace: the fit reported integration = "grid" as if the
tensor had been asked for, and an error raised inside the flood was folded
into the same quiet fallback (gcol33/tulpa#915). The reason is now recorded as
integration_declined ("adaptive_small_grid", "adaptive_diffuse_seed",
"adaptive_flood_cap", "adaptive_kept_cap", "adaptive_seed_failed",
"adaptive_degenerate_lattice" or "adaptive_error"), the field a declined
CCD already fills; verbose = TRUE announces adaptive lattice declined ->
tensor grid; and an error in the flood is raised as a warning carrying its
message.control$max_grid_cells on a joint multi-block fit counted the latent cells
before an active phi_grid was crossed on top, so a 2500-cell ceiling
admitted 2500 latent cells times 4 dispersion values, 10000 inner solves. The
cap now counts the crossed grid, the number of inner solves the fit pays, and
the refusal names the dispersion axes beside the block axes (... x phi_pos
4). This covers the dense tensor that a declined "grid_adaptive" falls
back to. A fit that set the cap for the latent grid alone raises it by the
dispersion node count.ziformula reached model.matrix(), which
stopped with "contrasts can be applied only to factors with 2 or more levels"
and no variable name. A model fitted over many subsets (one species at a time)
could not say which column collapsed. The error now names the variable, the
formula part it sits in and the level it is stuck at.control$max_grid_cells was refused with
"Reduce per-block grid sizes", which does not help a fit that set no grid:
its size came from the default axes (gcol33/tulpa#913). Two copied ICAR
blocks at the default field-SD axis (5 nodes) and copy axis (atom + 9 slab
nodes since 0.5.1) cross to 50 x 50 = 2500 cells, past the 2048 default,
whatever the data; before 0.5.1 the same model was 30 x 30 = 900. The error
now lists each block's rows and per-axis levels, and on the joint driver
offers control$integration = "grid_adaptive" (the same lattice, solved where
the posterior mass sits) ahead of "ccd", naming either one as declined once
the fit has declined it. The cap and the grid are unchanged.integration = "grid_adaptive" stays on the lattice at four or more axescontrol$integration = "grid_adaptive" ran the CCD, the design "auto"
picks at that axis count, and the fit reported integration = "ccd" with no
decline recorded (gcol33/tulpa#914). The CCD now engages only under "auto"
and "ccd"; "grid_adaptive" integrates on the mass-carrying subset of the
tensor lattice at any axis count, and declines to the dense tensor as before.weights = no longer switches mode = "auto" / "structured" from
integrating the random-effect scale to conditioning on sigma_re = 1.
tulpa_re_cov_nested() gains a weights argument and weighted
random-effect models route there; the RE SD now equals the fit on rows
duplicated by weight (gcol33/tulpa#874). mode = "auto" also honours
control$re_cov, which it previously ignored.mode = "smc" targets the posterior: it bridges from its starting Gaussian
to the posterior with an exact tempering path and mutates with HMC scaled
to the particle cloud. Random-effect SDs no longer collapse 3-10x
(gaussian (1 | g): 0.24 -> 0.77 against HMC 0.81) (gcol33/tulpa#876).mode = "ess" updates every log-SD and correlation parameter of a slope
term; the slope SD of (1 + x | g) was pinned at 1. Any parameter no block
updates is now routed to a Metropolis step (gcol33/tulpa#877).mode = "ess" mixes on hierarchical models. Each sweep now draws the level
the intercept shares with the group intercepts exactly (as re_cov_gibbs
does), moves each covariate together with the between-group part the group
effects absorb, and slice-samples each random-effect SD holding the latent
values fixed, holding the effects fixed, and along a partially non-centered
path between the two. Warmup now adapts by default
(control$ess_adapt_during_warmup = TRUE), including an ellipse fitted to
each Gaussian block's warmup draws. On y ~ x + (1 | g), 30 groups x 8,
four default-length chains: gaussian intercept split-R-hat 1.15 -> 1.00 and
bulk ESS 20 -> 3310 (random-effect log-SD 69 -> 3722); poisson intercept
R-hat 1.75 -> 1.00, bulk ESS 6 -> 2633 (log-SD 14 -> 1188). Across 32
chains on 8 datasets every chain now passes the fit-time check; the lowest
per-chain bulk ESS was 152 of 1000. Estimates match mode = "hmc". The RE
block's slice ellipse is now its N(0, 1) non-centered prior rather than
N(0, sigma_re^2), and ess_joint_proposal_sd is the initial slice width of
the scale moves (gcol33/tulpa#877).re_cov_gibbs draws the shift shared by a fixed effect and its matching
random-effect column exactly each sweep; bulk ESS went from 1-11 to over
1300 of 2000 (gcol33/tulpa#875). mala() gains mass_matrix, and
tulpa(mode = "mala") starts at the posterior mode scaled by the Laplace
covariance; bulk ESS went from 1-9 to about 190. auto no longer routes
weighted RE models to mala (gcol33/tulpa#878).tulpa() now warns when a sampler chain fails a convergence floor
(split-R-hat above 1.05 or bulk ESS below 100), records $convergence, and
print() reports it. It flagged ess on every hierarchical fit until the
mixing fix above (gcol33/tulpa#875, gcol33/tulpa#878).tulpa_simulate(theta = fit) simulates at the fit's own random effects
(the draws ranef() / posterior_predict() use, matched to data by
group level) instead of setting every random effect to zero. A level the
fit never saw, or a fit with no random-effect posterior, is an error;
theta$u blocks are validated for shape (gcol33/tulpa#891).rownames(adjacency), whole-number labels are read as node numbers, and
other character labels on an adjacency without rownames are refused; they
were attached in sorted order, scrambling the field (8x8 lattice
field-truth correlation 0.16 -> 0.98). Numeric-looking time labels are
ordered numerically, and non-numeric character time labels are refused
(gcol33/tulpa#900).laplace / nested_laplace are conditioned on the field's
sum-to-zero constraint; the intercept SE was 3-4x the exact posterior SD
(gcol33/tulpa#901).spatial_bym2() scales each connected component separately and gives
islands unit structured variance (Freni-Sterrantino et al. 2018) instead
of an infinite scale factor that produced NA or frozen fits. The samplers
(mode = "hmc" and the other ModelData modes) fit the same per-component
model: each component's scale rides on the structured field's prior as a
per-node precision multiplier, the construction the Laplace,
nested-Laplace and Gibbs kernels use. check_adjacency() reports
connected components (gcol33/tulpa#902). This adds a field to the exported
ModelData struct (TULPA_ABI_VERSION 45), so packages linking to tulpa
must be rebuilt.spatial_svc(approx = "nngp") accepts rows that share coordinates. Each
term's field lives on the distinct locations and every row reads the value
at its own site, as spatial_gp() already did; duplicated coordinates were
refused before. svc() still reports one row per observation. SVCData
gains obs_to_loc (same ABI bump).temporal() on a nested-Laplace fit with a spatial and a temporal field
returns the temporal field, located through the fit's block layout
(gcol33/tulpa#903), and reports the Gaussian mixture over grid cells
including each cell's conditional variance (conditioned on the sum-to-zero
constraint for an intrinsic field); 95% intervals covered the truth 57% of
the time (gcol33/tulpa#904). Its draws are taken from each cell's joint
Gaussian, which the fit now retains as $grid_field_cov, so they carry the
correlation between time points and not only each point's marginal. A
quantity derived from them now has the right spread; on an AR1 fit the SD
of a year-on-year change read 1.34 against an exact 0.18, and now reads
0.19. A field whose covariance would exceed 4 million values over the grid
keeps only the marginals.temporal() for a non-centered temporal_gp() sampler fit returns the
field rather than its whitened coordinates, with the time values in the
time column (gcol33/tulpa#905).scale_coords = TRUE scales all coordinates by one common factor, so
isotropic kernels stay isotropic, and spatial_range() / svc() report
ranges and coordinates in the original units. Temporal-GP and GP-TVC
lengthscale supports are placed relative to the spread of the times and
reported in time units (gcol33/tulpa#907).spatial_range() / temporal_corr() no longer report a (1 | g) SD as
sigma_temporal, report both hyperparameters of BYM2 and proper-CAR
sampler fits, keep proper-CAR and AR1 correlation intervals inside their
support, and name duplicate field rows by block (gcol33/tulpa#906). They
and temporal() read fit_st_nested() fits, whose field columns are now
also right with a random-intercept term (gcol33/tulpa#910).VarCorr() labels a re_cov_gibbs covariance "sampled" and reports a
Residual row for gaussian fits; confint() validates level / parm
and reads sampler hyperparameters off the draws; durbin_watson() accepts
a fit and a time index (gcol33/tulpa#881, gcol33/tulpa#899).validate_mode() matches the backend a fit ran as well as its tier and
accepts a set of modes (gcol33/tulpa#879).rubins_pool() standardizes the pooled skewness by the mixture variance,
pools every submodel any draw carries, and refuses mismatched draws
(gcol33/tulpa#888). geweke_test() is computed per chain and validates its
windows (gcol33/tulpa#895). tulpa_psis() keeps one weight per draw and
reports pareto_k = Inf for an infinite ratio (gcol33/tulpa#892).tulpa_gaussian() runs the AD-gradient NUTS sampler (2000 iterations in
about 0.1 s where one iteration took ~1 s) and reports a real
accept_rate (gcol33/tulpa#897). tulpa_tgmrf() no longer prints the inner
progress line per evaluation (gcol33/tulpa#890).tulpa_profile() times more than the sparse joint solver: the
single-response Laplace solve behind tulpa_laplace() /
tulpa(mode = "laplace") and every nested-Laplace cell, the dense joint
solve, the nested-Laplace outer grid (outer_grid_cell, one call per
cell) and the NUTS sampler (nuts_warmup / nuts_sampling per iteration,
gradient per evaluation). New leaf phases hessian_extract and
inner_diagnostics time the final pass. The enclosing phases take a share
of NA, so the leaf shares still sum to one. Timing is off outside
tulpa_profile(), so an ordinary fit no longer takes a lock per phase. It
still warns when an expression reaches no timed phase
(gcol33/tulpa#887).sprintf() print again
(gcol33/tulpa#885).tulpa() reads a logical binomial response as 0/1, refuses other
non-numeric responses and NA grouping values by name, drops unused
grouping levels, and drops aliased columns of a rank-deficient design with
a warning, as lm() does. y ~ 0 + (1 | g) is refused by name on the
backends that need a fixed effect (gcol33/tulpa#880, gcol33/tulpa#881).tulpa_ep() decodes cbind(successes, failures) and refuses
y > n_trials (gcol33/tulpa#882); tulpa_laplace() accepts re_list
idx = 0 with return_hessian = TRUE (gcol33/tulpa#883).tulpa_nested_laplace() keeps a lone supplied axis of a two-axis prior and
crosses it with the other's default; fully supplied axes are paired cells
and must share a length; an AR1 rho_grid outside (-1, 1) is refused
(gcol33/tulpa#884).tulpa_eb(), tulpa_laplace(), agq_fit(),
tulpa_ordinal(), tulpa_multinomial(), mala()) refuse family typos,
non-integer n_quad, NA rows, y > n_trials and non-positive scales by
name (gcol33/tulpa#886). A non-normal beta_prior and re_prior keys the
backend does not read are errors (gcol33/tulpa#893).spatial_spde() checks its PC anchors at construction and a non-positive
LKJ eta is refused wherever it is set (gcol33/tulpa#894).sbc_*(), prior_*(), ccd_grid(), tulpa_grid_axis(),
tulpa_check_control() and hyper_axis_spec() refuse invalid input by
name (gcol33/tulpa#896).fit_spde() / tulpa_nuts_spde() refuse a response whose length differs
from the projector's rows (gcol33/tulpa#908). spatial_spde() accepts
repeated sites; spatial and temporal constructors refuse NA / Inf /
constant coordinates, NA or negative adjacency entries and missing times
(gcol33/tulpa#909).mode = "auto" no longer errors on control$n_threads when it picks a
sampler (gcol33/tulpa#911). Inline field errors list every accepted mode,
and mode = "gibbs" with an SPDE field is refused instead of silently
running NUTS (gcol33/tulpa#912).matvec, the sparse CSR product, the SE kernel product and the NNGP
gradient loops) size their OpenMP team by the arithmetic they split rather
than by the row count, so a small model no longer opens a full-width
parallel region per leapfrog step to share a few hundred multiply-adds.
A 40 x 2 product took 1.47 ms per call on a four-thread team on a loaded
machine and now takes 0.06 microseconds on one thread; a 20000 x 10 product
keeps its four threads (gcol33/tulpa#897).hyperprior = "flat" in tulpa_eb() / tulpa_re_cov_nested() /
mode = "eb" is documented as the REML-type estimate it is (matches
lmer(REML = TRUE)), not ML (gcol33/tulpa#889).tulpa_priors() / priors_default() describe what they feed and the
defaults tulpa() actually applies; return-value docs fixed; runnable
examples added to 49 exports (gcol33/tulpa#898).family = Gamma() fits the inverse link it namesstats::family() object's link decided the <base>_<link> suffix by
comparison with stats' canonical link rather than with the engine's own
default, and the two differ for gamma and inverse gaussian: stats' Gamma()
defaults to the inverse link, tulpa's bare "gamma" means log, so
family = Gamma() fitted a log-link model with no warning. The object is
now read against the engine's defaults, so Gamma() fits gamma_inverse
(the coefficients glm(, Gamma()) reports) and Gamma("log") the bare
gamma. inverse.gaussian() resolves to inverse_gaussian (it was
"Unknown family") with stats' 1/mu^2 link carried as 1mu2, and a link
the base family does not admit is refused naming the object
(gcol33/tulpa#873).warmup >= n_iter is refused before any sampler runsmala and imh_laplace checked the run length. hmc, ess,
sghmc, sgld, tulpa_nuts_beta() and tulpa_gibbs() returned a fit with
no draws at n_iter == warmup (coef() read zeros under NA names,
summary() failed), mode = "gibbs" returned NaN, and warmup > n_iter
aborted inside C++ with vector::_M_default_append. One check,
.check_run_length(), now runs in every sampling fitter once its defaults
are resolved, and states what n_iter counts there. A bare n_iter below
a 1000-iteration default warmup (tulpa_gibbs(control = list(n_iter =
500))) is caught by the same check (gcol33/tulpa#872).?tulpa's control section now documents that n_iter is the TOTAL
iteration count (warmup included) on hmc, ess, sghmc, sgld,
gibbs, mala and imh_laplace, and the KEPT count (warmup on top) on
re_cov_gibbs and mclmc, where warmup >= n_iter stays valid.mode = "laplace" reads its control knobslaplace branches built their tulpa_laplace()
call with no control field, so max_iter, tol and n_threads passed
tulpa()'s validation and were then dropped (a max_iter = 1 fit reported
the fully converged answer), and knobs only another backend reads
(n_iter, adapt_delta, adaptive_grid, checkpoint, ...) were accepted
in silence. The three numerical knobs are now forwarded, and anything else
errors naming the knobs mode = "laplace" reads, as the sampler backends do
since gcol33/tulpa#770 (gcol33/tulpa#870).mode = "laplace", the conditional path with no grid, so its
control$checkpoint was one of those dropped knobs and no file was ever
written. It now runs mode = "structured", the nested-Laplace fit it
describes.mode = "laplace" and mode = "eb" fits the linear-predictor draws
behind posterior_predict(), pp_check(), bayes_R2() and the WAIC / LOO
/ CPO / DIC criteria took the fixed effects from N(coef, vcov) -- a
marginal spread that includes the intercept's aliasing with the random
effects -- and held the random effects at their mode, dropping the negative
correlation that cancels it. Every draw shifted the whole linear predictor:
on a 10-group Poisson GLMM, p_waic read 28.7 against HMC's 11.3, and
compare_models() ranked the Laplace fit about 17 elpd below the same model
fitted by HMC. Each draw's random effects now come from their Gaussian
conditional given its fixed effects under the joint Laplace precision of
[beta | b] (kept on the fit as H_latent), which is the joint Gaussian
draw; the same model reads p_waic 11.0 / elpd -313.3 against HMC's
11.3 / -313.6, and the EB fit's p_loo 11.0 where it read 19.8
(gcol33/tulpa#871). The spatial conditional-Laplace path carries no joint
precision and still holds its random effects and field at the mode; the
documentation now says that over-disperses the linear predictor.mode = "laplace", "eb" and "re_cov_nested" (and the Laplace / EB
source fit behind warm_start) passed the raw beta_prior to their fitter.
When none was supplied, that meant tulpa_laplace()'s own N(0, 100^2)
rather than the documented engine default prior_normal(0, 2.5), and
fit$beta_prior read NULL. Under separation the slope ran to 131 where
the default prior holds it near 9. These backends now fit under the same
resolved prior as every other backend, and report it
(gcol33/tulpa#869, a regression of gcol33/tulpa#408). The spatial Laplace
path carries no fixed-effect prior channel and still reports NULL.sigma_re, and VarCorr() / print()
recovered it by re-evaluating the call in the wrong frame: a sigma_re
passed through a variable fell back to 1 or picked up an unrelated global
of the same name. The front door now stores the resolved value as
fit$sigma_re_conditioned, default included, and VarCorr() reads it
(gcol33/tulpa#868).mode = "hmc", "vi", "ess", ...) samples its
random effects non-centered and stores the standardized z, where the
effects are b = sigma * L z. ranef() summarized z as though it were b,
off by 1 / sigma on a scalar term and mixing a correlated term's slope
effect with its intercept, while pointwise_loglik() and
posterior_predict() on the same fit used b. ranef() now applies the
same map the density does, and the reconstructed effects reproduce the
engine's own linear predictor exactly (gcol33/tulpa#866).(1 + x | g) term, VarCorr() ignored
the sampled correlation Cholesky factor (L_re) and returned a diagonal
covariance, reporting a correlation of exactly zero. It now reports the
posterior mean of the per-draw correlation L L', with the posterior-mean
SDs on the diagonal as before (gcol33/tulpa#867).hyper_share reached attr(diagnostics(fit), "summary") but not
diagnostic_summary(), which already carries the other outer-grid facts
(quad_ess, outer_regime, interval_read). diagnostic_summary() now
reports hyper_share_min / hyper_share_max and prints the range. Both
doors read one helper, so they cannot drift. It carries no threshold and
never moves status: a marginal that rides almost entirely on the
within-cell Gaussian may be perfectly calibrated, which is what the new
coverage gate measures (gcol33/tulpa#862).test-spatial-beta-coverage.R
closes that gap, measuring confint() against a simulated truth on an ICAR
lattice in a well-identified and a weakly identified regime: 0.981 pooled
over 160 trials. Tier 3, so it runs on the release gate
(gcol33/tulpa#862).within + between under the law of total
variance over the outer grid, and between is the part that exists BECAUSE
the hyperparameter was integrated rather than held at a point. Nothing
reported that split, so a fit whose marginal barely moves when its
hyperparameter posterior does looked identical to one that integrated
properly. diagnostics() now carries hyper_share per coefficient with
hyper_share_min / hyper_share_max beside it, prints the range, and the
summary row a consumer writes to a reliability CSV carries the two bounds.
On the occu_cover fixture behind gcol33/tulpa#862 the affected coefficient
reads 3.4% -- its interval is almost entirely the within-cell Gaussian --
while another coefficient on the same fit reads 25%, so the number
discriminates rather than restating the grid size. Reported without a
threshold: what counts as too little depends on the model. A fit that
retained no per-cell pieces reports the share as ABSENT rather than zero,
since zero would be a claim about the fit rather than about the record
(gcol33/tulpa#862).unreliable reports the marginal read off
that same Gaussian, and the engine ships two corrections for exactly that
condition -- control$subspace_debias (exact Metropolis on the flagged
coordinates, the rest carried at their Gaussian conditional) and
control$cila (the whole inner Gaussian reweighted by the exact joint
density). Both are off by default and nothing in the reporting said they
existed, so the measured misfit and its remedy never met. diagnostics()
now carries the remedy as inner_debias_note, prints it with the other
notes, and diagnostic_summary() lists it last in recommendations, after
the bands that motivate it rather than before them. It is silent on a fit
that already ran a correction, which would otherwise read as though the
correction had not taken (gcol33/tulpa#862).control$subspace_debias$closure grows the corrected set over strongly
coupled precision-graph neighbours, because a coordinate coupled to a member
of the set and left out of it is carried linearly -- precisely the error the
correction exists to remove. It needs the joint precision, and a grid or
joint fit computed one per cell and then discarded it, so the closure
declined with closure_needs_joint_hessian on every such fit.
.nl_modal_joint_precision() assembles the MODAL cell's copy from the CSC
scratch the inner solves already return, and both drivers read that one
assembler, so they cannot disagree about which cell the coupling came from.
One cell, not n_grid: the closure is a selection decision made once. The
retention rides the closure request, so a fit that did not ask for one
retains nothing extra; the joint driver turns store_Q on for it and hands
back only the assembled precision, not the per-cell scratch. Measured on a
sparse ICAR fit banding both inner scores unreliable, the corrected set
grows 1 -> 2 -> 3 -> 5 -> 9 as the coupling threshold loosens where it
previously could not grow at all (gcol33/tulpa#862).attr(diagnostics(fit), "summary") -- the one-row frame a consumer writes to
a reliability CSV -- carried outer_regime without the note saying what a
collapse means and which axis to widen, so the reading was reachable only
from the printed form and each consumer re-derived an
ess_grid >= 2 && max_weight <= 0.9 rule of its own. It now carries
outer_regime_note beside the code. The classification itself was already
correct and independent of the opt-in k-hat: the quadrature numbers come from
the stored weights unconditionally and the regime is attached before the
diagnose_k branch, so a collapse is reported whether or not pareto_k ran.
Nothing asserted that end to end, which is how an occu_cover batch shipped
ess_grid = 1.0000067 of 27 cells at max_weight = 0.99999 with its
consumer reporting nothing; test-outer-grid-collapse-reporting.R now pins
all four doors (gcol33/tulpa#863).tulpa_hyper_draws() no longer falls back to
independent draws on a coarse grid under a strong correlation. The quadratic
that sets its target is fitted to the grid's log density with each cell
weighted by its mass, and the off-ridge cells' masses (about 1e-130 at
3 x 3 nodes, log correlation -0.97) fell below the fit's rank test, so it
read as rank-deficient and returned the identity. Those weights are now
floored where the rank test sees them, a factor 1e-10 of the modal cell,
which leaves every grid with all cells above that fitted exactly as before.
On two log scales at log correlation -0.97 and 3 x 3 nodes, the log
product's sd goes from 0.580 to 0.182 against an exact 0.122. At -0.99 on
4 x 4 and 5 x 5 nodes it goes from 0.398 / 0.302 to 0.109 / 0.086 against
0.071 (gcol33/tulpa#860).tulpa_hyper_draws() (and so the "theta" attribute of
tulpa_posterior_draws()) ties the within-cell draws of the outer axes
together through a Gaussian copula. Each axis's marginal is unchanged: every
cell-conditional is still reached by its own inverse CDF, so the draws
reproduce theta_ci_lo / theta_median / theta_ci_hi exactly as before.
The joint follows the posterior's orientation. The copula correlation is
solved so the draws' total correlation between two axes equals that of the
local Gaussian fitted to the grid's log density on each axis's unconstrained
coordinate. Independent jitter added the full box variance of both axes in
the direction a strongly correlated posterior pins down, so a product of
anticorrelated scales came out too wide: sigma_pos_field on the
cover-glaser HP760 J = 10 fit, where log sigma and log alpha correlate at
-0.936 (gcol33/tulpa#859). On an analytic two-scale posterior with log
correlation -0.9 (exact log-product sd 0.224) and a 7 x 7 box grid, the
draws' log correlation goes from -0.852 to -0.900 and the log product's sd
from 0.281 to 0.232. On a 5 x 5 grid it goes from 0.309 to 0.239, where the
node weights' own correlation would give 0.072. The copula matrix is returned
as attr(., "within_cell_copula").tulpa_hyper_draws() (and so the "theta" attribute of
tulpa_posterior_draws()) give every cell the box the quadrature measured
it with. A slice point re-tiles only the row it was placed in
(.hyper_refined_log_quad()); every other row keeps the declared levels'
cells. Both reads used to lay one partition over every distinct value the
axis carried, so a base cell in an untouched row was drawn on the narrow box
between the slice points beside its level while it held its whole base
box's mass. On the cover-glaser HP760 J = 10 reference fit the phi_pos
draws' interquartile range held 0.453 of the fit's own measure instead of
0.5; the J = 3 fit, with no consistency slices on that axis, held 0.497
(#858). An axis no pass refined is read exactly as before.tulpa_grid_axis("copy_alpha"))
is the atom at 0 plus nine log-spaced nodes on [0.1, 3], a node ratio of
1.53, the resolution the other outer axes of a copy fit run at. With five
(ratio 2.34) a posterior over alpha sat on two adjacent nodes; an SBC on the
cover-glaser HP760 fixture (occu_cover beta arm, n_sim = 300) had 9/11 and
7/11 parameters inside the family-wise band at J = 3 / 10 with five nodes
and 11/11 at both with nine (#858).1 / axis_sd_ess of the
continuum's mass, re-reads the axis ESS, and repeats until the axis reaches
the floor or has taken .nl_diag("axis_refine_nodes") (8) new nodes. It used
to place four points once at mu * exp(+-{0.7, 1.5} * sd / mu), sd the
modal parabola, which on a marginal sitting on two nodes reads the grid
spacing and put all four points inside the one gap between them. A copy
axis's point mass is excluded from the ESS the pass reads and from the gaps
it bisects, since no continuum node changes the share it holds.
var_of_means_consistency_info reports ess_after beside ess_before and
no longer carries sd_laplace. A marginal peaking on an outermost level is
left to the placement rescues: the span truncates it, and bisecting towards
that level only shrinks its box.sigma axis read per axis (.nl_axis_railed(): the collapsed grid's
dominant cell on its node, or its own marginal maximal at an endpoint), the
trigger the registry and dispersion rescues already used. Gated on the whole
grid's collapsed_edge regime, a sigma carrying 0.91 of its mass on its
ceiling went unplaced once the consistency pass had resolved a dispersion
axis beside it, because the grid ESS then read 5.9 against a collapse
threshold of 2. The spatiotemporal rescue (fit_st_nested()) reads
the same predicate over its free axes; on the ten rescue calls the ST test
fixtures make, every placement decision is unchanged (7 recentred, 2
grid_not_collapsed, 1 grid_knobs_overridden).<tulpa/re_term_prior.h> carries the prior of one random-effect term
and the effect values it implies: a half-Cauchy on each SD, the
partial-correlation Cholesky factor with its LKJ(2) density for a correlated
term, and the non-centered z. re_term_scales(), re_term_group_effect()
and re_term_log_prior_add() are what the engine's random-effect block now
evaluates, and what a consumer package calls for a random effect it carries
among its own extra parameters (tulpaObs's random effects over visit rows),
so the two put one density on one parameter vector. The scalar math they are
written in (<tulpa/ad_scalar_math.h>: safe_exp, safe_log, safe_sqrt,
safe_max, inv_logit, safe_tanh, log_prior_half_cauchy, for double,
forward-mode and arena reverse-mode scalars) and the Cholesky map
(<tulpa/lkj_chol.h>: build_L_from_raw(), lkj_cholesky_log_density_add())
moved out of src/ with them.alpha = 0 is not a point of the copy scale's continuum;
it is a model in its own right, carrying a declared prior probability that
the grid's node count cannot move. Three readers already split the axis
there, all on the one rule .hyper_axis_scale() states -- a zero level on a
log-scale axis: the measure that integrates the axis weighs the level at its
declared probability and lays its cell widths over the continuum in log, the
support the fit reports covers the continuum alone, and the hyperprior folds
against the same predicate. The reporting geometry was the fourth reader and
the only one that did not split, because it took the axis's support from the
outer Pareto-k PROPOSAL's coordinate -- identity, which a proposal needs in
order to reach zero at all, and which says the axis is unbounded.So the level was given a cell, half a node spacing wide, reaching below the
axis's own support, and two things followed from that one geometry: the
level's posterior mass was reproduced by no draw, and draws left the support.
On a 25-cell ICAR joint fit whose alpha nodes are 0, 0.25, 0.5 and 1, with
0.460 of the posterior on the level, tulpa_hyper_draws() put 0.000 of
50000 draws on it and 22.8% of them below zero, down to -0.125; the fit's own
reported 2.5% bound for the amplitude was -0.111. Each now reads 0.461, 0,
and 0.
The split is one declaration, so every read takes it: the reported interval
(theta_ci_lo / theta_median / theta_ci_hi) under either within-cell
construction, the moment-matched interval a CCD design is summarized with,
tulpa_hyper_draws() and the "theta" attribute of
tulpa_posterior_draws(), and the per-axis hyperparameter summaries. The
copy axis's reported resolution (outer_grid_cell_width /
outer_grid_h_over_sd) is now its continuum's, measured in the coordinate
the continuum is laid out in; the level owns no cell and so contributes no
width. Downstream, a consumer deriving a field SD as alpha * sigma no
longer receives a negative one.
A copy fit whose alpha grid carries no zero node also moves, by the same
correction and in the third decimal: its outermost cells are now closed half
a node step out in log, the coordinate the axis is spaced and integrated in,
rather than in the value. Nothing inside the outermost nodes changes, and no
axis that was already reported on its own coordinate moves at all.
cpp_nested_laplace_joint_multi_batch() documents each
element of its result as what cpp_nested_laplace_joint_multi() returns for
that species on the same grid, and since 0.4.12 it returned neither
fitted_eta nor fitted_eta_var: the single-species entry attached the
predictor and asked the inner solve for its variance, and the batch loop did
neither. The solve itself was never wrong -- log-marginal, mode shape and
modes agreed species for species -- so what was missing was the result, which
is what a grid-mixture predictive read draws from. Both are attached per
species now, under the same one-arm gate the single entry uses, and the
switch control$fitted_var sets is read in ONE place
(.nl_want_fitted_var()) so a fused species cannot integrate under a
different answer from the fit it reproduces.auto_grid(x, place = FALSE) declares a default the engine must integrate
as written (gcol33/tulpaObs#361). Provenance -- whose nodes these are --
and placement policy -- whether the auto-placement pass may move them -- are
different questions, and the marker answered both at once: a wrapper package
that had MEASURED its own default as the one to integrate could only get that
by leaving the mark off, which says the user pinned the axis. The engine then
reported exactly that back to a user who had pinned nothing. Such an axis is
now treated as a pin everywhere the engine acts on it (the pass leaves it, no
curvature is computed for it, and refinement densifies within its span rather
than following the posterior past the end nodes) and reported as
"default_axis_pinned", with ?tulpa_nested_laplace_joint's own lever
advice pointing at the package's argument rather than at a pin the reader did
not write. auto_grid_place() reads the declaration back, for a wrapper
rebuilding a value that as.numeric() stripped.
outer_grid_recenter_declined reduces over the passes that wrote it. A
joint fit's field SD and its per-arm dispersion are placed by two passes over
one grid, and the slot held whichever spoke last. An axis-scoped reason --
one that is a property of a single axis's declaration rather than of the
fit's grid, its curvature or a control knob -- no longer stands as the fit's
answer while a pass that had a movable axis has one. The measured case:
occu_cover() fits whose sigma axis was defaulted and simply needed no
placement came back "axis_pinned", from the dispersion axis beside it. The
per-axis outer_grid_axis_declined record is unchanged and still answers per
axis.
A dispersion axis's refinement rung follows its provenance again. The
joint front door strips the auto_grid() markers before the first fit, so
nothing downstream sees an attributed numeric -- and phi_<arm>'s refinement
mode was read off that stripped grid, which made every dispersion axis read
as stated whoever wrote it. It now travels as the front door's own record, so
a marked axis is refined as placed ("extend") and a pinned or held one as
stated ("densify").
pointwise_loglik()'s ndraws is documented. Its tulpa_fit method
takes the argument and the shared Rd page did not describe it, which is an
R CMD check WARNING ("Undocumented arguments in Rd file
'criteria_doors.Rd'") raised only when the reference manual is built.
ENGINEERING_HISTORY.md is excluded from the build. The closed-issue
write-ups are a repository file, not package material, and at the top level
of a tarball they are a non-standard file.
0.2 * sd(time) (gcol33/tulpa#851). It started at the midpoint of the
(0.01, 10) support, and both doors standardize their time values, so that
is a lengthscale five times the data's own spread: every pair of instants is
correlated to within rounding, the dense T x T covariance is numerically
rank-one, and its Cholesky jitter binds at the starting point. The runtime
gradient check then deviated on the LENGTHSCALE by an amount ordered by
kernel smoothness, and temporal_gp(parameterization = "centered") fell
back to numerical gradients for the whole run. Measured deviation against
the starting ratio lengthscale / sd(time): Matern 5/2 clean to 1.0 then
9.3e-04 at 2.0 and 2.4e-03 at 5.0; Gaussian clean to 0.25 then 1.2e-03 at
0.5, 6.5e-03 at 2.0 and 8.3e-03 at 5.0. That ladder is a floor binding, not
a wrong derivative -- the jitter is protecting a genuinely singular
factorization and is untouched.Only the centered parameterization evaluates that log-determinant in the
target, which is why the non-centered default never showed it: there the
Cholesky enters only through f = L z. Both arms of both doors now pass the
check. init_gp_lengthscales() is the one starter, shared with the GP TVC
gcol33/tulpa#847 added it for.
spatial_rsr(spatial_gp(...)) is fitted (gcol33/tulpa#848). RSR exists
for spatially smooth covariates -- a climate surface, elevation -- and those
are exactly where a continuous field is the natural prior and an areal one a
discretisation of convenience, so the mitigation was available only on the
field shape that needs it least. cpp_pg_binomial_gibbs_gp_rsr() is the
counterpart of the areal kernel: the same projection, applied each sweep to
an NNGP field whose prior precision is assembled from its Vecchia factors at
the current range rather than read off a fixed adjacency.Measured on a confounded fixture (150 points, binomial, 5 seeds): the
reported field is orthogonal to the restricted design to 6.2e-15 of its own
scale, where an unrestricted fit's is 14.3.
One projector, over the observation -> field map.
.rsr_unit_projection() was already almost map-agnostic; it now takes that
map by name, so the areal route passes spatial_idx and the continuous one
obs_to_loc and the two differ in nothing else. tulpa() no longer re-types
every $rsr spec as areal: the modifier keeps the field's own $type for
validation -- which is what made a continuous RSR spec demand a
spatial(col) term it has no use for -- and only the backend selector sees
"rsr". pg_nngp_scale_update() is split into the sparse field sweep and
the hyperparameter update, so the projected kernel reuses the second without
the first, which its dense conditional replaces.
What the restriction buys is the MARGINAL association, not a less biased
version of the slope conditional on the field. ?spatial_rsr now says so,
with the measurement (5 seeds, conditional slope 1.0, marginal 1.71:
restricted fit 0.06 from the marginal value and 0.74 from the conditional,
unrestricted 0.10 and 0.63) and with the coverage caveat the method carries:
Hanks et al. (2015) measured poorer coverage under RSR than under the
unrestricted spatial model in the geostatistical setting, and Khan and Calder
(2022) report the same areally. Those references are on the page now; the
capability is offered with them rather than without.
temporal_tvc(structure = "gp") is fitted (gcol33/tulpa#847). rw1,
rw2 and ar1 read the time index as a position on a grid, so a ten-fold
gap and a unit gap are penalised alike; gp is the continuous-time
structure, a Gaussian process over the distinct time VALUES. It is not a
second spelling of [temporal_gp()]: that is a GP over time entering the
linear predictor additively (eta_i += f(t_i)), where this one IS the
coefficient (eta_i += x_i w(t_i)).Measured on 24 irregularly-spaced instants (exponential gaps, poisson,
n = 288) against a smooth true trajectory: the GP recovers it at
cor 0.999 / RMSE 0.042, the rw1 that assumes the spacing away at
0.989 / 0.114. temporal_tvc() gains cov / nu / period /
scale_coords, and sigma_prior_U / sigma_prior_alpha are read on this
structure too -- the same anchor pair, whether the field samples a
log-precision or a log-variance.
One kernel, read by both doors. The Ornstein-Uhlenbeck chain that makes
the exponential kernel O(T) was written inline in
compute_temporal_prior(); it is now ou_chain() / ou_log_density() /
ou_forward() in src/temporal_gp_kernel.h, which both densities call, so
the transform's scale and the centered conditional variance stay one number
and the correlation floor binds at one place. The kernel's own validation is
likewise one read_temporal_gp_kernel() at the C++ spec boundary and one
.check_temporal_gp_kernel() in R.
TULPA_ABI_VERSION 43 -> 44. TVCData gained the distinct time_values
and the kernel fields the GP structure reads, ModelData the lengthscale
bounds, and ParamLayout the per-coefficient log_sigma2_tvc_gp /
logit_phi_tvc_gp spans. A model package linking against tulpa rebuilds.
(0.01, 10) and
the time values are standardized, so the start is a lengthscale five times
the data's own spread: every pair of instants is correlated to within
rounding, the dense T x T covariance is numerically rank-one, and its
Cholesky jitter binds. The runtime gradient check then deviates on the
LENGTHSCALE, ordered by kernel smoothness -- Matern 5/2 clean up to a ratio
of 1.0 then 9.3e-04 at 2.0; Gaussian clean up to 0.25 then 1.2e-03 at 0.5
and 8.3e-03 at 5.0. That ladder is a floor binding, not a wrong derivative.A GP TVC now starts at 0.2 * sd(time) (init_tvc_gp_lengthscale()), and
all four kernels pass the check. temporal_gp(parameterization =
"centered") starts at the midpoint still and reproduces those deviations to
the last digit, which is what attributes them; it is gcol33/tulpa#851.
temporal_corr() reported a GP lengthscale as the logit's position in the
unit interval rather than as the lengthscale it maps to, and now reports the
lengthscale. It also reports a GP TVC's own amplitude and lengthscale, which
it carried no pattern for.tulpa_nested_laplace_joint() fit stored no linear predictor, so the
dispersion axis it had paid to integrate reached no predictive read
(gcol33/tulpa#850). gcol33/tulpa#825 taught posterior_predict(),
bayes_R2() and the pointwise log-likelihood to sample at the cell's own
phi_<arm> value, and the two halves never met: a fit that CARRIES such an
axis comes from the joint driver, which left modes behind and no per-cell
eta, while a fit that REACHES the grid-mixture draw comes from the tulpa()
front door, where phi_grid is not an argument. The contract was implemented
and untestable on a real fit.cpp_nested_laplace_joint_multi() now attaches fitted_eta by replaying the
driver's own eta accumulator at each cell's stored mode, so the arm's offset,
every block kind and each cell's block scaling are the ones the inner solve
used. A cell the cheap screen pruned reads NA rather than a predictor
evaluated at the zero row the grid runner left it -- the predicate is the
cell's own log_marginal, not the mode row's contents. fitted_eta_var, the
within-cell spread a replicate is drawn with, follows control$fitted_var
(new on this front door, TRUE by default) rather than being pinned off, and
is requested only at one arm, where the mean it belongs to is stored.
The per-cell side data now travels through refinement from one table.
Three sites listed which fields a refinement pass carries cell for cell --
slice the initial result, slice a pass's result, glue the merged list back --
and a field added to one and not the others indexes the grid the fit had
BEFORE refinement. They read one .JOINT_CELL_FIELDS table. Local-CCD
refinement rebuilds the grid from its own node solves and carries neither, so
it drops the predictor rather than leaving it to be read against cells that
are no longer the fit's.
A one-arm joint fit resolves its response, design and offset off the arm.
A joint fit carries a family, a dispersion, a response and a design PER ARM,
and $y is a per-arm list, so fitted(), residuals(), the criteria layer
and .resolve_obs() were reading flat fields a joint fit does not have --
fitted() failed on the missing $model_matrix and the criteria read
refused for a missing family. One predicate, .tulpa_single_arm(), answers
it for all of them, and the resolved design carries the fit's own
$fixed_names, which is that arm's column naming by construction.
test-posterior-predict-phi-axis.R's third claim runs on a real fit instead
of a hand-built grid standing in for one: the dispersion contrast is the same
fit with one grid column pinned, so the per-cell predictor, the within-cell
variance, the weights and the cells each replicate is drawn in are identical
and only the dispersion moves.
tulpa(y ~ x + (1 | g), spatial = spatial_spde(...)) conditioned the
random-effect SD instead of integrating it, under mode = "auto",
"structured" and an explicit "nested_laplace" alike
(gcol33/tulpa#817). All three resolve to the spde backend, whose outer
grid is (range, sigma) and whose sigma_re is a scalar it conditions on,
so on the path whose whole point is integrating the hyperparameters the RE
was the one variance component never estimated -- held at the default of 1,
a number the data never produced.The generic nested driver already carries an spde block type, so the field
now goes there as a block beside the RE term's own iid block and both SDs
sit on one outer grid. theta_grid gains a third axis, and
tulpa_hyper_draws() reports a posterior for it. Measured over
sd_re in {0.2, 0.8, 2.0} x 8 seeds, the integrated posterior mean comes out
0.268 / 1.175 / 2.727 -- monotone in the truth and the right order of
magnitude, against the flat 1 the conditioned path used whatever the data
said. The fixed-effect slope barely moves (mean |error| 0.063 vs 0.076 at
sd_re = 0.2, 0.072 vs 0.092 at 2.0, inside a seed-to-seed spread of ~0.07),
so this is about the variance component being estimated at all.
sigma_re still conditions, as the degenerate one-node case of
the same axis rather than a second route. An SPDE field with no RE term
still reaches fit_spde() -- the redirect is narrowed, not removed. Integer
nu only: fractional nu is the operator-based rational construction
fit_spde() owns, and it refuses a random effect regardless.temporal_tvc(structure = "gp") was documented and accepted, and refused
by every mode (gcol33/tulpa#814): the sampler entry has no GP branch and
the Laplace-family modes carry no TVC field at all. Worse, the Laplace-mode
refusal recommended mode = "exact" without checking the structure, so for
"gp" it pointed at a mode that also refuses. structure now takes the
three the TVC block's density has a branch for, "gp" is named explicitly
with a pointer to the wiring issue (gcol33/tulpa#847), and the wrong-mode
message only recommends mode = "exact" when that mode would in fact take
the spec -- one predicate, .tvc_structure_or_stop(), asked by both.
spatial_rsr(spatial_gp(...)), the ?spatial_rsr example, could not be
fitted by any mode (gcol33/tulpa#815). The projection is applied by one
kernel, cpp_pg_binomial_gibbs_rsr(), which conditions on an areal
neighbour list; tulpa() re-typed every $rsr spec as areal, then demanded
a spatial(col) term and failed on the missing adjacency with "non-numeric
matrix extent" -- a message about neither the spec nor the argument. A
non-areal spec is now refused at construction, where the argument that
caused it is still in hand, and the @param and the opening example use an
areal field. The capability for continuous fields is gcol33/tulpa#848.
(range, sigma) cell, and each assembly ran
.spde_rational_roots() from scratch on that cell's own spectrum interval.
Measured on the issue's fixture: 1102 calls for a single n = 40 fit, 1049
of them at distinct spectrum ratios, at ~0.55 s apiece -- 606 s of the
616 s the fit took. Caching cannot help (the ratios are distinct), and
sharing one wide interval across the grid is not free either: at order 2 the
minimax error of x^{-beta} is 7.2e-07 on [0.5, 1] but 1.6e-02 on
[0.01, 1], and the fit visits ratios down to 2.9e-06.The search is now compiled (src/brasil.h), 49x faster per call, with
R/brasil.R kept as the reference oracle and pinned to it by
test-brasil-cpp-oracle.R. n = 120, the issue's headline case: nu = 1.5
647 s -> 45.9 s, nu = 0.5 600 s -> 51.0 s; n = 40 nu = 1.5 321 s ->
32.2 s. The fitted coefficients are unchanged (n = 40, nu = 1.5: 0.313,
0.630, the values the issue logged), and the two implementations agree on
the roots to 1.5e-07 relative across orders 1-4, five beta, four spectrum
ratios.
vi_max_iter was almost never what ended a VI run (gcol33/tulpa#821).
The loop stopped on a patience rule that counted an iteration as "no
improvement" when its ELBO gain was below tol_rel_elbo * |ELBO|. An ELBO
carries an arbitrary additive constant -- the normalizing terms of the
likelihood and the prior -- so |ELBO| is not the scale of anything, and at
ELBO ~ -742 the 1% default made any gain under 7.4 nats per iteration
count as no progress. Fifty of those stopped the run. Testing each iteration
separately also let the rule discard up to patience times its own
tolerance.The rule now compares the ELBO gain ACROSS the patience window -- the mean
of its second half minus the mean of its first -- against
tol_rel_elbo * (best - worst), the span the run has covered. Both sides
are differences, so the additive constant cancels;
test-vi-stopping-rule.R pins that the stop does not move when a whole run
is shifted by up to 1e5 or rescaled by 100x.
control knobs -- vi_tol_grad,
vi_tol_rel_elbo, vi_patience -- and a VI fit reports vi_iterations and
converged_reason ("patience", "gradient_norm" or "max_iter"), so a
run that stopped far short of its budget can be told from one that used it.
converged = TRUE alone could not express the difference.test-spde-re.R:68 errored on fit$draws[, "beta[2]"] and had been
erroring since the fixed-effect names became the resolved ones
(gcol33/tulpa#846). The assertion that subscript guards is the slope
recovery for the exact SPDE sampler, so it had not run at all. Read by the
name the fit carries it passes: mean(draws[, "x"]) = 0.9467 against a
truth of 0.8, well inside the 0.3 gate.Added test-family-registry-compiled.R, covering every entry of
family_names() (gcol33/tulpa#824). mala() / pathfinder() /
imh_laplace() build their target from the R registry through
build_glmm_logpost(), and glmm_weights() rebuilds the Laplace Hessian
from it for the post-fit marginal SEs on GP / NNGP / HSGP / SPDE fits, while
every other backend runs the compiled kernels. The existing cross-checks
reached 7 non-canonical families; plain gaussian, poisson, binomial,
neg_binomial_2, gamma and beta had none. The log-likelihood and score agree
across all 14 to machine precision. The curvature did not:
glmm_weights() returned the expected weight where the engine's Laplace
Hessian carries the observed one. The compiled neg_binomial_2 branch
deliberately returns (y + phi) phi mu / (mu + phi)^2 -- and
working_weight_is_observed() names it -- while the registry's y-free
weight is the expected mu phi / (mu + phi). At phi = 1.7, eta = 1.3,
y = 9 those are 2.315 and 1.162. Every vcov() / confint() that
reconstructs H = D'WD + Q on a neg_binomial_2 spatial fit used a different
curvature from the fit. glmm_weights() now takes y and routes through
the compiled predicate, so the weight it returns is the one the fit used for
every family; y is required exactly where it changes the answer.
.family_obs_weight() substituted the expected weight for six
families. It answered from the registry's obs_weight closure where one
was registered and fell back to the y-free weight where none was -- exact
only where the response enters the log-likelihood linearly in eta. For
beta, gamma, inverse_gaussian, beta_binomial, tweedie and t it is a
different function and can carry the opposite sign: at y = 12,
eta = -0.9 the beta_binomial observed curvature is -0.125 against an
expected weight of +0.658. beta_binomial is in .ZI_FAMILIES, so the
zero-inflation mixture in R/family_zi.R differentiated through the wrong
one. It is now one compiled path (cpp_family_obs_weight()) for every
family, with the three registry closures kept as the R oracle it is pinned
against.
tulpa_posterior_draws() sampled hyperparameters as bare grid-node atoms.nl_mixture_draw()'s draw_cell());
the hyperparameter half had nothing, so every consumer read
theta_grid[attr(draws, "cells"), col] and a fit integrating a 5-node
sigma axis produced draws taking 5 distinct values on it whatever the
sample size. That is the object gcol33/tulpa#337 described -- "the read
collapses each box to its midpoint" -- which #337 and #353 fixed for the
reported interval (.nl_summary_quantile()) and nowhere else.tulpa_hyper_draws(fit, cells) is the new door, and
tulpa_posterior_draws() attaches it as attr(., "theta") so a consumer
reading the attribute is correct by default. It is not a new
construction. Each within-cell read the engine ships already defines a
density on an axis and each is a per-cell mixture with a closed-form
component, so what a draw samples is that component conditional on its own
cell: Uniform(e_c, e_{c+1}) over the cell's own box under box_uniform,
and an equal mixture of Uniform(v_{c-1}, v_c) / Uniform(v_c, v_{c+1})
under chord. Summed over cells each reproduces the read it came from
exactly, so the draws' own quantiles reproduce the fit's theta_ci_lo /
theta_median / theta_ci_hi -- measured to 5e-3 at 4e5 draws on a two-axis
fixture and to 2e-3 on an end-to-end BYM2 fit, against numbers produced by
CDF inversion in a separate code path. Neither conditional reads the
weights, so the continuization cannot disagree with the sampler that chose
the cell.
The attachment runs with the RNG restored: the latent draws are formed
before it and are bit-for-bit what they were, and an existing set.seed()
script's downstream numbers do not shift because an extra attribute appeared.
An explicit tulpa_hyper_draws() call consumes the stream, as a draw should.
An axis whose support does not admit the box read falls back to chord and
one that reaches no CDF at all (a CCD moment rule) keeps its node value, both
recorded per axis in attr(., "within_cell") /
attr(., "within_cell_declined") from the vocabulary
.nl_summary_quantile_read() already uses.
log_marginal is the analytic
log-likelihood plus prior -- so the outer read is the only thing under test
and the exact posterior CDF is available as a reference arm -- 400
prior-predictive replicates at 5 nodes: the node-atom PIT takes 185 distinct
values against 400 for the exact posterior, and its uniformity test is
rejected outright (KS p = 0.0000 against 0.92 exact); the continuized PIT
takes 357 and reads 0.34. The rate at which the atom pins the PIT to exactly
0 or 1 EQUALS the rate at which the truth falls outside the node range
(0.015 / 0.035 / 0.022 over three configurations, to the digit); the box
read's outer half-cell takes it to 0.005 / 0.000 / 0.007 against the exact
arm's 0.005 / 0.002 / 0.000. At 15 nodes the atom is nearly harmless
(KS p 0.092), which is the sense in which this is a coarse-grid defect.A truth-correlated PIT is not on its own evidence of this defect, which
#823's own reading assumed. The same measurement with an exact-posterior arm
gives Spearman cor(truth, pit) of +0.490 exact against +0.472 atom at
n = 5, +0.281 against +0.286 at n = 20, and -0.029 against -0.036 at
n = 200: the correlation is a property of how informative the data are about
the hyperparameter, and a weakly identified one produces it under the EXACT
posterior too. What the atom provably breaks is uniformity. Any re-read of
the occu_cover SBC evidence needs a reference arm before a quartile-bucketed
rate is scored against a flat 0.10.
spatial_gp() (nngp / hsgp) with a (1 | g) term could not be fitted by
mode = "auto" or mode = "nested_laplace" (gcol33/tulpa#794): the RE term
turns the fit into a multi-block prior, and the multi-block converter behind
nested_laplace (.nl_block_spec_for_cpp()) has no gp / nngp / hsgp
arm, so both errored ("Block type '...' is not supported in multi-block
priors") after building the full outer grid. auto_select_mode() now
refuses nested_laplace for this combination and falls back to the exact
ModelData NUTS sampler (hmc), which threads the field directly; an
explicit mode = "nested_laplace" now refuses up front with a message
naming mode = "laplace" / mode = "exact" instead of reaching the deep
C++ error. An areal field (icar / bym2 / car_proper) + RE is unaffected.posterior_predict(), simulate(), WAIC and LOO omitted the field
on conditional-Laplace spatial fits (icar / car / car_proper / bym2),
Polya-Gamma Gibbs fits, and inline spatial() / temporal() field fits
(gcol33/tulpa#795): the in-sample linear predictor fell back to a
fixed-effects-only assembly, so replicates and pointwise log-likelihoods
understated the model by its whole spatial/temporal structure --
compare_models() ranked the same model fitted two ways 153 elpd units
apart. .tulpa_eta_draws() now reads the field's posterior-mean
contribution off a conditional-Laplace fit's own mode (the same
(n_field, Z_field) design each .marginal_H_beta_* areal helper already
builds), the field's own sampled phi_spatial[k] / theta_spatial[k]
draws off a Gibbs fit, and the field's weighted-mode posterior mean
(computed at fit time) off an inline joint field fit.spatial() / temporal() field fit's $y was the joint
driver's per-arm response LIST, not the response vector (gcol33/tulpa#796):
every diagnostic reading fit$y (residuals(), pit_residuals(),
moran_i(), test_dispersion(), check_model(), pp_check(), ...) errored
with a coercion failure, and fit$n_trials was always NULL. Both
constructors now overwrite $y / $n_trials / $phi from the bundle the
fit was actually built from, in one shared finalizer the two share.An inline correlated (MCAR) field's fixed-effect standard errors read
~75 where the uncorrelated (||) fit of the same data read 0.08, and
coef() moved between identical calls (gcol33/tulpa#797): the sum-to-zero
constraint treated a p-field MCAR block's p * n_units latent as ONE
group instead of p groups of n_units, leaving one of the block's p
near-null constant directions unconstrained. .joint_constraint_cols() now
emits one group per field for an mcar block.
mode = "auto" still selected backends that refused the very call that
selected them (gcol33/tulpa#769): a ziformula sent the default MALA arm,
the latent/spatial nested arm, and a temporal-only model (no arm existed for
one at all) to a backend without a zero-inflation channel, and the slope
redirect applied after selection ignored the call's features entirely (a
(1 + x | g) model with weights picked MALA, then was force-redirected to
re_cov_gibbs, which also refuses weights). Every arm in auto_select_mode()
now consults .auto_backend_ok() and falls back to the exact ModelData NUTS
sampler (hmc) where it carries the call and nested Laplace does not (areal
icar/bym2/car_proper and continuous gp/nngp/hsgp fields; not SPDE or plain
intrinsic car, which hmc does not thread); the slope redirect re-checks
both RE-covariance integrators before redirecting and refuses, naming the
conflicting feature, when neither carries the call. The spatial_multiscale
(approx = "hsgp") advice text no longer recommends a nested-Laplace mode,
which refuses multi-scale fields entirely.
control knob only some OTHER backend reads passed tulpa()'s union
check and was then silently ignored (gcol33/tulpa#770): control$seed on
mode = "mala" / "imh_laplace" / "pathfinder" did nothing (two runs with
the same seed gave different draws), mala's n_chains / thin were
dropped, and mode = "agq" accepted n_iter / seed / n_chains / thin
with no effect (agq_fit() is a marginal-likelihood maximizer with no
sampler knobs at all). mala(), imh_laplace() and pathfinder() now take
a scoped seed argument; tulpa() re-validates control against the
selected backend's own key set once the backend is fixed, so an unread knob
now errors naming what is actually allowed instead of doing nothing.mode = "re_cov_gibbs" on a model with no random-effect term failed with
the internal C++ message `blocks` must hold at least one block
instead of the plain refusal re_cov_nested already gives for the same call
(gcol33/tulpa#771). tulpa_re_cov_gibbs() now refuses before the pilot solve.plot_diagnostics() errored argument is of length zero on any fit with
no posterior draws (mode = "laplace", fit_spde(), tulpa_eb(), ...)
instead of the same draws-provenance message plot_rhat() / plot_ess()
give (gcol33/tulpa#772). It now gates on .tulpa_is_chain() like its
siblings.offset() was silently dropped by agq, ep and gibbs, including when
mode = "auto" routed a binomial areal model to gibbs (gcol33/tulpa#764).
agq_fit() now threads the offset into the shared compiled GLMM oracle
(cpp_glmm_oracle_make() takes an offset argument); ep_fit() /
tulpa_ep() fold it into each site's cavity before matching tilted moments
and un-fold it from the site update, so the fitted posterior is over the
fixed effects alone. The Polya-Gamma kernels behind gibbs carry no offset
term at all, so it is refused there (as re_cov_gibbs already did) rather
than dropped, and carries_offset is now declared on agq (TRUE), ep
(TRUE) and gibbs (FALSE) so mode = "auto" steers away from gibbs
under an offset instead of silently fitting the wrong model.mode = "ep" returned wrong coefficients with converged = TRUE on
inverse_gaussian, neg_binomial_1 and large-count poisson (gcol33/tulpa#765).
The per-site tilted moments were Gauss-Hermite quadrature with the nodes
placed on the cavity; when the likelihood in eta is much narrower than the
cavity (a large count) or flat over part of it (inverse_gaussian, log link),
the fixed cavity-centred rule misplaces or misses the mass and a site's
variance can collapse to its numerical floor. .ep_tilted_moments() now
finds each site's own tilted mode and curvature and lays the quadrature
there instead (self-normalized importance reweighting against that Gaussian
proposal, exact for any placement -- the old cavity-centred rule is the
special case where the proposal equals the cavity). A site whose mode search
ran into its bracket, or whose variance still floors, is recorded
(n_site_not_converged, n_site_floored) and turns converged FALSE
rather than reporting a silently unresolved site as a success.family = "tweedie" ModelData sampler fit errored
family 'tweedie' needs phi2 ... (gcol33/tulpa#766): the fit records phi2
in $model_inputs, but the sampler-layout probe (cpp_tulpa_glmm_layout(),
behind coef(), summary(), confint(), vcov(), fitted(), print(),
and the warm-start layout) hardcoded it to NA_REAL. phi2 is now a real
argument threaded from mi$phi2 / args$phi2.fit_spde(mode = "nuts") /
tulpa_nuts_spde()) crashed with an internal vector::_M_default_append
when n_iter <= n_warmup, silently dropped control$warmup (the
tulpa()-wide spelling), and silently ignored control$n_chains
(gcol33/tulpa#767). tulpa_nuts_spde() now validates 0 <= n_warmup <
n_iter at entry; .control_subset()'s n_warmup/warmup alias is now
symmetric (it used to rewrite only n_warmup -> warmup, never the reverse,
so a fitter listing n_warmup alone silently lost a warmup argument); an
explicit n_chains != 1 on this single-chain route is now refused rather
than ignored.mode was replaced by a different backend with no override
recorded, in two places (gcol33/tulpa#768). An SPDE field under any
explicit Tier-1 backend other than the natural mode = "exact" / "hmc"
route (e.g. mode = "gibbs") is fitted by the SPDE NUTS engine without going
through .sel_redirect(), so the override was invisible on the fit and
never warned about; it now is, via the same sel$overridden / notify
mechanism every other backend redirect uses. An areal + temporal field
redirected every non-ModelData selection to nested_laplace with
notify = FALSE, which is right for a Tier-2 mode = "laplace" request
(nothing is lost) but wrong for an explicit Tier-1 request (gibbs, mala,
imh_laplace, ...), which loses its exact-sampler tier to a Tier-2
approximation; notify now follows whether the ORIGINAL selection was
Tier-1, not just whether the backend name changed.intercept + field
held, which is Gaussian because both priors are and does not involve the
likelihood. An intrinsic field (ICAR, RW1, RW2) keeps its level removed. Each
route records log_prob, and logLik() reads it.
Scored against an adaptive random-walk Metropolis chain on the same log
density: on a 6-group negative-binomial fixture the posterior median of
sigma_re was 0.535 against 0.635 and the intercept's 90% interval 0.321
wide against 1.023; it is now 0.634 and 1.025. On a 10-location binomial
NNGP fixture the median of sigma2 was 0.0044 against 0.448, and is now 0.400
(batch-means standard errors 0.026 and 0.020) with its 5% quantile at 0.008
against 0.036; the intercept, slope and range quantiles agree to within 0.041.
Scripts: dev_notes/issue761/.N(intercept + level; 0, sd^2). They left it out, so the level took a flat
prior. Each site now carries that factor through a running field sum
(PgInterceptLevel). On a 3x3 ICAR fixture with prior_beta_sd = 0.7, the
intercept median was 2.157 against the recorded model's 2.073 and tau was
1.208 against 1.322; it is now 2.0725 and 1.321.NA. The design is now
resolved once off the declared per-block grid (.hp_declare(),
.joint_multi_declared_axes()), the fold evaluates the density at any rows in
that design's coordinates, and the cell measure reads the same declared
design, so an adaptive subset of the tensor is measured on its own levels.
Points laid in the grid's log-Cholesky columns (a CCD design and its
mode-find, the outer k-hat's importance draws) read the log-Cholesky form of
the prior, which is the two-field density carried by the map's Jacobian;
a CCD-integrated MCAR fit, which had declined the prior on the design, now
carries it.st_parameterization = 1 on an HSGP-ST interaction ran the Knorr-Held
Type IV Kronecker branch (gcol33/tulpa#759), which reads neither HSGP
hyperparameter and treats basis weights as sites. The non-centred transform is
the Kronecker form's, and one predicate (st_non_centered()) now decides it
for the density and the mass override; the layout refuses the combination by
name. Reached only through a ModelData built by a linking package.An output-contract sweep over every tulpa() backend (34 fits) found the
accessors below returning shapes, names or values their generics do not
specify. Re-run on this release: 82 checks that failed now pass, none that
passed now fails, and every remaining refusal names the fit class and reason.
predict(),
simulate() and posterior_predict() rebuild the ziformula design, and
every response-scale read uses the structural-zero mixture: fitted() and
predict(type = "response") return (1 - pi) mu (previously fitted()
returned the count mean), simulation draws the zero indicator first, and the
pointwise log-likelihood behind cpo() / dic() / waic() / loo() /
kfold() is the mixture density. coef() / vcov() / summary() locate
the zi_ columns through the engine layout, which on a sampler fit with
random effects places them after the random-effect block; coef() there had
reported log_sigma_re as zi_(Intercept). Held-out kfold() scoring and
posterior_predict(newdata) now include the offset.y and N (gcol33/tulpa#753) and
gain fitted() (N x K class probabilities), per-class residuals(),
predict() on the link and response scales, posterior_predict() /
simulate() of class labels, and cpo() / dic() / waic() / loo() on
log P(observed class). The scalar-response checks (pit_residuals(),
test_dispersion(), check_model(), ...) refuse on them by name. Joint
nested-Laplace fits record their observations, so nobs() works; their
observation-level accessors refuse, since the fit keeps no per-arm linear
predictor.logLik() reports a value or names why not (gcol33/tulpa#750). ESS,
SGHMC, SGLD, MCLMC, SMC and VI return a per-draw log_prob evaluated by the
NUTS kernel's own log posterior (agreement 2.8e-14), and the RE-covariance
Gibbs sweep (the default route for a Poisson random-intercept model) and the
Polya-Gamma Gibbs routes record the joint density of the target they sample,
including the negative-binomial iid, NNGP and multiscale NNGP routes once
they sample a stated target (gcol33/tulpa#761, below).mode = "gibbs" fits carry their chain (gcol33/tulpa#751) as
draws / chain_id / n_chains with the column names the HMC sampler uses,
so diagnostics(), as_draws_df() and loo() read it. The kernels already
returned every retained sample; the wrapper had kept them as separate blocks.coef()
/ vcov() / summary() / tidy() / confint() agree exactly; the
class-specific coef / vcov methods are removed. coef() on an ordinal
fit now reads the same table as summary() and still omits the cutpoints.cpo() and dic() have tulpa_fit methods, and loo::waic() /
loo::loo() dispatch on a tulpa_fit (gcol33/tulpa#754), returning loo's
own objects. The two pointwise log-likelihood extractors behind kfold() and
compare_models() are one function; repeated calls on a nested fit now return
the same matrix.print(diagnostics()) describes the backend that produced the fit
(gcol33/tulpa#755); the outer-grid / inner-Laplace wording appears only on
nested fits.stats::confint() names them, "2.5 %"
(gcol33/tulpa#756). simulate() sets the "seed" attribute
(gcol33/tulpa#757). fit$means is named by parameter on mala,
imh_laplace and pathfinder fits (gcol33/tulpa#758).logit_rho_st while the density had
RW1 and RW2 forms only, so an AR1 margin contributed no quadratic form and the
RW2 rank. Type II, Type IV (centred and non-centred) and HSGP-ST now carry
R(rho) and its log-determinant, matching a dense reference to 1e-14, and the
Type-IV mass override emits the AR1 rows. The separable and non-separable GP
interaction types, for which the engine has no space-time kernel, are refused
when the layout is built instead of sampling two unread ranges. Reached only
through a ModelData built by a linking package.A cell evaluated as part of a batch lost the hyperprior on every axis the
batch held constant (gcol33/tulpa#760). The joint drivers read which axes
carry a density off the matrix they were evaluating, so the adaptive-grid and
var-of-means slices (which share their off-axis coordinates), and in the
multi-block driver the CCD, adaptive-grid, local-CCD and Pareto-k batches,
folded no density on those axes. Each slice's log_marginal was too high by
the missing log densities: 1.0 to 3.1 nats on a coupled occu_cover fit,
enough to put 0.52 of the posterior on the row the slices sit in against 0.21
on the unrefined grid. The integrated axes are now read once off the declared
grid (.hp_integrated_axes() of the initial grid,
.joint_multi_integrated_axes() of the per-block grids) and passed to every
evaluation. Fits whose refinement passes add no cells, and grids without
refinement, are unchanged. Present in every tag since v0.4.1, which first
carried the default hyperprior (#730).
The tulpa_nested_laplace() placement stencil and inner-skew probe read
the prior off the rows they evaluated (gcol33/tulpa#760). Both write their
rows onto the block (or hand them to the multi-block dispatch as an override),
so a stencil that moves a column the grid holds at one value folded that
column's density, and the one-row probe folded none. On a 40-region BYM2 fit
with sigma held at 0.8 the stencil's centre row read 1.02 nats below the
grid cell it sits on, and with rho held at 0.5 the probe row read 1.01 nats
above it; the stencil's sigma variance moved from 0.0408 to 0.0430. The
probe's marginal is not reported, so only the placement pass changes.
.nl_dispatch() now takes the block whose grid the fit declared, the
multi-block dispatch reads each block's axes off its declared grid (the k-hat
draws reach it as an override of the fit's own prior), and .hp_collect(),
.nl_block_log_hyperprior() and .st_log_hyperprior() take axes with no
default, so no caller infers them from the matrix it evaluates.
ComputationInfo. Eigen's LLT leaves that field unset until
compute() runs. The NNGP gradient kernels filled their per-thread
workspaces by copying one prototype; they now construct each in place through
tulpa_thread_workspaces() (src/omp_threads.h). The low-rank mass term's
factor is moved into the metric before it is factorized, and now starts from
a computed empty matrix. No result changes: the field was always written
before tulpa read it..nl_attach_axis_sd() summed
cell mass by level, so a refinement slice level held one row's narrow box
while the base level held every other row's wide one, and the axis marginal
carried a spike at the base node that both the weighted SD and the 3-point
parabola read. .nl_axis_row_projection() now spreads each cell's mass over
the box its row gives it and collects it on the level boxes, and the parabola
takes the density rather than the log mass. Against the same fit on a dense
33-node axis with refinement off, mean |log(sd / dense sd)| goes 1.188 ->
0.058 on a refined 9-node base and 0.552 -> 0.327 on a refined 5-node base.
The per-axis mean, the interval read and every unrefined fit are unchanged..hp_block_extent() now takes the
dimension as the rank of the centred coordinates; full-rank coordinates are
unchanged.phi = 0.0625, SD 0.25, against data drawn at SD 0.5), and every
figure read off it is re-measured (gcol33/tulpa#744).tulpa() states the outer prior, under one name at every doortulpa() takes hyperprior = c("proper", "flat") and forwards it to the
nested-Laplace path, the SPDE path, the random-slope RE-covariance redirect
and mode = "eb". Which backends read it is declared once in the backend
registry; "flat" on any other backend, or on the SPDE NUTS route, is an
error naming the ones that do. re_prior$hyperprior is refused with a
pointer to the argument (#745).tulpa_eb() and tulpa_re_cov_nested() name their default "proper",
as the nested doors do, instead of "pc_lkj". The prior itself (PC + LKJ) is
unchanged.fit_spde() and fit_st_nested() take the same argument. spatial_spde()
and spatial_spde_custom() record which of prior_range / prior_sigma the
caller stated (prior_stated), so under "flat" only a stated anchor keeps
its density.Open at this release: #744 (the outer-grid measurement fixture fits a residual variance four times smaller than it simulates).
tulpa_nested_laplace_joint() and tulpa_nested_laplace() take
hyperprior = c("proper", "flat"). "proper", the default, is the per-axis
set of #730 and leaves every fit bit-identical. "flat" folds no density of
the engine's own: each axis the call states no density for is integrated under
its cell measure alone, is named in log_hyperprior_declined as
"flat_hyperprior", and log_evidence declines with "improper_hyperprior".
A density the call states (prior_sigma, prior_alpha, prior_phi, a
block's rho_prior / prior_range / prior_sigma, the copy slab, a tgmrf
prior) applies under either. Both doors resolve it through one function,
.hp_axis_prior(). The joint door's prior_sigma / prior_alpha /
prior_phi documentation no longer describes NULL as flat, and the
tulpa() and tulpa_re_cov_nested() pages now give "pc_lkj" as their
default.control$local_ccd$rank chooses what local-CCD refinement ranks its
candidate cells by: "weight" (default, unchanged) or "mass_moved", the
weight times |exp(log_box_ratio) - 1|.dev_notes/issue327/bary327.R, dev_notes/issue331/coverage331.R,
dev_notes/issue333/plane333.R + analyse333.R,
dev_notes/issue_328/measure_fit_ranking.R (which now selects the ranking
through rank instead of replacing a package internal) and
dev_notes/issue730/re_cov_ccd_interval.R produce every figure the
barycentre, descriptor-plane, #331 recovery and RE-covariance tests quote,
each stamping the build, prior and within-cell read into its output. Under
hyperprior = "flat" the #331 table reproduces to the digit, and under flat
with the chord read so do the #327 four-arm tables and #328's 14 of 24.
The measurement fixtures state hyperprior = "flat"; the files record both
priors side by side.cpp_nested_laplace_joint_multi_batch() returned per-species
log-marginals and modes that a consumer assembled into a fit by hand, so the
object lacked the class, fields and post-processing of an ordinary
tulpa_nested_laplace_joint() fit, and draws, prediction and fitted() on it
failed. tulpa_joint_grid_batch(fits) now takes one zero-argument function
per species, each performing an ordinary fit. Each fit runs to its main outer
grid solve and hands the kernel request over; one fused solve answers every
request; each fit is then replayed with that solve served from the fused
result, so every species is built by the code that builds it alone. The
requests must agree on everything except arm responses, arm dispersions and
dispersion-axis nodes, and a setting the fused driver does not carry
(pruning, a warm start, outer threads, tiles, checkpoints, debias, CILA, the
inner skew probe) refuses the batch with a tulpa_grid_batch_ineligible
error rather than solving something else; a replay whose request differs
from its capture is refused the same way, and the random number state is put
back to where the batch was called before the error leaves. The fused entry
gained the single-species driver's hessian_pd_mode,
step_curvature_mode, force_sparse and fixed-effect block request, runs
the single-species final pass at each species' mode
(joint_newton_finalize_dense() / _sparse(), extracted from the two joint
Newton loops) and packs each species through nl_pack_grid_results(),
extracted from run_nested_laplace_grid(). tulpa_nl_joint_batch() now
returns, per species, the list cpp_nested_laplace_joint_multi() returns for
that species alone; the hyperprior fold and weights it used to attach are
built by the replayed fit. On the coupled occupancy fixture
a batched species equals its front-door fit bit for bit, on a fixed grid and
on the engine defaults (placement, refinement, outer k-hat): every field,
coef(), vcov(), confint() and posterior draws at one seed, with the
random number stream left where the sequential fits leave it.tulpa_nl_joint_batch() returned a plain softmax of the kernel's
log-marginals as each species' weights. No hyperprior was folded in and no
cell measure applied, so a fused batch fit was weighted against a different
posterior from the one the multi-block driver reports for the same species on
the same grid. On the two-arm occupancy fixture with sigma nodes
c(0.4, 0.6, 1.5) the two differed by up to 0.10 per cell. The C++ batch
entry now returns log-marginals only, and each species goes through
.joint_multi_attach_integration(), extracted from .joint_dispatch_multi()
so both drivers take one construction: the regularizing hyperprior folded
into log_marginal, then log_quad, axis_support and the weights. The
batch entry takes the multi-block driver's prior_sigma, prior_alpha,
prior_phi, copy_atom_mass and copy_slab.
test-nested-laplace-joint-batch-equivalence.R checks each species' weights
and log_quad against tulpa_nested_laplace_joint() fitted alone on the same
fixed grid, and compares the single-species oracle's log-marginal with the
fold added.recov_fit_joint_phi_crossed() is
meant to fit a gaussian arm at a quarter of its simulated residual variance,
the gcol33/tulpa#332 crossing, and its gate asserts the slope interval
narrows by sqrt(2) and loses coverage. Handing the door
.phi_to_kernel(cfg$phi) at RESID_VAR = 0.5 instead fits variance
sqrt(0.5), so the interval WIDENED by 0.5^(-1/4) = 1.19 and the gate read a
width ratio of 0.84 with equal coverage. It was red at fa704fb too and
only the slow tier runs it. The arm now hands the door
.phi_to_registry(cfg$phi), the value whose kernel read is the generator's
number, and the block passes 16 of 16.min_sd_u = 0.15 scores a
summed |coverage - nominal| of 0.1614 against 0.7107 at 0.05 (304 trials
lost, none won), 0.2457 at 0.30 (53 lost) and 0.5164 at 0.50; declining on
the floor scores 0.3379. No fit reaches the ceiling, so the ceiling policies
are identical on this ladder. The within-cell read at the shipped placement:
box-uniform 0.1300 against chord 0.2933 at a fixed truth (previously 0.1233 /
0.2900), and 0.0252 against 0.2106 over the truth-swept fits whose placed axis
contained the truth (0.0361 / 0.2004). test-hyperparameter-coverage.R
passes 9 of 9, and the BYM2 rho and ICAR tau rail lifts read 2.7707 and 9.0
against a threshold of 2.log_evidence = NA
(gcol33/tulpa#735). A free-covariance block lays its outer axes as the
log-Cholesky coordinates of Sigma, and its default two-field grid is a tensor
in (log sigma_1, log sigma_2, rho) converted to those columns, so no column had
a measure of its own and every one declined logchol_design_measure. The
block is now given one prior and one cell measure over all of its columns, on
the coordinates its grid is a tensor in (.hp_logchol_design()): the
two-field default carries independent PC priors on both standard deviations
and an LKJ prior on rho, with its cells measured in (log sigma_1,
log sigma_2, rho); a grid that is a tensor in the log-Cholesky columns
carries the same PC + LKJ prior pushed to those columns
(re_cov_pc_lkj_prior()) and their widths. The two forms agree through the
map's Jacobian s2 / (1 - rho^2) to 1e-15, and the default prior integrates
to one on a wide grid. A block with a fixed column declines
logchol_partial_block; one whose grid is a tensor in neither declines as
before. The default posterior on Sigma now carries this prior where it
carried none.dispersion_prior_unsourced
(gcol33/tulpa#736). They now carry R-INLA's shipped defaults for the same
parameters, loggamma(1, 0.01) on the gamma family's precision parameter and
loggamma(1, 0.1) on the beta family's (rinla/R/models.R), which are
exponential priors on phi itself in the parameterisation tulpa's kernels
use. The rates are .NL_HYPERPRIOR$gamma_shape_rate and $beta_precision_rate.
Neither is a PC prior: R-INLA ships one for the mean-one Gamma(1/a, 1/a) but
applies it to no likelihood's shape.control$prune ranked outer cells by the likelihood alone
(gcol33/tulpa#734). The screen runs inside the kernel and softmax-normalised
its cheap log-marginals, while since the #730 defaults every axis's
hyperprior is folded into log_marginal in R after the kernel returns and the
cell measure enters only the weights. A cell its prior or its width favoured
could be pruned, and one they suppressed kept, and the safety gate compared two
prior-less argmaxes, so it could not see it. Every screened entry now receives
screen_log_offset, each cell's log hyperprior plus its log cell measure, both
fixed by the grid before any cell is solved (.nl_screen_log_offset()), and
ranks, cuts, restores and gates on cheap log-marginal + offset: the eleven
single-block entries through the shared entry bundle, the multi-block and
joint entries, and fit_spde()'s grid with its (log range, log sigma) prior.
The fit carries the offset as prune_screen_log_offset, the gate's ESS reads
the same posterior, and prune_cheap_log_marginal and the cheap-vs-full gap
stay on the kernel's scale, where the offset cancels. A fit that does not
screen is unchanged..hyper_log_quad_weights()
measured every axis by the widths of its global levels. Each slice cell also
carried a calibration term standing in for the rest of its level, and every
per-axis read dropped the cells another axis's pass had placed. The weights,
the draws and the reported moments therefore described three different
distributions: on the coupled occu_cover SBC fixture the sigma mean read
0.924 from the masked cells against 0.874 over all of them and 0.872 with
refinement off. A grid carrying slice cells is now measured cell by cell
(.hyper_refined_log_quad()). Every cell owns a box; a slice cell re-tiles
its own row of the base tensor, and where refinements on two axes meet inside
one base box the corner is split equally between them, so the base area is
conserved exactly (an extension past the outermost node adds that row's
extension region). The calibration term and the per-axis masks are gone, a
refinement pass anchors only at base cells or cells on its own axis, and a
read with no cell measure sums the base tensor alone. A grid with no slice
cells keeps the product rule bit for bit..nl_posterior_moments() handed .nl_axis_quantiles() the raw
log_marginal, so on an unevenly measured axis (a graded grid, a domain
clamp, the copy scale's atom) the median and interval weighed every node
alike while the mean and SD carried the widths. Both now read
log_marginal + log_quad..nl_grid_log_quad(refining =) rebuilds its axis specs from the base cells,
so a prior read off the declared nodes (the copy scale's exponential rate) is
not moved by a node a refinement pass appended.axis_support and axis_span$integrated did not report the span a refined
grid integrates. Both were read off every distinct level on the axis, slice
points included, as if the grid were a tensor. Densifying a row shrinks the
half step at the ends of that node set, so a stated copy axis c(0.2, 0.5)
refined on the tulpaObs share() fixture reported [0.171, 0.526] while its
cells integrate [0.126, 0.791]; on a domain-closed axis the same rule could
also report past where the mass stops (0.975 against 0.95 for a rho_car row
densified at 0.75). .hyper_grid_supports(refining =) now reads a refined
axis's span from the construction the cell measure uses
(.hyper_fibre_tiling()): the support of the base levels, widened only by the
cells that slice points own past it. The joint driver passes its
refining_axis to both fields, so a densified axis reports the span its
declared nodes had and an extended one reaches the outer edge of the
extension's cell. A grid with no slice cells reports bit for bit what it did.k
steps past the outermost node left (k - 1) / 2 steps of its row with no
owner. Every node of a re-tiled row now owns its nearest-node cell inside the
row's region (the base span together with the slice points' cells), a base
node held to its base cell inside the base span; the outermost base node owns
the stretch past the base edge up to the midpoint, and its base cell carries
prod_k f_k + sum_k e_k / |B_k|. On the 5 x 5 flat fixture that cell's
absolute measure goes from 0.870 to 1.306 at k = 2 and to 1.741 at k = 3,
and the row integrates its region to 1e-12. Unrefined grids, densified rows
and the refine pass's evenly spaced extensions keep their weights bit for bit.
Two rows extended past the same base corner on two axes still leave the
product of their extensions outside every cell.The linear predictor behind every WAIC / LOO read and every posterior
predictive replicate dropped temporal, areal and GP fields
(gcol33/tulpa#721). .tulpa_eta_draws() assembled eta in R from the fixed
effects, the offset, the formula random effects and an SPDE field, so a
temporal_rw1() fit was scored as a GLM: on vignettes/temporal-models.Rmd
WAIC ranked the model without the trend first. The draws now come from what
the fit carries. A ModelData sampler fit records the model its draws were laid
out on (fit$model_inputs) and evaluates the engine's own linear predictor at
each draw (cpp_tulpa_glmm_eta_draws(), the generic_eta_at() assembly the
sampler's observation loop reads), matching an independent assembly from the
draw columns to 1e-16. A nested-Laplace fit draws from its per-cell
fitted_eta / fitted_eta_var mixture; the single-arm fits run through the
joint driver (NNGP, HSGP, SPDE, the spatiotemporal entries) now return
fitted_eta too, through that driver's own eta accumulator. On the vignette's
data the temporal fit now leads by 66.6 elpd, where it trailed by 1.9. predict() stays population
level, as documented, and says so for areal and temporal fields.
The NNGP, HSGP, SPDE and spatiotemporal fits carried fitted_eta without
fitted_eta_var (gcol33/tulpa#727), so their WAIC, LOO and posterior
predictive replicates held the spread of eta across the grid's cells and
none of its variance within a cell. Both joint Newton loops now read
var(eta_r | theta_k, y) = a_r' Sigma_k a_r off the snapshot of each cell's
precision at the mode the fixed-effect block already reads, one solve per
distinct loading vector. Where the sparse scatter left a large intrinsic
field's sum-to-zero pin off the stored precision it is folded back in by
Woodbury, so the variance is that of the precision the cell's log-determinant
was taken of: a fit forced onto the fold reproduces the stored-pin fit to
9e-13. Against the stored precision inverted in R the values agree to machine
precision on a well-conditioned NNGP fixture and to 3e-8 on an intrinsic ST
fit whose precision has condition number 1.2e8. control$fitted_var = FALSE
removes it on these entries as on the others, and a checkpointed grid
resumes it (checkpoint magic TLPACKP7).
row_weight into an amplitude that already carried it, so an areal
svc_weight block scattered the gradient and curvature of w^2 u against an
objective in w u wherever that path ran (a basis block alongside it, or the
sparse gradient gate). The row's block loadings now come from one walk,
for_each_row_block_latent(), behind that scatter, the coupled row scatter
and the new per-row variance.fitted_eta and fitted_eta_var without an
INDEXED_MULTI block's row weight (gcol33/tulpa#729), which its inner solve
applies; the walk now folds it into the amplitude on both kinds.logLik() summed a nested fit's per-cell marginal likelihoods without the
cells' prior masses (gcol33/tulpa#722), so the value grew with the node
count: by 3.3 nats between 9 and 129 nodes on one RW1 support. It is now
log sum_k exp(log_marginal_k + log_cell_k), where log_marginal carries each
axis's normalised hyperprior density and log_cell is each cell's absolute
volume on the integration coordinate (see the next section). Every grid producer records it
as fit$log_evidence where it builds its weights (single- and multi-block,
joint, spatiotemporal, the generic hyper-grid driver, RE-covariance and SPDE
grids, and CILA's corrected marginals); a CCD design reports NA with
log_evidence_declined = "moment_rule_design". At a fixed support the value
is flat in the node count (to 1e-5 over 9 to 129 nodes, agreeing with an
independent trapezoid) and converges on a two-axis grid..nl_grid_log_quad()
keys axes by column name and a one-axis theta_grid is a bare vector, so
every RW1 / ICAR fit integrated with equal weights whatever its node spacing,
and reported no axis_support. .nl_theta_matrix() names the axis for every
reader. On an evenly spaced grid the weights are unchanged.log_marginal and recorded in log_hyperprior / log_hyperprior_axes:
a standard deviation, variance or precision takes the PC prior
P(sigma > 3) = 0.01; a Matern range, GP lengthscale or HSGP ell takes the
range PC prior in the coordinates' dimension, P(range < U) = 0.5 at U one
fifth of the bounding-box diagonal; a bounded axis (a BYM2 or CAR mixing
weight, a copy coefficient's slab) takes the uniform on its domain and AR1
rho its Beta; a negative-binomial size takes R-INLA's pc.mgamma density
with lambda = 7. The evidence is then lse(log_marginal + log_cell) with no
renormalisation over the grid: on an RW1 fit it agrees across 9, 33 and 129
nodes and a support ten times wider to 2e-3, and with a trapezoid of the
kernel's marginal against the density written out by hand. An axis with no
sourced prior declines by name (log_evidence_declined = "improper_hyperprior",
log_evidence_declined_axes): beta precision and gamma shape, where I found
no PC prior in the literature or in R-INLA; the MCAR / MIID log-Cholesky
axes, whose default grid has no per-column cell measure; a range axis on a
block without coordinates.spatial_spde()'s default prior_sigma,
tulpa_re_cov_nested() and tulpa_eb() read the same anchor, and the last
two now default to hyperprior = "pc_lkj". spatial_spde() /
spatial_spde_custom() resolve prior_range from the coordinates
(spatial_spde_custom() takes coords = or errors without prior_range).re_cov_pc_lkj_prior() now
carries it (Lewandowski, Kurowicka & Joe 2009).fit_spde() weighted its grid and CCD cells, and scored its k-hat, with the
PC density on (range, sigma) while integrating in (log range, log sigma)
(gcol33/tulpa#731), missing log range + log sigma. .spde_log_hyperprior()
is the one density behind all three.prior(theta) reached the weights but not
log_hyperprior (gcol33/tulpa#732); it is recorded now, and the kernel-side
part separately as log_hyperprior_in_kernel.log_marginal. It divided the folded marginal back out of the grid weight
and multiplied the kernel's unfolded corrected marginal back in, so the AR1
rho Beta and the joint hyperpriors fell out of a corrected fit's weights,
and with #730 every default prior would have. The folded part stays in the
grid weight and is added to the adopted marginal.log_marginal
(gcol33/tulpa#723), which cannot tell "conditional on estimated
hyperparameters" from "nothing to integrate". It now reads what the fit did:
"log_evidence" for a deterministic fit that estimated no hyperparameter
(integrated, or supplied by the caller as part of the model),
"log_marginal_likelihood" with conditioned_on naming what empirical Bayes
or estimate_phi estimated from the same data, "log_likelihood" for a fit
maximised over every parameter (agq_fit()), and "log_posterior_mean" for a
sampler. compare_models(criterion = "loglik") ranks a Laplace GLM against a
nested model and refuses a set that differs in quantity or in
conditioned_on. AIC() / BIC() refuse every quantity but a maximised
log-likelihood, and count every maximised parameter for one.tulpa() dropped an offset() term on every fit it routed to
nested_laplace (gcol33/tulpa#726): a temporal, areal or GP field, s(),
latent(). The formula's offset was stored on the fit and never reached the
inner solve, so the coefficients were bit-identical with and without it and
the intercept absorbed its mean. tulpa_nested_laplace() gains offset, the
shared entry bundle carries it to all eleven single-block entries and the
multi-block entry (and into the checkpoint fingerprint), and fitted_eta
includes it. A constant offset now shifts the intercept by exactly its value
on every driver.temporal-models and spatial-models compared a nested fit through a hand
log-sum-exp of $log_marginal, the #722 sum; both now use
compare_models(criterion = "loglik"), and temporal-models shows the WAIC
comparison it had said the base engine could not make (gcol33/tulpa#724).phi_grid axis lost the reason sigma did not move
(gcol33/tulpa#720). The field-SD rescues wrote their reason only to the
whole-fit outer_grid_recenter_declined slot, which is cleared by any
placement, and .nl_carry_recenter_stamps() carried nothing from an unplaced
predecessor. diagnostics() therefore gave the generic "add nodes" advice on
a coarsest sigma axis the pass had declined for a named reason such as
axis_pinned. Both field-SD rescues now write outer_grid_axis_declined
(sigma, or b<k>.sigma per copy block, with attempts_exhausted for a
block still railed when the attempts ran out on a sibling), the carry merges
that record whatever the predecessor's placement, and an axis a later pass
moved drops its old decline. The field-SD rescues also list the axes they
moved in outer_grid_recenter_axes, which they had left empty, so a fit
placed on both sigma and a dispersion axis reported only the latter as
moved. The fit itself is unchanged; only the record is.progress.* control knob silently switched the console progress
bar off (gcol33/tulpa#719). progress is a strict prefix of
progress.every, progress.throttle and progress.file, and $ on a list
partial-matches, so .nl_progress_args() reading control$progress got back
whichever one of the three was set. isTRUE() of a file path or a cadence is
FALSE, so asking for the heartbeat file -- the only channel that survives a
detached run -- turned off the bar on the same call, and progress.every,
which exists to tune the bar's cadence, switched it off instead. Setting two
of the three made the prefix ambiguous, $ returned NULL, and the default
came back: the resolved setting was not monotone in how many knobs were set,
which is what kept it out of sight. Both reads take control[["progress"]]
now, so the two channels are independent as documented and only an explicit
progress = FALSE turns the bar off.fit_st_nested() had no control-key check (gcol33/tulpa#673), so a
misspelling (rho_spatail, n_thread) was accepted in silence and the fit
ran at the default. It calls tulpa_check_control() against a
.CONTROL_KEYS$st_nested set like every other nested door now, and the two
knobs it reads without documenting them -- rho_spatial and within_cell --
are in the @param control block. rho_spatial's default moves out of an
inline literal into .NL_ST_GRID, where the rest of that driver's grid
defaults already live.control$sigma_eps was a second spelling of phi, in the other
convention (gcol33/tulpa#675): an SD against the residual VARIANCE every
other door reads. It is refused by name, with a message saying which argument
to use, rather than arriving as a generic "unknown control knob".
control$marginal stays where it is and is now documented for what it is: the
one statistical knob in control, scoped to mode = "eb", turning on the
marginal-Laplace covariance correction. (Promoting it to a tulpa() formal is
a live option; it would be a formal that means nothing on fifteen of sixteen
backends, so it is left as a documented exception.)tulpa() dispatch restated defaults the backend signatures already
carry (gcol33/tulpa#676) -- mala's n_iter / warmup / epsilon,
pathfinder's n_draws, imh's n_iter / scale, agq's n_quad /
sigma_init / max_iter / tol -- so the same number lived in two files and
a bump on one side would have been invisible from the other, which is the
drift gcol33/tulpa#632 measured on k_samples. A knob the caller did not set
is omitted (.drop_null()), so the fitter's own formal supplies it.?tulpa and ?tulpa_nested_laplace (gcol33/tulpa#678,
gcol33/tulpa#674): sigma_re said "message" where the code warns and listed
the backends that ignore it incompletely; the latent-block section claimed at
most one (1 | g) term may accompany the blocks, which is the Polya-Gamma
spatial Gibbs sampler's restriction and not the nested path's;
@param temporal listed three of the five temporal constructors;
@param control named four knobs of the roughly hundred accepted;
screen_iters was documented as 5L against an engine default of 2L
(gcol33/tulpa#640 measured the depth down); and checkpoint plus the four
progress* keys were accepted and undocumented.log_marginal on two conventions
(gcol33/tulpa#698). log_prior_per_arm_re() dropped the weak default
fixed-effect prior -- on the ground that a joint log-marginal should stay
comparable to two single-arm fits -- while the single-arm spec path
(laplace_spec.cpp) included it. icar / bym2 / car_proper / temporal route to
the spec path and nngp / hsgp / the five st_* to the joint one, so
compare_models() or logLik() across the two families read a constant
offset as evidence. It also left the joint objective missing a term its own
gradient and Hessian applied, which is the exact failure the informative-prior
note beside it describes.
The mode does not move -- the gradient already carried the term -- and
softmax cell weights within a fit are unchanged, because the shift is common
to every cell. What moves is the reported log_marginal on the joint-path
entries, by the weak prior's density and normalizer at the mode. The two
families now agree cell for cell on the same ICAR model
(test-entry-conventions.R).cpp_nested_laplace_spde was the one grid entry outside
TULPA_NL_ENTRY_INPUTS (gcol33/tulpa#699). It hand-rolled its driver call,
so it hardcoded prune_tol = 0.0 and passed neither the subspace debias, CILA
nor a screen depth: control$prune / $prune_tol / $prune_log_gap /
$screen_iters / $fitted_var and the debias were all unreachable on the
SPDE grid, and a knob added to the bundle would not have reached it either --
the drift nl_entry_inputs.h's own header says the bundle exists to prevent,
and the same class as gcol33/tulpa#639's hardcoded-off screen. It goes through
the shared bundle and nl_run_joint_sparse_entry() now, fit_spde() resolves
the knobs exactly as tulpa_nested_laplace() does, and a test asserts all
twelve entries carry them. Its n_trials argument is renamed n, the name
every other grid entry uses.divergent -- so the event never
reached result.divergent[], n_divergent(fit) or diagnostic_summary(),
and its only trace was a verbose print. The flag stays TRUE (the trajectory
the chain was asked for did diverge) and the rescue is reported beside it as
n_softabs_rescued, so a reader can tell a rescued divergence from one that
was not.
The retry's other half is a real limitation and is now named as one: its
transition kernel is chosen CONDITIONAL on the first trajectory's outcome and
repeats until it succeeds, with no delayed-rejection correction, so the
mixture is not invariant for the target. It is opt-in (riemannian = 1) and
says so; the documented riemannian == -1 "auto for BYM2/ICAR + dense mass"
branch is deleted, because no caller could select it and an auto path into a
non-invariant kernel is not something to leave one flag value away.n_max_treedepth counted warmup iterations and reached no reader
(gcol33/tulpa#703). Treedepth saturation while epsilon is still adapting is
normal, so the count was inflated by design; it is post-warmup now, summed
over chains, and returned to R beside the per-sample treedepth vector that
covers the same iterations.TLPACKP5 -> TLPACKP6: an existing chain-checkpoint file errors and points
at a fresh path rather than being replayed field-by-field into the new layout.NoCheapEval returned a LaplaceResult with three indeterminate fields
(gcol33/tulpa#705): it default-constructs one and sets log_marginal alone,
and LaplaceResult declared no initializers, so log_det_Q, n_iter and
converged were read from whatever the stack held. Latent today --
prune_active requires cheap_eval_supplied, which is false exactly when
CheapEval is NoCheapEval -- and closed by in-class initializers, which
every other construction site overwrites.add_re_beta_priors hardcoded BetaPrior(), so
reviving it would have silently dropped a caller-supplied beta_prior; every
prior on that path routes through spec_inner_solve. The helpers and the two
sink accessors that existed only for them are gone, and
nested_laplace_multi.h's contract says where the priors actually come from.REGroupOracle's LAYOUT RULE is written down (gcol33/tulpa#689). A
consumer subclasses it and the engine calls through the vtable, so the ORDER
of the virtuals is exported layout under the same rule as a struct field --
and check_abi_version() cannot see a violation, because the two version
numbers still agree. Commit 6c0cad5 inserted has_theta_score() mid-vtable
with the version left at 40; the ABI has since moved to 43 for unrelated
reasons, so no such build can still bind, and the rule is now stated in the
header that exports it.field_coef branch of the placement pilot recorded a moved
alpha axis and not a left-alone one, so outer_grid_pilot$axes_kept
under-reported (gcol33/tulpa#707) -- an unrecorded decline that a performance
knob changes, which is the shape axes_kept exists to prevent..CONTROL_KEYS$tulpa's comment described key sets that no longer carry the
names it named (gcol33/tulpa#708). The subtraction is defensive, not active,
and the comment says so.re_prior$hyperprior was rejected as an unknown key (gcol33/tulpa#667),
though ?tulpa documents it and the front door reads it: .RE_PRIOR_KEYS did
not list it and tulpa_check_control() runs first.control$re_cov was read only when a term carried a slope
(gcol33/tulpa#668), so on a (1 | g) model any value -- including a typo --
was accepted, had no effect, and the fit silently conditioned at
sigma_re = 1. It is validated on every call now and, when the caller names
an integrator, honoured on any random-effect model; an unset knob still
redirects only for a slope, so the default path is unchanged.sigma_re was dropped in silence by agq / gibbs / hmc (gcol33/tulpa#669)
while ?tulpa promised those backends warn. The warning reads one
registry-derived list of the backends that DETERMINE the RE scale -- by
integrating, sampling or maximizing over it -- so the doc and the code have a
single referent.n_trials was read differently by different doors (gcol33/tulpa#677): a
scalar errored at the C++ boundary on laplace and was recycled on mala /
imh_laplace, and one handed to a non-binomial family was read by nothing at
all -- no signal for a user who meant a binomial and typed poisson. One R-side
check now recycles a scalar, refuses a wrong length, and refuses the argument
on a family that does not have denominators.family = binomial() gave "the condition has length > 1",
mode = NULL gave "argument is of length zero", a one-sided formula gave
"is.numeric(y) || is.integer(y) is not TRUE", and an empty data gave "no
non-missing arguments to max". A stats::family() object is now accepted --
a non-canonical link rides the name in the engine's own <base>_<link>
convention rather than being silently fitted at the canonical one -- and the
other four are named..zi_design() detected random effects with a regex on deparsed code
(gcol33/tulpa#680), which false-positives on a | inside a string literal and
whose any() was papering over a deparse that wraps a long line. It walks the
AST with findbars(), two hundred lines above in the same file.spatial_gp(sigma_prior_U =, sigma_prior_alpha =) was accepted on the NNGP
branch and silently dropped (gcol33/tulpa#700): only the HSGP branch
validated and stored them, and .gp_sampler_spec() hardcoded (2.0, 0.05)
against the engine's own (1.0, 0.01) -- one PC anchor with two defaults in
two files. The NNGP spec carries the anchors, the sampler reads them, and the
default is the engine's. An NNGP fit under a sampler mode that did not set
the anchors moves from P(sigma > 2) = 0.05 to P(sigma > 1) = 0.01, which
is the prior spatial_gp()'s own argument defaults have always advertised.spatial_multiscale(nu =) was unvalidated (gcol33/tulpa#701) while its
sibling rejected anything but 1.5 or 2.5, so an unsupported smoothness was
stored at construction and failed deep in the fit. .check_matern_nu() is the
shared check.temporal() constructor ignored shared = FALSE
(gcol33/tulpa#702) while its spatial twin warned -- the asymmetry
.warn_nonshared() was centralised to remove.Rplots.pdf is in .Rbuildignore (gcol33/tulpa#682). It is what a plotting
call in a non-interactive session leaves behind, so it reappears after any
Rscript or test-file run that draws, and .gitignore -- which is where it
was listed -- is not read by R CMD build.
The package could not be compiled on Windows at unoptimized flags
(gcol33/tulpa#717), which is the path devtools::load_all() takes:
pkgbuild::compile_dll(debug = TRUE) is the default and replaces R's flags
with -UNDEBUG -Wall -pedantic -g -O0. A plain COFF object holds 32767
sections, gcc emits one comdat section per template instantiation and merges
none of them without optimization, so nested_laplace_joint_multi.cpp
assembled 50241 sections and aghq_re.cpp 36959 and the assembler stopped
with "file too big". src/Makevars.win now carries -Wa,-mbig-obj, which
selects the bigobj object variant, and the whole package builds, links and
fits at those flags. This is not a property of one file: six further
translation units sit between 21000 and 31000 sections there, while at R's
own -O2 the largest of all 109 is 2221, 6.8% of the ceiling, so the
flag is added only to a build whose CXXFLAGS carries no -O2 / -O3 /
-Os / -Ofast (gcol33/tulpa#725): R CMD check reports it as a
non-portable flag whenever a compile command carries it, which it did on
win-builder r-devel while it was unconditional.
What made the failure look machine-specific was the repo's own .Rprofile,
which sets options(pkg.build_extra_flags = FALSE) and so keeps load_all()
on R's -O2. R reads a project .Rprofile only when the session STARTS in
the package root and startup files are not skipped, so the same clone with
the same toolchain built from an interactive session in the root and failed
from a driver launched elsewhere or under --vanilla.
control$k_samples still moved the reported number -- the one property
gcol33/tulpa#631 exists to hold. .k_dispatch() resolves the held fraction
internally, and .joint_pareto_uncertainty() then re-fitted the shape at the
raw request, which is the published rule whenever the caller named nothing:
the reported k was a different quantile of the weight distribution from the
one the choice was made on. The joint driver resolves once at the top now and
hands the resolved value to every scorer and every re-fit. .k_outer_tail_points()
is idempotent and returns NULL at the reference budget, so a default fit is
unchanged.by_arm_k had the same split (gcol33/tulpa#691), so a per-arm k-hat was
not comparable with the pareto_k printed beside it on the same fit..tulpa_psis_k_uncertainty() applies the 20% ceiling
silently, by design, so the bootstrap re-fits do not each warn -- and only the
joint driver said it once per fit, leaving a control$k_tail_points past the
ceiling reduced with no signal on the single-block, SPDE and RE-covariance
paths. .k_tail_cap_warn() is the shared warning, called once per fit by each
backend's attach point, and .k_dispatch_report() reports the tail it fitted
on, which every fit now carries as pareto_k_tail_points. The single-block
grid path also accepted a tail_points argument and dropped it, so an
explicit request never reached its scorer at all.BACKEND_REGISTRY -- the families it fits, whether it carries an offset, how
many random-effect terms its sweep updates -- and .auto_backend_ok() is the
one predicate the selector consults, reading those fields and the existing
registry-derived .zi_backends() / .phi2_backends() sets. Declaring a
restriction is a registry edit rather than another condition in the selector.re_cov_gibbs
(gcol33/tulpa#666), which then refused an offset, weights, a ziformula, a
phi2, or a gamma / beta / t family at dispatch. Auto takes the exact debias
where it fits and falls through to re_cov_nested -- the deterministic
integration of the same covariance -- where it does not. A plain
(1 | g) model still gets the Gibbs debias, unchanged.mode = "nested_laplace" fits the same model. The sweep's limit is
max_re_terms on its registry entry now.temporal_gp() and temporal_multiscale() had no auto branch at all
(gcol33/tulpa#672): the selector has arms for rw1 / rw2 / ar1, so the fall
through redirected to nested Laplace, the one door that refuses a
continuous-time temporal field. They route to the exact ModelData sampler
they are wired for, the same way a spatially- or temporally-varying
coefficient already does.psis_k to a -1 "not computed" sentinel, and -1 is
FINITE, so the decline branch written for exactly this case (!is.finite(k))
was never reached and .tulpa_khat_band(-1) returned "good". A genuine
Pareto shape can be negative, so the sentinel is translated to NA at the one
place it is known to be one -- the kernel's own result assembly -- rather than
guarded downstream by a sign test.rhat / ess_bulk / ess_tail
on i.i.d. draws (gcol33/tulpa#713), which are the quantities the
draws-provenance gate withholds on a non-chain fit precisely because they are
vacuous there -- and this table is where the gate DISPATCHES. A prose
disclaimer in the print method does not reach check_diagnostics(),
plot_rhat() or a programmatic read; the column names do. The table now
carries n_draws and mcse_mean, the quantity the vacuous columns were
standing in for.param1..paramN on mala /
imh_laplace / pathfinder / vi (gcol33/tulpa#714), which store an unnamed
$draws matrix beside a fully populated $param_names. A warning reading
"Parameters with Rhat > 1.01: param3" could not be traced back to a
coefficient without counting columns by hand. .tulpa_draw_names() is the one
resolver -- the matrix's own names, then the fit's, then positional -- behind
the chain table, the i.i.d. table, tulpa_draws_array() and the pooled-draws
accessor.pareto_k_proposal_source and pareto_k_first_pass never reached
diagnostics() (gcol33/tulpa#715), though both are stamped on the fit and
documented in the fitters' @return. The GAP between the reported k-hat and
the first pass is the actionable number -- a large one says the nodes are
badly scaled around the posterior even though the verdict is fine -- so a fit
rescued by a later proposal candidate read as unconditionally clean.sbc() reported proper_prior = "verified" on a run that probed nothing
(gcol33/tulpa#716). The guard returns early with n_probe = 0 when every
scored quantity is of the "rank" kind. .sbc_premise_proper_prior() is now
the one place that word is chosen, and the print method drops the
"over 0 probed simulations" count rather than pairing it with a verification.logLik() returned three incomparable quantities under one name
(gcol33/tulpa#712): a mean log POSTERIOR over draws (sampler tier), a log
MARGINAL LIKELIHOOD (Laplace), and a log EVIDENCE with the hyperparameters
already integrated out (nested) -- -201.2 / -210.3 / -216.2 on the same
data and formula -- and compare_models(criterion = "loglik") ranked them in
one table, making a model choice on an artefact of which tier fitted each. The
value each tier can give is still the best available there, so it is returned;
it now carries a quantity attribute, compare_models() refuses a set that
disagrees on it and names what each reported, and ?logLik.tulpa_fit says
which tier gives which and why AIC() / BIC() on a nested fit penalise a
value that has already integrated the hyperparameters out.glance() returned one row per outer grid cell on a nested fit
(gcol33/tulpa#711), against the documented single-row broom contract: a
nested fit's $converged is a per-cell logical vector, and data.frame()
recycled every other column to its length. A
do.call(rbind, lapply(fits, glance)) therefore produced a table whose row
count depended on each fit's grid size.VarCorr() errored and ranef() returned an empty frame on every AGQ fit
(gcol33/tulpa#710). agq_fit() ships a 0-ROW draws matrix, so
.varcorr_from_draws() took colMeans() of nothing, built diag(NaN, 1) and
errored out of all(sd_m > 0) -- which .print_re_section()'s tryCatch
swallowed, printing the fit with no Random-effects section while the estimated
sigma_re sat on it. .varcorr_from_point_sigma() reads that estimate,
.re_draws_mat() no longer hands back a 0-row tail as RE draws, and the fit
carries ranef_unavailable naming why AGQ has no per-group posterior --
an empty table being indistinguishable from a model with no random effects.family = "tweedie" was unreachable on every sampler backend
(gcol33/tulpa#694). build_sampler_model_inputs() calls resp.prepare(),
which hard-errors when the family is tweedie and phi2 is NaN, and the caller
assigned phi2 only afterwards -- so it was always NaN at that point. phi2
is a builder argument now, set before prepare().control$n_chains was silently ignored by every backend except NUTS
(gcol33/tulpa#704): a caller asking for four chains got one particle set with
no chain_id and no signal. The same function already hard-refuses
mass_matrix and a warm start on those backends, so this is refused beside
them, in R where a default is still distinguishable from a request.tulpa_sample_glmm() stamped neither backend nor draws kind
(gcol33/tulpa#693), and the provenance gate treats an untagged fit as a chain
-- so mcmc_diagnostics() computed Rhat and ESS on SMC particles and VI
draws. It closes through .finalize_fit() now, which reads emits off the
registry.inst/include/tulpa/joint_nested_laplace_api.h advertised a C callable
registered nowhere (gcol33/tulpa#688). R_GetCCallable on an unregistered
name is a hard R error, so a LinkingTo: tulpa package compiled cleanly
against the header and died in its user's session at the first call. The
header is deleted -- no consumer includes it, and no shim behind it exists --
and test-ccallable-registry.R now scrapes every R_GetCCallable name out of
the INSTALLED headers and asserts each resolves, so a header added without its
registration fails in this suite instead of downstream.nu = 2.5 request was fitted with the Gaussian kernel on every
sampler mode (gcol33/tulpa#686). The integer had two meanings:
gp_cov_type_for_laplace() returned 2L for Matern-5/2, the Laplace NNGP
scatter's private code for it, and .gp_sampler_spec() / .msgp_sampler_spec()
/ .svc_sampler_spec() handed that same value to a cast into tulpa::CovType,
where 2 is GAUSSIAN. So spatial_gp(cov = "matern", nu = 2.5) under
mode = "hmc" sampled exp(-(d/phi)^2), and predict() on a Matern-5/2
Laplace fit used it too, since cpp_gp_field_predict() reads the fit's stored
code through the sampler kernel. Nothing errored, because cov = "gaussian"
is unreachable through the door's match.arg: nu = 2.5 was the only way to
reach that kernel and it happened silently. CovType now carries
MATERN52 = 4 and the kernels themselves live in one exported header,
inst/include/tulpa/cov_kernel.h, read by the exact-NUTS NNGP/SVC kernels,
the Laplace NNGP scatter, the Polya-Gamma sweep and the field predictor alike
-- so the smoothness travels IN the code and no path reads a nu beside it to
learn which kernel it is evaluating. gp_cov_type() (renamed from
gp_cov_type_for_laplace(), since it is no longer the Laplace path's own
mapping) returns 0 / 1 / 4. ABI break (TULPA_ABI_VERSION 42 -> 43): a
consumer passing a raw cov_type integer must re-read it against the enum,
and the bump is what makes that a rebuild rather than a different kernel.temporal_gp(cov = "periodic", period =) collapsed to nothing under the
default scale_coords = TRUE (gcol33/tulpa#687). validate_temporal_gp()
standardizes the time axis and the kernel evaluates sin(pi * d / period)
against those scaled lags, while period travelled untouched: on a 60-month
series with period = 12 the maximum scaled lag is 3.32 against a period of
12, over which the sine is monotone and small, so the fitted kernel had no
periodic structure in it at all. period is a LAG and now makes the same trip
the axis does. The declared number is left on the spec, so it prints and
validates as the user wrote it; period_scaled is that period in the
kernel's own units and is what the fit reads.rw2_quadratic_form annihilates a ramp and a penalty on the sum reaches the
constant only, so M improper directions rode into the target unpenalized --
the defect gcol33/tulpa#600 fixed for the Kronecker and Type-II paths, in the
one branch it did not touch. The ramp pin is now read from the same predicate:
temporal_has_trend_null() is the RW2 fact by itself and st_needs_trend_pin()
is that fact plus the interaction structure, so the density, the matrix form
and the mass override still share one answer.rank_t = T - 1
powers of prec_j against a quadratic form of rank T - 2, per basis
function. This is gcol33/tulpa#596's defect in the sibling branch. Both now
read the fit's own temporal_cyclic.src/test_st_hsgp_prior.cpp is the fixture that makes the branch reachable
from R at all -- nothing in tulpa sets ModelData::has_spatiotemporal, so a
consumer-shaped ModelData is the only way in, as it is for the Type-IV
fixture beside it. It drives the shipped density, not a copy.n_trials and the offset were not checked (gcol33/tulpa#665).
tulpa() guarded X and y alone, and tulpa_laplace() -- a front door in
its own right, which does not route through .validate_glm_design() --
guarded neither. An NA in either reached the kernel and came back as an
all-zero coefficient vector with no error and no warning. Both arms are now
in .assert_finite_model_inputs() and both doors call it with them.run_hmc_chain() (the R wrapper) and
run_hmc_parallel_chains_cpp(), while run_hmc_chain_cpp was entered
directly from six other production sites -- including
tulpa_run_nuts_generic, the R_RegisterCCallable a LinkingTo consumer
reaches for backend = "hmc". A consumer's hand-coded spec->gradient_fn was
therefore checked when it asked for several chains and unchecked when it asked
for one, which is the reverse of what design principle 3 says. The gate is now
ensure_gradient_verified() inside run_hmc_chain_cpp itself: it skips inside
an across-chain OpenMP region (where the producer has already run it on the
main thread and an R warning would be unsafe) and skips when the fit in scope
has already been verified, so the check runs once per fit however many chains
it runs. g_gradient_verified travels with GradientModeFitScope, so a
fallback is still confined to the fit that triggered it.nobars() / no_latent_terms() / no_special_terms() rebuilt every call
from at most two positional children, discarding argument names and any third
argument (gcol33/tulpa#664). nobars() runs on every formula, so the damage
reached every backend: poly(x, 2, raw = TRUE) was refitted as the ORTHOGONAL
basis (coefficients 6.16 / -0.14 against lm()'s 0.47 / -0.009 on a
200-row fixture), splines::ns(x, df = 3) became a backtick-quoted
`splines::ns`(x, 0.5) that errors "could not find function", and
scale(center =), log(base =), cut(breaks =), bs(degree =) all lost
their named arguments silently. The three strippers are now one walker,
.strip_rhs(), which descends the formula OPERATOR tree (.FORMULA_OPS)
and treats every other call as a leaf: head, arguments and names carried
through as they stand. A raw polynomial now reproduces lm() to 1e-3.run_hmc_parallel_chains_cpp folded
n_iter, n_warmup, L, n_chains, seed, max_treedepth, metric_type, adapt_delta,
riemannian, layout.total_params, data.N plus the per-chain init and metric,
and never the contents of y, X or the offset, so every chain read as
already complete and identical(fit1$draws, fit2$draws) was TRUE across two
different responses. Both hmc_chain_checkpoint.h and the CLAUDE.md
checkpoint section stated the guarantee the code did not provide.ModelData::model_response_data, an opaque pointer
owned by the model package, so no field-by-field fold reaches it. What is
reachable is the quantity the chains sample: fold_target_identity()
evaluates the log posterior at engine-fixed probe positions (chain 0's init
plus splitmix64 perturbations of it, never R's stream), which reads the
response through the likelihood the fit will use, the designs and offsets
through eta, and every prior hyperparameter ModelData carries. The perturbed
probes are what make the DESIGN visible -- at beta = 0 the design drops out
of eta -- and a changed X of the same shape is now refused too. A model with
no generic LikelihoodSpec warns that its fingerprint covers settings and
dimensions only rather than claiming more. Requesting a checkpoint still moves
no draw: a same-data resume is bit-for-bit identical.y and X by value
(make_nl_grid_checkpoint, since the #431-#451 batch); this closes the chain
half.temporal_idx was validated at no entry (gcol33/tulpa#685). block.idx
reads temporal_idx[i] for every i < N through Rcpp::IntegerVector::
operator[], which is unchecked pointer arithmetic, so a vector shorter than
N read past the allocation and returned a finite, session-dependent
log_marginal; an out-of-range, zero, negative or NA value was dropped by
the eta walk's own guard and contributed nothing to that row, silently.check_latent_obs_index() (src/areal_input_check.h) is now the one gate for
every per-observation latent index, with check_areal_site_index() and the
new check_temporal_index() as its two named callers. The temporal check runs
inside make_temporal_latent_block() -- the one place every temporal-carrying
entry builds its block, so cpp_nested_laplace_temporal and all five st_*
entries inherit it rather than each calling it -- and at the three index reads
of cpp_nested_laplace_multi, which had the same gap on its own spatial_idx
and temporal_idx.spatial = list(type =, adjacency =) never reached
.validate_adjacency_arg(), so an asymmetric graph fitted silently
(gcol33/tulpa#670). The ICAR precision built from an asymmetric W is not the
ICAR of any graph. The check now runs on whatever spatial$adjacency holds,
and is idempotent (a graph a spatial_*() constructor already passed is not
re-reported), so the two entry styles meet at one gate.type = "ICAR" errored "not yet supported in Laplace" (gcol33/tulpa#671).
tulpa() lowercased the type for its own branch and handed the raw string to
select_inference_mode(), whose comparisons are case-sensitive. The type is
normalised once at the door and written back onto the spec, so every consumer
reads the canonical spelling; "ICAR" and "icar" now give identical
coefficients.control$checkpoint with resume = FALSE reset the file once per outer-grid
SOLVE, not once per fit (gcol33/tulpaObs#316). An outer-grid placement
rescue refits, so .tulpa_nl_joint_once() runs several times within one fit;
it held the reset, and the post-rescue solve therefore removed every cell the
pre-placement solve had just written. Two consequences: a crash-resume lost
the widest, most expensive pass, and a later resume = TRUE run re-solved and
re-appended that whole pre-placement grid on top of the cells that survived,
growing the file about 2.4x per run with no duplicate keys and an unchanged
log_marginal. The reset moved up to tulpa_nested_laplace_joint(), the one
frame that spans every solve of a fit; the per-solve function now only
publishes the path.phi_grid becoming movable in 0.3.1 is what
gave the cover-hurdle fits a rescue that fires, and therefore what surfaced
it.tests/testthat/helper-phi-placement.R -- the placement fixture moved out of
test-phi-grid-placement.R so the checkpoint file can build a fit that
actually triggers a rescue. A regression test that cannot see one placed is
worthless, so the new block asserts outer_grid_placement first.phi_grid axes were outside the outer-grid placement machinery, and the
fit's own diagnostic named one as its coarsest (gcol33/tulpa#663).
.NL_REGISTRY_AXIS_FIELD enumerates the axes a rescue may move keyed by
spatial prior BLOCK type; a dispersion axis is a hyperparameter of an ARM and
is in no entry, so no rescue walked it. On a 12,179-cell occu_cover() fit
the reported grid_coarsest_axis was phi_pos at h / sd = 55.79 against
4.93 for the auto-placed field SD, grid_resolved = FALSE, and the
dispersion posterior sat at 7.746 with no node within a factor of two of it --
naming as coarsest the one axis nothing could move, with advice (add nodes)
that would have taken ~170 of them..joint_phi_grid_rescue() now moves it. Everything the placement needs was
already there -- the transform registry tags a phi_<arm> column "log",
and the FD stencil's re-evaluation already varies it, since
.joint_grids_from_cells() hands grids$phi_<arm> straight back to
.joint_phi_grid_per_arm() -- so what is new is a slot source, a provenance
read and a write target, not a second placement machine. It fires on the
axis's OWN sizing (railed, or h / sd past .NL_RECENTER$resolve_mult)
rather than on the whole grid's collapsed_edge regime, because a dispersion
axis is crossed onto the tensor independently of the field's geometry.
The placement stencil now runs on a collapsed_interior grid when a
movable dispersion axis wants it. .joint_attach_pareto_k_placement() exists
for the two field rescues, whose trigger IS collapsed_edge, so it computed a
mode and Hessian only on such a grid -- and the reported fit was
collapsed_interior (weight concentrated, modal cell interior on every axis),
which is why no curvature existed to place from. The extra test reads stored
weights (.nl_placement_axis_wanted()) and a caller naming no movable axis
runs exactly the gate it ran before.
Provenance decides, as everywhere else. The engine has no default
dispersion axis, so .nl_axis_is_pinned()'s "equal to the engine's own
default" branch has no counterpart here: an axis marked with auto_grid() is
a default and is placed, an unmarked one is a pin and is integrated exactly as
written. control$auto_recenter = FALSE holds it like any other.
Measured on a BYM2 + Gaussian copy-arm fixture (spatial hyperparameters pinned
at truth, four nodes over residual SD 0.02 to 2, true phi 0.09): pinned,
h / sd 98.2 and a posterior mean of 0.1745; marked, a posterior mean of
0.0898. The placement is the estimate, not only the report.
outer_grid_axis_declined records why the placement pass left an axis
alone, PER AXIS. The whole-fit outer_grid_recenter_declined carries the
reason from the one rescue that could have run and is written only while the
fit is unplaced, so a fit whose field-SD axis moved and whose dispersion axis
did not said auto_recentered and nothing about the axis its own
grid_coarsest_axis was naming. diagnostic_summary()'s resolution note
reads it and reports the lever that applies -- the pin, not the node count.
n_grid counts the cells SOLVED, not the ones that kept usable weight.
.tulpa_grid_reliability() filtered non-positive weights and then took
length() of what survived, so a posterior sharp enough to underflow all but
one of 124 cells reported outer grid quadrature ESS = 1.00 of 1 cells -- a
one-cell grid integrated perfectly, rather than a 124-cell grid collapsed onto
one of them. ess_grid and max_weight are unchanged (dropping zero weights
moves neither); rel_ess_grid is now the share of the solved grid the
quadrature uses.
Closes every open engine issue. 0.2.14 closed seven of eleven; the remaining four are below.
control$ccd_stencil_reuse shipped in 0.2.14 opt-in and unmeasured, and the
measurement does not support it (gcol33/tulpa#662). It replaced the full
1 + 2d + 4*C(d,2) finite-difference stencil on most mode-find rounds with an
axial one plus an SR1 secant update of the off-diagonal block. Over 16 paired
joint fits -- same data, same seed, the knob the only difference -- it spent
no fewer inner solves (evals ratio 1.046 at d = 3, winning 0 of 8 pairs;
1.000 at d = 4, 2 of 8), because the secant model produces worse steps and
the walk runs 1.6x longer, spending the per-round saving again on rounds. That
refutes the projection it shipped on, which assumed the same number of rounds
in both arms.
It also centred the design somewhere worse, which is the failure mode the
issue named: the best inner log-marginal on the design fell by up to 7.5 nats
at d = 3 and 16.3 at d = 4, moving the reported fixed effects by up to
1.6e-02, their standard errors by up to 1.1e-02, and the grid's quadrature ESS
from 8.35 to 6.92. The closing stencil is always a full one, so the design's
SCALE is measured either way and only its CENTRE moves -- which is why nothing
inside the placement could see it.
control$ccd_stencil_reuse and control$ccd_refresh_every are removed and
refused at the door rather than ignored. The mode-find measures a full stencil
every round, which is what every shipped fit already did.
657f179 moved every door to the residual variance and four fixtures were
left describing the SD (gcol33/tulpa#661, gcol33/tulpa#659). The recovery
suite's own convention pin fired, as its header says it exists to
("so a change on the door's side fails loudly here instead of silently
rescaling every gaussian fixture"), and the three failures beside it were that
rescaling: a seed-specific measurement taken at residual variance 0.5 and
re-read at 0.7071, and two joint alpha gates handed sd_pos where the door
reads sd_pos^2 -- a 41% over-statement of the residual scale, which the ICAR
block's own comment names as what makes alpha unidentifiable. The engine has
crossed this seam twice in opposite directions: gcol33/tulpa#332 was the same
crossing the other way, costing a slope interval a factor of 1.4079 against
sqrt(2) and coverage 146/150 -> 120/150.
The SBC harness was crossed the same way and its phi_crossed NEGATIVE
CONTROL had swapped roles with its main arm; test-sbc-crps.R reads 8 failures
against the unconverted helper and none against the converted one.
The simulators now read the conversion from the engine
(.phi_to_kernel()) rather than restating it, so a further move of the
convention moves every fixture's draw with it instead of rescaling it
silently. The fixture constants stay in the door's own convention.
A boundary-truncation label was weakened by the very thing it was asked
about (gcol33/tulpa#660). 53a2ef9 folded the outer cells' quadrature
weights into the axis marginal for every caller at once; #657 put the rail
back on the unweighted read and the other two callers were never re-asked.
.nl_axis_edge_mass() names an axis holding material weight on a boundary
node, and the weights include .hyper_domain_clamp(), which SHORTENS the
outermost cell when the naive half-step mirror would reach past the axis's
declared support -- so a bounded axis read weakest exactly at the boundary the
label is about. Measured over 192 BYM2 (sigma, rho) configurations: the
shipped lift runs at median 0.7826 of the unweighted read and as low as
0.5100, thirteen axes lost the label and none gained one. The label now reads
a third measure, "span", which keeps the widths a node's own spacing gives
it -- an outer node owning a wide cell holds mass its node count does not see
-- and drops only the domain closure. .nl_axis_marginal_w(measure =) names
which of the three each caller wants; the rail keeps "inner" and every
reported mean, interval and spread keeps "posterior".
The "resolve" placement trigger was checked against the same folding and is
unchanged: its threshold was calibrated twenty days before it, and over 240
axis-configurations the two reads flip no fire decision (ratio 0.777 to 1.341,
and equal to the bit on any single-axis log-spaced grid).
outer_grid_h_over_sd on a fit and .nl_axis_h_over_sd() are two different
quantities, and the comment describing the first was written about the second.
They agree on none of twelve measured axes and sit six orders apart on a
collapsed one. Both are kept and both now say what they are.
n_threads_outer was clamped to omp_get_max_threads() with no signal, so
a 10x reduction in the parallelism a caller asked for left no trace
(gcol33/tulpa#651). The clamp is correct and stays -- NlCellCache sizes its
slot array from omp_get_max_threads(), so an unclamped num_threads(n)
would map excess workers onto slot 0 and corrupt the shared CHOLMOD factor.
What was missing was the report: a clamped-to-1 run and a serial-by-design run
printed byte-identical progress lines. A fit whose realised width is below the
requested one now says so once, naming the binding cap and how to lift it, and
records n_threads_outer_requested / n_threads_outer_realised so a timing
going into a report can state what it actually ran at. Measured motivation: a
25 km occu_cover fit launched at n_threads_outer = 10 under
OMP_NUM_THREADS=1 ran 32.5 h at 99% of one core and produced no checkpoint.
The CCD mode-find ran for hours printing nothing, so a working fit and a
hung process were indistinguishable (gcol33/tulpa#652). The progress reporter
belongs to the outer-grid loop, which the mode-find runs before, and at
d = 4 one round is 33 full inner solves. The seed pass, each calibration
round and each mode-find round now emit a heartbeat on the fit's own
progress switch, carrying the spend against the ceiling, the step size, the
log-posterior and per-eval timing; a configured progress_file carries the
outer reporter's own four-number format, so a detached placement phase is no
longer an empty file.
CCD placement cost was unbounded by the integration it replaces
(gcol33/tulpa#653). Every reason in .CCD_DECLINE_REASONS was about whether a
Gaussian design is definable; none was about whether placing it is worth it,
so an expensive model could spend up to ~1976 inner solves at d = 4 placing
a 25-node design. control$ccd_budget (default 1) caps the placement at the
tensor grid it would otherwise integrate: break-even, so a successful
placement never costs more than the integration it replaces and a failed one
caps the waste at the same number. It declines with
integration_declined = "placement_budget" up front when even the cheapest
placement cannot fit, and mid-placement when the spend reaches the ceiling.
The cost is reported as ccd_modefind_evals / _budget / _rounds /
_seconds. control$ccd_stencil_reuse (default FALSE, and unmeasured)
trades a stencil's mixed corners for an SR1 secant update, keeping the
diagonal measured and the design's closing curvature a genuine finite
difference; it is off by default because its value has not been measured on a
real fit, which is the remaining open half of the issue.
control$prune = TRUE could return a displaced posterior and report a grid
boundary as an estimate, tripping neither safety-gate trigger
(gcol33/tulpa#656). Three separate defects, all fixed:
no_usable_curvature, and left the axis on its default span
with the mass on the top node -- a reported sigma of exactly 3.000 that was
the top of an un-recentred axis rather than an estimate. The screen now
floors the kept set at .NL_SCREEN$min_keep (5): a central second difference
needs 3 collinear nodes, so 5 is the smallest floor under which no axis can
be a point mass.prune_tol is a normalised weight and loses resolution on a steep surface.
control$prune_log_gap states the cut in nats instead, and a fit reports
the realised cut (prune_log_gap_cut) against the spread of the whole
screened surface (prune_cheap_lm_spread) -- a cut of 6.9 nats against a
spread of 98000 is the pathology, now readable off the fit.?tulpa_nested_laplace claimed a silently-wrong pruned posterior was
impossible. It was not, and that sentence is gone: the gate bounds a
mis-ranking among the cells that were solved, and a cell it discarded and
never solved is outside what it can see.
.hyper_axis_support() returned 1.010 as the support of a spatial
correlation (gcol33/tulpa#657). The half-node-step rule extrapolated past the
end of the parameter's own domain: rho_car's default nodes
c(0.5, 0.8, 0.95, 0.99) closed at 1.01, and a sampler taking that support as
its flat prior reaches a region where the proper-CAR loading is numerically
intrinsic. The node spacing is not what was wrong -- .hyper_axis_level_weights()
exists so an unevenly placed node set integrates the declared measure, and
flat-in-logit would be improper on a bounded domain and put as much mass on
the last percent below 1 as on the whole middle. The cell closure was
wrong. One rule now: an outer cell owns half the distance to whatever comes
next on that side, the next node or the boundary. A domain registry
(.NL_AXIS_DOMAIN) makes the class unreachable rather than fixed once.This moves outer weights on any fit carrying a correlation axis whose top
cell reached past 1: rho_car's outermost prior weight falls 0.0606 to 0.0388,
BYM2 rho's 0.00944 to 0.00527. Point estimates and modes are unaffected; the
grid-marginalised posterior moves. An evenly spaced grid whose top node crowds
the boundary no longer gets equal weights, which is deliberate -- its top cell
cannot extend past 1.
Also fixed here: rho names four different parameters (BYM2 mixing,
proper-CAR, AR1, and the multi-output cross-field correlation), so the
name-keyed c(0, 1) domain was a false claim that made the shipped hsgp_mo
default grid unconstructible. Axis bounds now resolve through the one registry.
c(0.2, 0.5) got a fit
integrating [0.020, 4.94], about 100x the range they asked for, reported
nowhere. A stated axis is now a bound. It is still refined -- more finely,
inside the range given -- and a mode outside it shows up as mass at the edge
instead of moving the range. control$axis_refine = c(alpha = "none" |
"densify" | "extend") sets it per axis; auto_grid() on the nodes marks an
axis a wrapper package computed as engine-placed, so it keeps extending. A fit
reports axis_span per axis, separating the refinement term from the exact
half-node-step term (k / (k - 1) of the stated range: 2x at two nodes,
1.125x at nine).This narrows some existing fits. Any fit that stated an axis and relied on
adaptive_grid = TRUE extending it now integrates the stated range. It also
reaches the var-of-means consistency pass, which runs by default on every joint
fit and not only under adaptive_grid = TRUE; and since the engine has no
default for phi_grid, every phi_* axis counts as stated. Restore the old
behaviour per axis with axis_refine = c(alpha = "extend").
prior_alpha and prior_sigma on a multi-block joint grid regularized only
the first block carrying the axis, silently (gcol33/tulpa#655). A fit copying
two blocks got block 1 shrunk and block 2 flat with no message. One spec now
reaches every block carrying that axis, and a per-block list
(list(list(block = 1, prior = ...), ...), keyed the way the multi-block copy
spec already keys its amplitude side) regularizes blocks differently. Naming a
block that carries no such axis is an error listing the blocks that do. The
fold needed no new machinery -- it is per-axis-spec by construction, so N
blocks is N independent folds; under the old code block 2's atom was never
reached at all. A fit records what was applied in hyperprior_axes.This changes results for a direct tulpa_nested_laplace_joint() caller who
passed one spec to a grid carrying that axis on several blocks. Broadcast
rather than refusal is what keeps the engine's own
.joint_multi_sigma_grid_rescue() coherent: it sets a scalar prior_sigma on
a multi-block grid and refits, and fires exactly on the multi-copy-block
collapsed-edge fits this issue is about.
Rail detection was riding on prior mass placed outside the parameter's
domain. .nl_axis_rail()'s marginal folds the outer prior's cell masses, and
the domain closure above shortens exactly the cell at the boundary being
tested, which moved the argmax to an interior node and stopped a railed BYM2
mixing weight being detected at all. The rail now reads the inner marginal,
which carries no cell widths and is invariant to both the domain closure and
to refinement subdividing cells; the two clauses, the lift = m * w[k] form
and the threshold of 2 are unchanged. This was latent before this release:
#375 calibrated the guard at 2.511 on the 100-region BYM2 rho fixture, and
a later change folding log_quad into the same marginal moved the measured
quantity to 2.099 without re-tuning the threshold, leaving 5% of headroom. The
detector now reads 2.5106 with the domain closure on or off.
Axis bounds on a multi-block joint grid resolved on the block-prefixed name
while the scale and point-mass metadata resolved on the bare one, so b1.sigma
and b1.rho carried no domain at all.
logLik() resolved df to 0 whenever n_fixed was unset, so AIC() and
BIC() came back identical. The count was picked with a %||% chain over
length() calls -- n_fixed %||% length(mode) %||% length(means) -- and
length(NULL) is 0, not NULL, so the chain accepted the zero at its
middle term and the means fallback was unreachable. Both information
criteria then carried no parameter penalty and reduced to -2 logLik, which
is finite and plausible and ranks nothing. Resolved from the first candidate
that yields a positive count instead; the priority (n_fixed, then mode,
then means) is unchanged (#654)..joint_glue_extras_to_res() decided whether a
joint fit carries per-cell modes, iteration counts, precision triplets and
covariance blocks by testing extras[[1]] alone. Which side data a fit holds
is a property of the FIT (store_Q, the kernel's own outputs), but which
CELLS hold it is not: prune = TRUE never solves a screened-out cell, so
that cell has no mode, no precision and no covariance block whatever the fit
stored. When the screen dropped the grid's first corner, the test read a
pruned cell, the rewrite was skipped, and the lists stayed at their
pre-refinement length while the grid grew under adaptive refinement.
tulpa_posterior_draws() builds its mixture from those per-cell precisions
and refuses a list that does not index the grid, so prediction stopped on a
fit that had otherwise converged. Each list is now sized from the first cell
that actually carries the field; a pruned cell keeps its empty slot and is
skipped by the mixture rather than filled in.Every R-level phi is the residual VARIANCE for gaussian / lognormal
(gcol33/tulpa#650). It was the variance at tulpa(), tulpa_laplace(),
tulpa_ep() and build_glmm_logpost() and the residual SD at
tulpa_nested_laplace(), fit_st_nested(), fit_spde(), the
tulpa_nested_laplace_joint() arms and their phi_grid,
tulpa_sample_glmm(), the laplace_*_at() helpers and the rational-SPDE
helpers, so one model was phi at one door and sqrt(phi) at another.
.phi_to_kernel() is now called once per function, adjacent to the kernel
call that wants the SD; the joint tier converts at its single C++ boundary,
covering the arm scalar, the per-arm phi_grid override and the
[n_arms x n_batch] phi_batch. The compiled surface is not uniformly SD,
so the conversion is not blanket: cpp_glmm_oracle_make and
cpp_re_cov_gibbs_sweep take the variance and are handed phi raw, and
truncated_gaussian, interval_gaussian and inverse_gaussian are their own
families rather than variance families. fit_spde() stores $phi in that one
convention instead of $phi_kernel. This changes what an existing call
means: a script passing a residual SD to any of the flipped doors now states
its square, and passes phi^2 to describe the same model.
A phi_grid axis is the residual variance too, and the fixtures that carry
one were left on the SD scale (gcol33/tulpa#650, gcol33/tulpa#649). An entry
in phi_grid overrides the arm's scalar phi outright, so squaring the
scalar alone is a no-op on any fixture with an axis and the axis is the only
thing the kernel reads. Seven test fixtures and four vignettes are restated on
the variance scale, each describing the model it describes.
test-nested-laplace-joint-phi-grid.R was skip_on_cran()-gated and reading
7 FAIL / 7 PASS: every recovery assertion in it compared a posterior mean over
a variance axis against a true SD. test-recenter-pilot.R's fixture had
stopped collapsing its outer grid -- ESS 1.53 -> 4.95 against a < 2 trigger
-- so its placement never fired and the h / sd identity block was measuring
nothing.
The engine is unchanged across this. Built clean at 66e5f60 and at the fixed
fixtures and run on the same outer grid, every cell whose coordinates match
exactly returns a bit-identical log_marginal; the cells that differ are the
adaptive-refinement nodes, whose coordinates differ because the phi axis does.
phi means two different things on two sets of doors, and only some of
them said so (gcol33/tulpa#645). The R family registry parameterizes
gaussian / lognormal by the residual VARIANCE and the compiled kernels
by the residual SD; .phi_to_kernel() converts between them at exactly two
call sites, tulpa() and tulpa_laplace(). Every other door hands phi to
its consumer raw, so one model is phi on those front doors and
sqrt(phi) on tulpa_nested_laplace(), fit_st_nested() and fit_spde(),
and a reader taking one for the other is out by a factor of phi.
tulpa_nested_laplace() documented phi as "Dispersion (negbin/gamma)",
naming the two families where the value is neither convention and staying
silent on gaussian; fit_spde() and the internal laplace_spde_at() /
laplace_gp_at() said "negbin only" / "negbin / gamma only" for a phi
the shipped tests pass sqrt(0.05) to on a gaussian fit;
fit_st_nested(), tulpa_ep() and build_glmm_logpost() said only
"passed to the family". Each now names its convention, the per-family
reading and the counterpart door. tulpa_ep() and build_glmm_logpost()
evaluate the R registry, so their phi is the variance; the rest are
kernel doors and take the SD. tulpa_nested_laplace_joint(),
tulpa_sample_glmm(), fit_spde_nuts() and tulpa_laplace() already said
so and are unchanged. Documentation only: no argument, default or
numerical behaviour moved.cc3e8ed moved
.NL_SUPPORT$mixed$outside from "clamp" to "extend" so that a locally
refined fit and the unrefined fit of the same model could not report intervals
built two different ways. The gate in
test-nested-laplace-joint-ccd-local.R was not re-measured with it -- that
commit's own sweep ran at local_ccd = NULL, so the arm it scored was the
density path and the arm that moved was mixed -- and its mixed totals rose
from 3.18024 / 2.69889 to 3.46157 / 3.14443, taking :785 red under the slow
tier while staying invisible at TULPA_FAST=1.Its two comparators had meanwhile stayed on an inline outside = "clamp", so
the comparison varied the collapse construction and the edge policy together
when it exists to isolate the first. All three arms now go through the one
dispatch at the one within-cell construction. Scored on a common policy the
gate's verdict holds either way and the margin is wider under the shipped one
(lo: mixed 3.14443 against 4.79910 and 4.48729, where under clamp it is
2.69889 against 3.92249 and 3.83155), so what moved is the margin and not the
finding. Bisected across 205 commits, and the pins are re-measured rather than
regenerated: the comment records both the flip and the alignment.
A single-block outer cell's cost is the per-row predictive-variance loop,
and it is now both cheaper and optional (gcol33/tulpa#638). The loop fills
fitted_eta_var with one back-solve per observation per cell, and it is the
whole of the path's size dependence: with it switched off the per-cell cost is
flat in N (0.225 to 0.254 s over N = 900 to 10800 on a 30x30 ICAR lattice,
11 cells, serial), with it on it is 0.217 + 4.685e-4 N. Two changes. A row's
loading vector is (X row, RE group, per-block index and weight), none of
which moves with the outer cell, so rows agreeing on all of it carry the same
vector at every cell and the driver now solves one of them and hands the value
to the rest -- an intercept-only design with 900 distinct rows of 5400 costs
0.244 of what its N predicts and 1.047 of what its distinct count predicts,
4.07x. And control$fitted_var = FALSE skips the pass outright for a caller
that reads only fitted_eta, 21x at N = 10800. The key is compared on exact
bit patterns with every merge confirmed element-by-element, so a hash
collision cannot fuse two rows. SparseCholeskySolver also gained a
workspace-reusing cholmod_solve2 path, so the back-solves no longer allocate
and free per call.
Cheap-pass grid screening is reachable from tulpa_nested_laplace()
(gcol33/tulpa#639). Both entry runners hardcoded prune_tol = 0.0, so the
eleven single-block entries could not screen even though the shared driver,
the CheapEval closure and the safety gate were all built for them -- and
this is the path where a cell is expensive enough for screening to pay.
control$prune, $prune_tol, $screen_iters and $fitted_var now thread
through to every entry, under the same .joint_prune_safety_gate the joint
front door applies, so a single-block fit cannot silently return a pruned
answer either. The gate's warning now names the front door the fit came in
through.
The cheap screen's depth is a control knob with a measured default of 2,
down from a hardcoded 5 (gcol33/tulpa#640). The screen runs on every cell
and only has to RANK them, and each cell is warm-started from its
already-screened lattice neighbour, so every step above what the ranking needs
is paid on the whole grid including the cells it keeps. Over four fixtures at
depths 1/2/3/5 the answer does not move -- Spearman 1.000 against the full
solve, argmax kept, at most 1.2e-3 of the true posterior mass dropped, fixed
effects within 4.8e-5, safety gate never fired -- while the cost does: on the
fixture that isolates the screen's own price, depth 5 is a net loss at 0.75x
where depth 1 is 1.33x. The default is 2 rather than 1 because a small 8-cell
ICAR fixture ranked at Spearman 0.976 on one step and 1.000 on two. Set it per
fit with control$screen_iters; pruning itself remains opt-in.
The outer-grid placement pass no longer has to be paid for with a full grid
(gcol33/tulpa#636). Placement reads an argmax cell and an FD curvature
stencil, both off log_marginal, and neither off the integration the
detecting pass paid for -- so a placement that fires discards every inner
Newton solve of the grid it detected on. control$recenter_pilot on
tulpa_nested_laplace_joint() (default FALSE) detects on a THINNED grid over
the same spans -- both endpoints kept plus at most three nodes between them --
and solves the full grid once, at the placed axes. The rescues are unchanged:
each already detects on the fit it is handed and writes onto the prior it is
handed. Measured over 48 paired seeds on a (sigma, alpha, phi) donor + copy
fixture: 0.78x the cells solved where the placement fires on 10 of 12 seeds,
1.29x where it never fires, so the knob is off by default and pays exactly
when the pilot's cells cost less than the firing rate times the full grid.
The pilot is a pre-screen, so it can only ADD a placement. A pilot whose
detection places nothing is followed by the full fit, which then gets its own
detection through the same rescue pair; a fit declining both is bit-identical
to the same fit with the knob off. A coarse grid does fire somewhat more
readily (the trigger is ess_grid < 2, which falls with the cell count): over
the same sweep the arms disagreed on 9 of 48 pairs, every one the pilot placing
where the full grid did not, 7 of the 9 reaching a higher maximum inner
log-marginal. outer_grid_pilot records the pilot's resolution, cells, thinned
and kept axes, and the regime / ESS / edge axes it DECIDED from -- the reported
grid is never the grid the trigger was read on.
A recentred axis's reported h / sd is 1.25 * sd_used / sd_realized, not
1.25. The node layout's ratio is in PLACEMENT SDs; .nl_axis_h_over_sd()
divides by the weighted posterior SD the placed grid realizes, so the two agree
only where the placement SD is the one the weights realize. Wherever the mode-SD
clamp SUBSTITUTED a bound they cannot: the floor exists to widen an axis sharper
than it. Measured, floor binding on 18 of 18 placed fits, sd_raw median 0.0543
against the substituted 0.15 and a realized 0.0489, reported 3.83. The comment
claiming 1.25 "by construction" is corrected, and the identity is pinned by
test. An axis no rescue moves carries no such relation at all.
.nl_grid_ess() is the one read of a grid's quadrature effective sample size,
behind .joint_pareto_grid_regime() and diagnostic_summary() alike.
A NUTS chain that ends by an exception no longer leaves the progress
reporter registered (gcol33/tulpa#635). g_active_grid_progress is a
process-global raw pointer to a reporter owned by a local unique_ptr, and
both places that set it cleared it only on the normal exit path. Any exception
in between -- a model's Rcpp::stop from its own gradient, a non-centered
field transform reporting Q not positive definite, bad_alloc from a
per-chain buffer -- destroyed the reporter with the frame and left the global
naming it. The next NUTS run in that process read the non-null pointer, took
it for an already-active reporter, declined to build its own, and ticked the
freed object once per iteration. ActiveGridProgressScope now ties the
global's lifetime to the frame that owns the reporter, closing every exit at
once. Measured before and after: unfixed, the pointer is still set once a
chain has been killed by a planted gradient exception and the following chain
is denied a reporter of its own; fixed, it is cleared and the following chain
builds one.
The outer-k candidate dispatch is unchanged at the default budget, verified
on 3300 configurations (gcol33/tulpa#634). gcol33/tulpa#629's synthetic sweep
re-run on current main is BIT-IDENTICAL to its committed baseline across all
3300 rows and all 24 columns, adopted proposal source included -- the same
standard gcol33/tulpa#630 used, and the strongest available statement that the
tail-rule change above does not move a default fit. Along the way the
skew-normal rescue was measured rather than assumed: it is adopted on 7 of
3300 rows, scored on 34, improves the k-hat by a median 0.106 where adopted,
and crosses a reported band on none of them. Every adopted row is a
target whose true skewness is exactly 0, so the gate is firing on
sample-skewness noise in a heavy tail rather than on asymmetry. Reading
pareto_k_proposal_source == "skew_normal" as "this fit needed a skew
correction" over-reads it.
One documentation claim corrected. The candidate dispatch was described as keeping the minimum over all four candidates, so that none could make a fit read worse. It keeps the better of each rung and its input, which is not the same: the mixture is compared to the GRID-MOMENT Gaussian rather than the moment-matched one, deliberately, so that a refined Gaussian which got under the mixture only by widening past the grid cannot mask a grid-width deficiency. A fit can therefore report a mixture at 1.161 with a moment-matched Gaussian of 0.889 in hand, which happens on 1 of the 3300 sweep cells.
The copy coefficient's outer axis can be integrated more finely without
restating its prior (gcol33/tulpa#633). It was the one outer axis a copy fit
could not raise: alpha_grid REPLACES the nodes, and the axis carries prior
structure -- the atom at 0 that gives the "no copy" base model posterior mass,
plus a log-spaced slab -- so a caller who only wanted a finer integration had
to restate the prior to get one, and consumers close the grid off entirely
under copy() for that reason. alpha_n (field_coef$n on the single-block
path) re-reads the engine's own axis at a higher RESOLUTION: same bounds, same
atom, more nodes between them. Supplying both it and alpha_grid is an error
rather than a silent ranking. Underneath, .nl_grid_axis(key, n =) re-reads
any declared axis at a different resolution and refuses on one declared as
explicit nodes, which has no resolution to vary.
The saturation is in the placement, not the prune. Measured engine-side
rather than through the consumer package, since the axis is the engine's:
raising the donor sigma_grid from 13 to 29 nodes leaves the alpha axis at
its declared 6 at every setting and grid ESS at 1.7 / 3.1 / 4.3 while the cell
count more than doubles, and prune = TRUE reproduces the same node counts
and the same ESS to the digit. With the resolution raised alongside, ESS runs
2.3 / 6.8 / 12.5. Whether the engine should instead solve for a declared ESS
floor -- the issue's second open question -- is not answered by this and is
not implied by it.
control$k_samples is a precision knob again (gcol33/tulpa#631). It was
documented as one and was not: under the published PSIS tail rule
min(S/5, 3 sqrt(S)) the fitted tail FRACTION shrinks as 3 / sqrt(S), so a
larger budget described a DEEPER quantile of the weight distribution rather
than the same one more precisely, and two fits of one model differing only in
their budget could report different reliability bands. The four outer
backends now resolve their tail size through .k_outer_tail_points(), which
holds the fraction the default budget implies. The fraction is INHERITED from
.nl_diag("k_samples") rather than chosen, so a default fit is bit-for-bit
unchanged on every backend -- the helper returns NULL there and the
explicit-request path, with its 20% cap and cap warning, is never entered.
Resolved once in .k_dispatch(), the single candidate loop gcol33/tulpa#630
put behind all four backends. tulpa_psis()'s own default stays the published
rule, so the loo::psis() equivalence oracle is untouched.
The held fraction is a FLOOR under the published rule, not a replacement.
Below the reference budget the published rule is in its S/5 regime and is
the more generous of the two -- 40 tail points at 200 draws against the
fraction's 27 -- so replacing it would have bought a stable estimand by making
every cheap diagnostic noisier. Measured cost of getting that wrong: a per-arm
k-hat crossing its reported band on the 200-draw fixture in
test-joint-pareto-k-proposal.R. Under the floor no budget is fitted on fewer
tail points than before, and the fitted fraction is confined to [13.5%, 20%]
over a 500x range of budgets instead of collapsing to 1.3% at 50000 draws.
k_tail_points now exists on the backends whose documentation named it.
It was a joint-path knob only, while tulpa_re_cov_nested(), fit_spde() and
tulpa_nested_laplace() documented it as the way to hold the estimand -- so
following that advice hard-errored on an unknown control knob. All four accept
it and thread it to the shared PSIS core.
A skewed hyperparameter marginal needs no rescue, and the fixture that said
otherwise was reading its own budget. test-outer-skew-rescue.R asserted
that a skewness-0.9 target reads unreliable on a symmetric proposal and is
repaired by the skew-normal one. Measured across budgets, that target reads
0.017 at the shipped budget and climbs to 1.627 at 10000 draws under the
published rule; the fixture reached the rescue only because it scores at 4000.
This is gcol33/tulpa#629's result -- skewness does not inflate an outer k-hat,
a heavy tail does -- arriving from the other side. The test now asserts the
measured property. That the rescue is consequently not adopted on any fixture
in the repo is recorded as gcol33/tulpa#634.
One outer-k draw budget across the four backends that report it
(gcol33/tulpa#632). R/settings.R opens by naming copy-pasted defaults as
the defect it exists to remove and lists k_samples among the five it
consolidated, but the joint path never read it: 500L was hardcoded at the
front door, at the escalation's fallback, and in four entry-point signatures,
while tulpa_nested_laplace(), fit_spde() and tulpa_re_cov_nested() read
.nl_diag("k_samples"), which was 200L. That is not cosmetic. Under
gcol33/tulpa#631 the budget sets the PSIS tail FRACTION as well as the
precision -- 20.0% at 200, where the S/5 cap binds, against 13.6% at 500 --
and the k-hat is read against the FIXED k_usable band, so the same
hyperparameter posterior scored on two backends was characterised at two
different quantiles of the weight distribution and could be banded
differently for a reason the fit did not record.
The one value is 500, which raises the grid, SPDE and RE-covariance
default from 200. It is the joint path's, and it is on the record rather than
on preference: the joint path was raised from 200 to 500 at gcol33/tulpa#127
when outer scoring stopped being adaptive-batched and the single budget
started carrying the whole estimate instead of a batch of it -- a reason that
covers all four backends now that they all score single-batch -- and 13.6% is
both the fraction gcol33/tulpa#631 measured the k-hat to be stable at and the
one every shipped outer-k number, the whole gcol33/tulpa#629 and #630 corpus
included, was read at. It costs those three paths 500 rather than 200 inner
Laplace solves when diagnose_k is on, since one draw is one off-grid
re-solve; control$k_samples sets it per fit on every front door.
k_samples_ok / k_samples_good (the budgets a k_quality target starts
from, 800L / 2000L) and k_bootstrap (1000L, unchanged in value) moved
into the registry with it.
The lint that missed it is re-keyed to the concept. test-settings.R
searched for k_samples next to the literal 200; gcol33/tulpa#127 renamed
the joint variable to diagnose_draws and raised it to 500 in the same
commit, so the drift walked out from under a rule keyed to one name and one
number. The rule now rejects any budget in this family defaulted to a numeric
literal outside the settings file, and a companion test evaluates every entry
point's default against the registry -- a source lint cannot see a second
registry-shaped default that reads a different key.
control$k_samples moves the outer Pareto-k-hat, not just its interval
(gcol33/tulpa#631). It was documented as the estimate's precision knob. Under
the automatic PSIS tail rule min(S/5, 3 sqrt(S)) the fitted tail FRACTION
shrinks as 3 / sqrt(S), so a larger budget describes a deeper quantile of the
weight distribution: on a synthetic heavy-tailed outer target the reported
shape runs 0.57 / 1.41 / 3.53 / 7.94 over 500 to 50000 draws. The estimator is
not what moves -- tulpa_psis() reproduces loo::psis() to 1e-13 at every one
of those budgets, a closed-form Pareto control is flat across the same range,
and holding the tail fraction gives 0.569 / 0.702 / 0.708 / 0.692 with the seed
spread narrowing.
The k_quality escalation's precision rung no longer moves the estimand.
That rung doubles k_samples on a miss whose bootstrap CI straddles a band
boundary, and it is the fallback precisely because it should only narrow the
interval around whatever the k-hat already is. It now pins the GPD tail size to
the fraction the fit's own first pass used, so the extra draws sharpen the same
number. An explicit control$k_tail_points is left alone, and the pinned
fraction is at most 1/5 by construction, so the 20% cap warning never fires.
The k_samples documentation on tulpa_nested_laplace_joint(),
tulpa_re_cov_nested() and the PSIS core is corrected to say what the knob
does. The outer-k default remains the published tail rule rather than a fixed
fraction; dev_notes/issue631/RESULTS631.md records why, and
tulpa_psis()'s own default is unchanged so the loo::psis equivalence
oracle still holds.
One proposal-candidate dispatch behind all four outer-k backends
(gcol33/tulpa#630). An outer Pareto-k-hat scores a PROPOSAL, so what it
reports depends on which proposal families were offered.
tulpa_nested_laplace_joint() scored four and kept the best;
tulpa_nested_laplace(), fit_spde() and tulpa_re_cov_nested() each called
the importance-sampling core once and reported the raw first pass. The
candidate layer now lives in R/outer_pareto_candidates.R on a
backend-agnostic contract (.k_cand_spec()), and each backend supplies a spec
rather than a scorer. A spec with no integration nodes -- fit_spde(method =
"ccd"), tulpa_re_cov_nested() -- withholds the grid mixture rather than
reading a design spacing as a grid resolution, and keeps its radius cap at
Inf as before. The joint path is unchanged bit for bit, asserted by
re-running gcol33/tulpa#629's 3300-configuration sweep against its committed
baseline. Over that issue's 165 synthetic cells the single-candidate read
calls 53 unreliable where the full dispatch calls 8, median k-hat 1.159 ->
0.736 -> 0.259.
A small-group binary RE-covariance k-hat was mostly the proposal's scale,
not a skewed posterior. On test-psis.R's tiny-binary fixture (25 groups x
3 binary observations) the proposal as the mode-find places it scores 14.6 to
49.1 over five seeds; re-estimated from its own importance-weighted moments it
scores 0.29 to 0.78 on four of them, with one genuinely beyond a Gaussian's
reach at 39.4. The package documented the high value as a correct signal; that
note is corrected, and the arbiter test now asserts the repair and its limits
instead of a separation that does not survive.
New reported fields on every nested backend: pareto_k_proposal_source and
pareto_k_first_pass. The first names which proposal family produced the
reported k-hat; the second is the k-hat of the proposal exactly as the backend
placed it, before refinement. A large gap says the nodes are badly scaled
around the hyperparameter posterior even where the verdict is fine -- the
"poor integration versus non-Gaussian posterior" distinction gcol33/tulpa#629
asked for. pareto_k_proposal_source also reports which integrator actually
ran: a fit_spde(method = "ccd") fit that fell back to the grid path says so.
.k_dispatch() declines below the PSIS floor before any candidate runs, so a
sub-floor control$k_samples no longer pays target evaluations to discover it
cannot fit a GPD tail.
k_quality ladder was thought to be unable to reach a genuinely SKEWED
hyperparameter posterior, with the proposal-side rescues
(.joint_pareto_score_mixture / _skew) as the untouched lever for it.
Neither half holds. Skewness is not what a k-hat measures -- the importance
ratio against a Gaussian proposal stays bounded on a skew-normal target, which
reads a median grid-moment k-hat of 0.224 at true skewness 0.851 and -0.073 at
0.967 -- so the skew-normal rescue is scored on 0-5.3% of rows and adopted on
0-1.7%, and where it is scored (on a heavy tail, the shape that DOES defeat a
Gaussian proposal) it reads worse than the Gaussian it was asked to rescue.
Over 165 synthetic configurations the largest gain any un-adopted candidate
offers on a miss cell is 0.089, crossing no band boundary, so no proposal-side
rung is added to k_refine; and every target reaches the good band on some
grid, including one at true skewness 6.74 and excess kurtosis 141, so the
escalation does not gain a decline rung either. One thing the sweep does say
about the shipped "grid" rung: on a heavy tail, densifying a too-narrow grid
moves the k-hat monotonically the wrong way while widening fixes it, so the
boundary-extension half is what earns its place. No behaviour change; the
verdict is pinned in tests/testthat/test-outer-proposal-lever.R and written
up in dev_notes/issue629/RESULTS629.md. Two findings from the same
measurement are filed rather than folded in: three of the four outer-k
backends score a single proposal candidate and read a worse band on 27% of
configurations (gcol33/tulpa#630), and the reported k-hat moves with
control$k_samples, documented as a precision knob only (gcol33/tulpa#631).The k_quality escalation reads which lever its miss actually takes, and
keeps chasing when refinement runs out. A miss has two causes: the outer
Pareto-k-hat can sit CONFIDENTLY outside the requested band (the integration
grid does not represent the hyperparameter posterior), or its bootstrap CI can
STRADDLE a band boundary (the point estimate may already be inside the band and
only the interval's width prevents confirming it). The loop read only
reached / not reached and answered both by refining the grid. Refining is
correct first on either -- it lowers the k-hat itself rather than the noise
around it, and can move an ambiguous k INTO the band, which is what the shipped
fixture measures -- but when refinement reports it added nothing, the two
separate. A confidently-bad k-hat on a grid that cannot be refined ends the
chase as before; an ambiguous one now spends its remaining rounds on the lever
refinement never touches, doubling the importance-draw budget so the GPD tail
is fitted on more actual tail ratios. Previously such a fit was abandoned with
rounds still in its budget, its integration possibly already meeting the
request. .joint_attach_k_quality() classifies every miss as
k_quality_miss = "resolution" / "precision" (NA when there is nothing to
classify) and the loop reads that classification rather than re-deriving it.
Two settings the escalation needed made explicit rather than implied. The
refinement switches (adaptive_grid, integration = "grid") are set inside
the refinement branch, not once before the loop: set before it, a round meant
to hold the grid still advanced it by the default one pass. The refinement
level counts refinement rounds (passes), not loop rounds, so a draw round no
longer burns a pass. k_precision_growth (2) joins R/settings.R with its
derivation -- a GPD shape estimate's standard error falls as 1 / sqrt(S), so
a constant factor is a constant proportional narrowing per round.
A fit records the chase, and says which read produced its band.
k_quality_k_trace is the outer k-hat round by round, starting at the first
fit's, so a chase that did not descend monotonically is visible instead of
leaving only the last round's number. The LAST round is returned, not the
lowest-k one: the refinement passes are nested, so each round's grid is a
strict superset of the last and each round's proposal is rebuilt from the grid
that round changed -- returning a smaller grid because a differently-built
proposal scored better would be optimising the diagnostic rather than the fit.
k_quality_best is documented as the band of the RETURNED fit accordingly.
Three user-facing messages named a knob that hard-errors.
control$diagnose_draws was renamed to control$k_samples and the joint front
door stops on the old name, but the shared PSIS decline vocabulary
(draws_too_few, surfaced through pareto_k_declined / diagnostic_summary()
on every nested-Laplace backend), the k_tail_points tail-cap warning, and the
k_quality band-confidence reason each still told the reader to raise it. The
roxygen also carried "the legacy k_samples name is accepted as an alias for
k_samples", self-referential and false -- there is no alias. All corrected to
control$k_samples, along with the internal comments that called
diagnose_draws "the user-facing precision knob" and were what those messages
were written from; fit$diagnose_draws remains the returned field's name. A
source lint in test-settings.R fails on an imperative aimed at a dead knob
name.
A declared hyperprior meets the outer grid in one coordinate, and a
declared point mass keeps the prior probability it declares. Three ways the
same rule was stated twice. The joint driver's prior_sigma / prior_alpha /
prior_phi families are documented as densities on the axis's natural scale
and were added straight to log_marginal, whose cell widths are measured on
the axis's integration coordinate, so the realised prior was
pi(theta) / theta rather than pi(theta): on a 9-node geometric sigma
grid a pc.prec(U = 2, alpha = 0.05) put 22.7 % of its mass on the smallest
node where the declared prior puts 4.3 %. The change of variables now happens
in one place, .hyper_prior_carry(), shared by that fold, the generic
driver's, and the axis quadrature, and the joint path reproduces the closed-
form cell probabilities of its own declared prior to quadrature error
(max 0.0034, from 0.184). A density on an axis describes that axis's
CONTINUUM, so it is no longer read at the zero level of an axis carrying an
atom_mass -- where the change of variables had made it log(0), giving the
atom zero weight whatever the declared mass said, and where a density finite
at the origin instead reweighted the atom against the continuum, so the
declared split was not the split the fit integrated. The continuum's declared
density now sets its SHAPE and 1 - atom_mass sets its share, and
.hyper_atom_fold_scale() weighs the atom on the continuum's own scale where
the density is folded into log_marginal rather than carried in the weights.
A declared atom_mass is therefore the prior probability the fit integrates,
whatever the density is and however far the nodes reach
(gcol33/tulpa#624, gcol33/tulpa#625, gcol33/tulpa#626).
The recommended prior_alpha on the copy scale moves from
c(U = 8.0, alpha = 0.01) to c(U = 4.0, alpha = 0.01), and the
half-normal from scale 2 to 1. The old numbers were calibrated against a
realised prior of pi(a)/a, so correcting the coordinate moves the whole U
scale. What sets it is the measure the axis carries with no prior at all,
and on the copy scale that is not flat: with no prior_alpha the axis takes
the engine's own exponential slab, whose rate is read off the declared grid
by putting 5 % of the prior above its largest node. On this fixture's alpha
grid (largest node 2.0) that is lambda = 1.50, the strength of a PC prior
at U = 3.07. A user prior_alpha REPLACES that slab rather than adding to
it, so it shrinks harder than the default only once its own
lambda = -log(alpha)/U beats 1.50 -- which is why the bias crosses truth
near the slab's own strength and why U = 8 (lambda = 0.58) reads as
anti-shrinking. Swept over U on
test-nested-laplace-joint-sigma-pos-prior.R's own fixture and 50 seeds
(alpha truth 1.0, coupled sigma truth 0.6), the alpha geometric bias is
monotone in U and crosses truth near U ~ 3.4: flat +0.056, U = 8
+0.116, U = 4 +0.049, U = 2 -0.071, U = 0.25 -0.491. U = 8 now shrinks
LESS than the axis's own no-prior measure, which is why it read as a
regression rather than a stale constant. Nothing in tulpaObs sets these
knobs: control$prior.sigma / .alpha / .phi are forwarded only when a
caller supplies them, so no downstream default was calibrated against the
old measure. The over-shrinking arms reproduce
gcol33/tulpa#22's cross-axis mechanism unchanged -- pulling alpha below its
truth lifts the coupled donor sigma above 0.6, to 0.94 at U = 0.25 -- so
the recommendation's REASONING stands and only its calibration moved. The
test now also asserts what #22 actually claims, that the regularizer beats
the flat axis, which no bound in it had been testing.
A reported per-axis theta_sd comes from the estimator that axis's own
resolution calls for. It was the 3-point parabola at the modal node
wherever that fit, which reads the curvature at the mode: a Gaussian summary,
and one that moves with the spacing of the three nodes it reads. On the copy
axis of a coupled occu_cover fit it measured 0.150 against 0.294 on two
grids over one data set, and sat four times below the spread it stood in for.
The report is now the weighted spread of the axis marginal wherever the axis
is resolved, and the parabola only where the marginal has collapsed onto too
few nodes to have a spread -- the case it was added for, where the weighted
read is a floor at zero. The two regimes are separated by the axis's own
quadrature effective sample size, read off the weights, so the choice is not
one estimator judging the other: .nl_diag("axis_sd_ess") carries the
threshold and the ladder it was measured off. On a Gamma(1.5, 2) marginal the
weighted read spans 2.26 to 2.30 across three grids where the parabola spans
1.16 to 1.88. theta_sd_source / theta_sd_ess /
theta_sd_stencil_declined travel on the fit, so which estimator produced a
reported SD is a property of the fit. The choice is made in
.nl_posterior_moments(), so every nested path reports one rule: the
single-block and spatiotemporal paths always reported the weighted read, a
floor at zero on a collapsed axis, and the joint paths carried their own call.
A design-weighted grid keeps the weighted read whatever its ESS, since a
central-composite design's nodes are not a per-axis lattice for a parabola to
read; that is recorded as design_weighted rather than left implicit.
The var-of-means consistency pass triggers on the axis ESS, not on one SD
estimator compared against the other. Its old test was the weighted SD
against the parabola, which with the weighted read now reported would have
been the estimator against itself and the pass would never have fired. It
asks the question it is actually about -- how many nodes the marginal spreads
over -- and keeps the parabola as the SCALE its new points are placed at,
which is the regime that estimator is right in. control$var_of_means_
tolerance is replaced by control$var_of_means_min_ess; the moved knob
hard-errors.
An outer axis holding weight on a boundary node is named, whether or not
its mode sits there. Both producers of an edge label tested the MODE: the
rail returns early unless the boundary node is the axis marginal's argmax,
and the grid regime returned "spread" without inspecting an axis as soon as
ess_grid >= 2. So an axis with a third of its marginal on its last node and
its mode one node in was reported clean, while truncating its own tail
exactly as a railed one does. outer_grid_edge_mass_axes (and the regime's
edge_mass_axes / _sides, read in every regime) is the second label, in the
rail's own currency -- the boundary node's weight against what a flat marginal
would put there, so the threshold means the same at any node count.
.nl_diag("edge_mass_lift") carries it: 1 is where the boundary node
carries what a flat marginal would, and over 1400 arrangements of four
marginal shapes it names none that leaves less than 0.7 % of its own marginal
outside the span, at a false-alarm rate of 0.011 against "more than 5 %
outside". railed keeps its own stronger statement, that the span does not
contain the axis's own mode.
The axis marginal every one of those labels reads carries the grid's
measure. .nl_axis_marginal_w() normalised log_marginal alone, so a rail,
a resolution read and a reported mean were taken against three-quarters of
the same object on a grid whose spacing is not uniform.
Refinement may not move a declared support. The consistency pass's slice
points are clipped to slab_bounds the way the extension proposals already
are, so a node outside the declared prior support is not solved for a cell
that would carry zero weight. An axis whose posterior presses on that support
reports it through the boundary-mass label above.
A declared axis prior says which coordinate it is a density on. An outer
axis prior could be declared two ways and the two were read on two
coordinates: slab_log_density is carried across explicitly (+ log(x) on a
log axis) while a spec's log_prior entered log_marginal, which is weighted
by cell widths measured on the integration coordinate -- so a caller writing
dexp(sigma) got a prior tilted by 1 / sigma.
hyper_axis_spec(log_prior_coord = ) names the coordinate:
"integration" (the default, and what every existing fit was taken under,
since it is also what the flat default's zero contribution is flat on) or
"natural", which declares a density on the axis value and picks up the
change of variables. Inert on a linear axis, where the two coincide
(gcol33/tulpa#623).
The outer hyperparameter grid is a quadrature rule for a declared prior,
not the prior itself. Every node carried the same prior weight, so a grid
that gained nodes after seeing the data carried a different measure than the
one it started with. On a coupled occu_cover fixture the copy axis went from
6 nodes to 11 and its span from [0, 3] to [0, 16.4], which moved the point
mass at alpha = 0 from a sixth of the prior to an eleventh and moved the
posterior mean of alpha by 18 %. Nodes now carry an integration weight built
from their spacing (R/hyper_quadrature.R), so placing them differently
changes the quadrature error and nothing else. On an evenly spaced grid the
weights are equal, which is the rule the engine already applied, so an
unrefined grid is unchanged.
The copy scale carries a declared prior. p(alpha) = w0 * delta_0 +
(1 - w0) * Exponential(lambda), with w0 = 0.5 and lambda set to put 5 %
of the prior above the largest node of the grid the caller declares, which is
Exponential(1) on the default grid. Both parts are fixed before the fit, so
refinement may follow the posterior past the declared nodes and still be
integrating the measure it started with. control$copy_atom_mass sets w0,
and a caller-supplied prior_alpha replaces the declared density rather than
compounding with it. sigma and the dispersion axes keep a flat measure over
their declared span.
fit$copy_atom reports the point mass at alpha = 0 against the posterior
mass left on it. A point mass competes with a continuum, so a posterior
concentrated at zero can come from either the data or w0, and the two
numbers together are what separate them. On a weakly identifying fixture the
default w0 = 0.5 leaves 0.81 of the posterior on the atom against a true
alpha of 1; at copy_atom_mass = 0.01 the same likelihood locates the mode.
Multi-block axes resolve their scale metadata on the bare axis name.
.joint_axis_specs() matched sigma and alpha exactly, so a multi-block
grid's b1.sigma fell through to the natural scale. This was invisible while
every node weighed the same and became a difference in the weights once they
tracked spacing.
Every path that turns log_marginal into posterior weights carries the same
measure. The single-block and multi-block tulpa_nested_laplace() paths and
both spatiotemporal paths formed their weights from the log-marginal alone, so
the declared prior reached only the joint path. All of them now go through
.nl_grid_log_quad(), and each fit records the span its quadrature integrated
in fit$axis_support so a second engine can read it rather than rebuild it.
An axis's integration coordinate is declared where its grid is built.
.hyper_axis_scale() covers the axes the joint and single-block dispatchers
name, .nl_st_axis_specs() declares the spatiotemporal ones beside
.st_log_grid(), and an axis neither covers carries no quadrature weight at
all, which leaves its nodes equally weighted. Reading the coordinate off the
node spacing instead would let the data choose the measure, and treating the
log-spaced tau_spatial / tau_temporal axes as linear moved the
spatiotemporal grid's weights.
The CCD designs a copy scale on the coordinate its measure lives on, and
splits at the atom. .joint_ccd_grid() took the copy alpha from the
outer Pareto-k registry, where it is carried as an unconstrained real, and
placed an affine design on it. That coordinate has no positivity, so design
nodes ran below zero: on a two-block coupled fixture, five of twenty-five
cells were solved at a negative copy scale, a sign-flipped coupling outside
the model's support. Nor did any node land at alpha = 0, so "no coupling"
carried no posterior weight on the CCD path at all while the tensor rule
integrated it as a level. The continuum is now designed in log alpha, and
because a copy scale is a point mass at zero beside that continuum, the outer
posterior over n such axes is a mixture of 2^n components: one design per
subset of couplings switched off, combined by the prior mass each
configuration declares (control$copy_atom_mass). The two-block fixture goes
from 25 nodes to 64, against 144 for the tensor grid it replaces. A grid with
no copy atom is a single component and is designed as before, on the same
nodes. Past 2^n = 8 components the CCD declines to the tensor rule
(copy_atom_components). A split design hands the outer Pareto-k no
mode-Hessian proposal: its components are built in log alpha while the
diagnostic unconstrains the same grid on the identity coordinate, the only one
that represents the alpha = 0 cells, and one Gaussian is not a mixture
either way. The diagnostic keeps its grid-weighted proposal, which the split's
spread of components supports; pareto_k_proposal_source reads grid_moment
on those fits.
A grid column holding one value across every cell is a fixed setting, not
an axis of the grid. .joint_axis_specs_from_grid() built an integration
axis for every column of theta_grid, so a constant column was handed to the
quadrature as a coordinate to integrate over. A constant contributes the same
factor to every cell and cancels when the weights are normalised, and some
such columns are not quantities to integrate at all: a Poisson count grid
carries an r = Inf node to pin the negative-binomial size, which
.nl_grid_log_quad() then rejected as a non-finite grid. Constant columns are
dropped before a spec is built for them, so a pinned sentinel leaves the
weights identical to the same grid without it, and a held-but-finite setting
shifts every cell by one common constant.
tulpa_nested_laplace() refuses a non-finite response instead of blaming
itself for one. .assert_finite_model_inputs() was wired into three doors
by hand and not into this one, so an NA response was absorbed rather than
rejected: every grid cell returned converged = FALSE with log_marginal =
0, the softmax over those weights was uniform, a fit came back, and the
grid-Hessian retention then found no Q on any cell and reported the user's
own missing data to them as "This is a tulpa bug; please report." The guard
now runs inside .validate_glm_design(), the one validator every (y, X,
n_trials) door goes through, so tulpa_gibbs(), fit_st_nested() and the
re_cov fitters are covered by the same fix and a door added later inherits
it (gcol33/tulpa#613).
The outer Pareto-k diagnostic no longer trips the caller-facing grid-size
warning. The re-evaluation substitutes control$k_samples (default 200)
importance draws for the block's grid axis and re-dispatches through the
ordinary fitter, so a 7-node fit warned that its grid had 200 cells and
advised reducing per-block grid sizes -- advice about a number the caller did
not choose and that remedy does not reach. .nl_internal_batch() is the one
predicate separating an internal batch from a grid the caller asked for, and
the joint side reaches it through .joint_with_quiet_opts(), which already
quiets the checkpoint and the progress bar for the same class of leak. The
hard cell cap is deliberately not gated: it is a resource ceiling, and an
internal batch costs exactly what a user one does (gcol33/tulpa#614).
The Windows arm64 parallel failure is the OpenMP reduction losing a
thread's contribution, and no fit goes through it. tulpa_parallel_sum()
cuts a range into contiguous chunks by index arithmetic and adds the chunk
totals in chunk order, precisely so nothing about a sum is left to the
runtime, and every hot loop was moved onto it -- but the two fixtures behind
test-parallel-equivalence.R were left on a raw reduction(+:) clause, so
the failures they produced were on a construct that ships nowhere. Read off
the r-universe R-release build's own numbers: log_lik = -669.2 against a
true -898.6, and the missing 229.417 is, to the four figures the report
printed, exactly the last chunk of a 4-thread static split (chunk sums
-221.97 / -219.51 / -227.70 / -229.44; dropping the fourth gives -669.18).
One private copy was not added in. The iterations ran -- the same build
passed the per-element results identity at every thread count -- so what
failed is the combination the runtime performs at the end of the region,
which is the one step tulpa_parallel_sum() never asks it for. Both
constructs are now computed over the same values in the same call, beside the
per-observation results, the requested thread count, the resolved team size
and the team the loop actually ran on; the reduction tolerance is the
summation bound the values imply rather than a fixed constant (2.6e-11 and
1.0e-10 absolute here, against the 1e-10 relative that failed, so the shipped
construct is held more strictly than before); and sum_shipped carries the
verdict while sum_omp_red warns with the numbers, since a red suite there
would be a third-party runtime defect on a construct no fit can reach
(gcol33/tulpa#610).
parallel_block_reduce() no longer carries a reduction(+:) clause. It
was the last one in the tree outside the test fixtures, in the package's own
linear-algebra header, named for the job tulpa_parallel_sum() does and
carrying the construct that job exists to avoid. It has no caller, so nothing
was non-reproducible; it is where the next caller would have picked it up
(gcol33/tulpa#618).
CI checks the arithmetic the package actually ships.
-ffp-contract=off pins a C++ result and the same expression in R to the
same rounding, which is what makes a numeric assertion comparable across the
matrix -- but with it on every job, nothing in CI compiled the arithmetic
CRAN and r-universe build. 0.2.0 was green on all four platforms and then
failed checking tests on every aarch64 r-universe build. fp-contract is
now an input on the build-env action, and macOS arm64 -- the only platform
in the matrix where FMA is reachable, since it is baseline on aarch64 and
absent from the generic x86-64 the others compile for -- runs both ways. A
failure the unpinned job alone shows is the contraction; one both show is the
algorithm (gcol33/tulpa#611).
plot.sbc()'s arguments are documented by the block attached to it. An
internal helper had been inserted between the @param tags and the function,
so the # comment opening it ended the roxygen block and the tags attached
to the helper, which carries no title and discards them without a warning.
man/sbc.Rd still listed the four arguments only because it predated the
move: regenerating the documentation deleted them, and the next R CMD check
after that would have reported Undocumented arguments in documentation
object 'sbc' -- a WARNING, on whichever unrelated commit happened to run
document() (gcol33/tulpa#619).
The calibration surface is documented. sbc() and the predictive shapes
it consumes shipped exported and tested, and appeared in no reference index,
no vignette and no README section, so the only way to find them was to read
NAMESPACE. vignette("sbc") is the new article: both experiments run
end to end on a nested-Laplace fit, with the deliberately mis-scaled control
arm the run has to reject, the guards each experiment's premises are checked
by, and the measured disagreement between this and the per-fit reliability
band in both directions (gcol33/tulpa#339). api.md gains the matching
section, and the README a Calibration, validated one.
Every export is in the reference index. _pkgdown.yml listed 63 of 164
exports and swept the rest into internal through the
lacks_concepts() catch-all, so sbc(), tulpa_laplace(), tulpa_eb(),
tulpa_ep(), tulpa_posterior_draws(), every prior builder, every
plot_*() and the whole as_draws / ranef / tidy / glance interop
surface were absent from the published index. Nothing user-facing is hidden
now; the sections gained are Priors, Reading a fit, Calibration,
Plots, and Outer-grid integration and utilities. The two
has_concept("spatial") / has_concept("temporal") selectors matched
nothing -- no Rd in the package carries a \concept{} -- and the explicit
lists that were doing the work are kept.
A folded SBC panel reads the folded verdict. plot.sbc(folded = TRUE)
drew the folded ECDF and annotated it with the RAW p-value and the RAW band
verdict. The fold exists to catch a symmetric dispersion error the raw ECDF
cancels, so the one case it matters for is exactly the one the mislabelling
broke: a panel visibly outside the band reporting inside. Both reads now
come from one .sbc_panel_note() (gcol33/tulpa#612).
api.md named spatial_hsgp(), find_latent_terms(), no_latent_terms()
and parse_bar_term() as public. The first does not exist -- the
Hilbert-space basis is spatial_gp(approx = "hsgp") -- and the other three
are internal. Thirty-eight exports it did not name are now listed, including
tulpa_re_aghq(), fit_st_nested(), tulpa_hyper_grid(),
tulpa_multinomial() / tulpa_ordinal(), temporal_ar() / temporal_ar2(),
VarCorr(), posterior_sample() / mcmc_draws(), diagnostics() and
laplace_diagnostics().
The published changelog was an empty page. pkgdown reads a version
heading as # <package> <version>; this file used ## <version> under a
# tulpa NEWS title, which build_news() reports as no version headings
found and renders as a Changelog heading with nothing under it. All 225
entries are now # tulpa <version>, which is also the form
usethis::use_news_md() writes and R's own NEWS.md reader parses
(225 versions, none with an NA version) (gcol33/tulpa#615).
The site build refused a correct build once the changelog rendered.
pkgdown/build_site.R asserts that CLAUDE.md / AGENTS.md / api.md /
todo.md reach neither docs/ nor the search index, and tested the second
by searching search.json for the string AGENTS.html. The release note
describing that guard contains that string, so the guard failed on its own
documentation. It now matches a search entry's path (gcol33/tulpa#616).
The README's release pin and citation still read 0.1.0.
eta_i += x_i w(s_i), so w -> w + c with beta -> beta - c leaves eta
exactly unchanged and the alias has to be removed before the field reaches
the likelihood. An NNGP field's prior is PROPER, so its constant direction
already carries a prior (precision 1' Sigma^-1 1) and needs no penalty
supplying one -- it needs removing from the likelihood, which is what
centring does. The non-centered path already centred; the centered one
carried a soft penalty on the sum at s2z_precision(n_obs) instead. Both now
go through svc_center_eta, and the stored draws are centred under either,
so svc_w sums to zero exactly rather than being pinned near zero.Measured, paired arms on the poisson SVC fixture differing only in the
constraint, 4 chains x 400 iterations: the penalty was NOT misbehaving on the
centered path. beta_x 1.661 (penalty) against 1.654 (centring) at n = 80
and 1.632 against 1.598 at n = 120, truths 1.650 and 1.647; Rhat 1.04 / 1.02
and 1.03 / 1.04; no divergences either way. The change is one construction
for one alias, not a repair of a measured defect.
s2z_centre_blocks() (tulpa/sum_to_zero.h) is the single centring the four
sites that had hand-written the mean subtraction now share, and that header
carries the proper-vs-intrinsic derivation, including why no constant makes a
ridge right: matched to the field's own prior on the sum it is a term the
field prior already carries, and anything else is a second, unstated prior on
the level. The unreachable plain-double compute_svc_eta and
svc_sum_to_zero_penalty in hmc_svc.h are deleted.
plot.sbc() keeps the uniformity p-value clear of the panel title. The
p line is drawn with mtext() under a title set at line = 1.9 in a top
margin widened to suit, so on a grid of panels the two no longer overprint.
A panel whose ECDF leaves the band reads , outside band rather than
OUTSIDE.
outer_regime = "collapsed_interior" is documented by what the collapse
costs. The laplace_diagnostics() return-value entry called an interior
collapse "benign". It now states that the fit is empirical Bayes at the
mode: the point estimates are sound and hyperparameter uncertainty is not
integrated. That is what the regime note printed with the fit already says,
and what the Details section spells out; only the one-line summary was
short enough to be read as an all-clear.
svc(), tvc() and temporal() can read the fits tulpa() produces
(gcol33/tulpa#607, #608, #609). All three accessors looked for the field spec
at $svc / $tvc and the posterior at $.internal$*_draws, and the front
door #158 wired sets neither: it attaches the spatial spec at $spatial and
puts the field in $draws under the name the sampler gave it. So every
accessor errored on every fit the engine could make, and told the user to
pass an svc = / tvc = argument that tulpa() does not have. tulpa()
now attaches the validated temporal spec beside the spatial one, and the
accessors read the spec from either slot and the field out of $draws. The
reshape follows the flat layout the eta assembly indexes -- observation-
fastest within a term for SVC, time-fastest within a (group, term) for TVC --
so a coefficient lands on the unit it belongs to; test-varying-coef-
accessors.R checks each element against the column the sampler wrote it to,
which is the assertion the front-door tests were missing (they read
colnames(fit$draws) and never called the accessor). A grouped TVC field is
refused rather than reported as its first group, since
tulpa_tvc_posterior carries no group axis.
The compile cache is reportable without being created, and clearable.
tulpa_cache_dir() takes create = FALSE (the default) and only reports the
path; the directory appears where a compile needs it. New
tulpa_cache_clear(older_than =) removes cached builds -- the cache holds
build artefacts only, so an entry it drops is rebuilt by the next
tgmrf_cpp() call on the same source.
Examples: thirteen help pages carried their example as commented-out code,
eleven of them entirely, and now fit a small model and call the function.
The SVC, TVC and multi-scale-temporal examples were written against argument
names (svc =, tvc =, iter =, chains =) that the front door does not
take and would have errored if run; they use the current API and run. The
labels calling SVC and TVC unsupported or experimental are gone.
Language: en-US declared in DESCRIPTION.
tulpa_re_aghq() returns the optimizer's evaluation counts. The joint
driver is one stats::optim call, and nothing on the fit said how much work
it took, so a consumer reporting that had only NA to report on a fit it had
just declared converged (gcol33/tulpaObs#281). counts is optim's own
function / gradient vector, passed through verbatim: BFGS counts
evaluations rather than iterations, and relabelling them would have made the
number say something it does not.cpp_aghq_objective() reports -1e10 where a group's solve failed -- a
finite value, since stats::optim needs one to reject. Nothing checked
whether the accepted optimum carried it, and it is finite, so it passed every
downstream guard: log_marginal came back as the sentinel (a consumer adds
that to other terms and weights a grid with it), theta_cov was the
finite-difference curvature of the penalty rather than of a likelihood, and
reltol = 1e-9 against |f| = 1e10 is an absolute tolerance of 10 nats, so
a run that started in the failure region declared convergence after one step
of whatever ridge was in play. tulpa_re_aghq() now refuses such a run twice
-- at the starting parameters, where the groups behind it can still be named,
and at the optimizer's own stopping point -- returning NULL with a warning
that names them, and agq_fit() errors instead of reporting the sentinel as
a log-likelihood. The value itself is kAghqFailPenalty in
src/aghq_re_core.h, read across the language boundary through
cpp_aghq_fail_penalty(), so neither the C++ producers nor the R consumers
write the literal (gcol33/tulpa#606).tulpa_re_aghq() computed
group_ok from cpp_aghq_blups(), raised a warning naming the first few
failed groups, and then discarded it, so a caller could learn WHICH groups
came back NA only by trapping the warning and parsing indices out of its
message text. It is now returned alongside the per-group quantities whose
NA entries it explains -- blup, blup_var, blup_cov_g, blup_cross_g
-- as a logical of length n_groups, so a consumer conditions its reads on
it. The detection itself is unchanged; only the reporting channel is new
(gcol33/tulpa#605).builtin_family_spec() refuses any family family_has_compiled_impl() does
not recognise, and that predicate asks the mu-space ladder.
interval_gaussian and truncated_gaussian are not on it: their densities
are functions of a per-observation bound rather than of mu, and the spec's
callbacks reach their closed forms through the response payload's bound arrays
instead. Both were refused at construction despite being fully compiled
(log_lik_/grad_hess_interval_gaussian and their truncated counterparts),
which took down every joint-arm fit declaring one -- on the only route those
families have. family_reads_response_bounds() names the pair and the gate
consults it beside the ladder.The same branch has a second side: a payload declaring one of these families
but carrying no bound fell past the callbacks' bound branches into the ladder,
which does not carry them and raises from inside an OpenMP reduction --
std::terminate rather than an R error. prepare() now checks the bound is
present on the calling thread, beside the tweedie phi2 check that is there
for the same reason. Tests: test-bound-family-joint-arm.R, covering both
sides of the branch.
rstantools is declared. The test suite reaches one borrowed generic from
it; it was used without being named in DESCRIPTION.linkinv was 0.5 + atan(eta) / pi, which in the lower tail subtracts two
nearly-equal quantities and loses a digit per decade of |eta|: measured
against pcauchy, relative error 5.5e-12 at eta = -1e5 rising to 2.0e-07
at -1e9. atan(x) + atan(1/x) = pi/2 gives each tail directly --
atan(-1/eta) / pi below zero, 1 - atan(1/eta) / pi above it -- which is
equal to the old expression in exact arithmetic and free of the cancellation.
linkinv is one function behind the generic family route, so this reaches the
density, the score, the Newton working weight and both curvature ladders for
binomial_cauchit and beta_cauchit alike. The R-side registry
(.LINKS$cauchit) now reads stats::pcauchy / stats::qcauchy, matching how
probit and logit already read pnorm / plogis. mu_eta and the higher
derivatives were stable and are unchanged.The convention this follows is stated once above linkinv: a bounded link's
tail is built, never recovered. logit branches on the sign of eta, cloglog
uses -expm1 (#454), cauchit uses the arctangent identity. The stable form
differs per link, so there is nothing to factor out -- the rule is what is
shared. Tests: test-family-link.R, reading the ladder against pcauchy on
relative tolerance and pinning that the cancelling form it replaces is wrong
by orders of magnitude more, and worse the further out it is read.
cpp_laplace_fit_multi_re checks the length of every argument it indexes
(#469). Rcpp::Vector::operator[], Matrix::operator() and
List::operator[] are unchecked, and this entry sizes its loops from y (or
from the RE term count read off re_ngroups) and then indexes X, n,
weights, offset, re_ncoefs, re_sigma_list and x_init, each supplied
separately. A short one was read past the end of its allocation: nothing
crashed, the solve converged, and the result carried no sign of which numbers
came from the data -- the same failure mode as the NNGP coordinate read in
#389. All seven are now checked. cpp_laplace_fit already reached its checks
through as_re_group_vec and build_spec_family_inputs.check_arg_length is the one place either entry raises that error, so the two
no longer carry their own copies of the message. Tests:
test-laplace-arg-lengths.R, which also pins that each argument is still
accepted at its own length and that supplying all of them reproduces the base
fit.
The copy-spec checks are pinned at all three entry points (#465 item 1).
resolve_copy_arm_of_block restored the two checks its clones had lost, but
nothing asserted they fire. test-joint-block-spec-guards.R now drives each
of the three entries -- cpp_nested_laplace_joint_multi, its batch sibling,
and build_joint_layout through cpp_test_joint_pattern -- with an unequal
pair, an out-of-range block, an out-of-range arm and a block claimed twice,
and holds the no-copy sentinel to being no copy at all.
Closed #468 with no code change: the sparse joint Newton loop's
factorization-failure fallback stopped diverging from the shared one at
12b641d. Both drivers call newton_damped_fallback and newton_step_tail.
force_sparse came to be
live at one spatio-temporal entry and discarded at two. NlEntryInputs
(src/nl_entry_inputs.h) now carries the eighteen shared response, design and
control arguments, and one runner per driver -- nl_run_multi_block_entry,
nl_run_joint_sparse_entry and run_st_spatial_entry -- owns the checkpoint,
the unwraps, the driver call and the axis attachment. An entry supplies its
field: the structural fingerprint, its blocks, its grid axes and the axes it
reports back.Collecting the bundle is one argument-free macro rather than a per-site
aggregate initializer. The package builds at C++17, where an aggregate
initializer of sixteen members -- several adjacent ints, several adjacent
doubles -- can be mis-ordered at one of eleven sites without the compiler
seeing it, and the resulting fit would run at the wrong iteration budget or
the wrong tolerance. Every entry declares these arguments under identical
names, so one member-by-member token sequence binds them at all eleven sites
and cannot bind max_iter to n_threads.
Behaviour is unchanged: the entries' existing test files are the gate, and
test-nl-entry-forwarding.R is the new arbiter for the failure the
duplication produced. It drives all eleven entries and asserts each shared
argument is observable in the fit -- max_iter caps the iteration count,
x_init moves a capped solve's mode, store_Q / compute_skew / debias /
cila each add their fields, and the twelve that enter the checkpoint
fingerprint each make a resume against a written checkpoint refuse. A build
with one bundle member dropped fails it at all eleven entries.
A block's per-row design weight is read on every contribution kind
(#463). LatentBlock::row_weight -- the svc_weight that makes a field a
varying coefficient -- was evaluated in one branch of each block walker, the
one-cell-per-row areal kind. A block reaching its latent through a projector
(SPDE, INDEXED_MULTI) or a basis (HSGP, DENSE_BASIS) dropped it. Both walkers
dropped it identically, so eta, the gradient and the Hessian stayed mutually
consistent and no finite-difference check could see it: the fit was simply the
unweighted model, returned with nothing on the object to say so.
block_row_weight() now lives beside the block definition and every walker --
the single-arm spec eta and scatter, the joint eta, the joint scatter, the
cached indexed plan and the coupled-row collector -- folds it into the block
amplitude ahead of the kind split. Unset is 1.0, so an unweighted block is
bit-identical. cpp_nested_laplace_multi reads svc_weight once for every
block type rather than inside the icar branch, and tulpa_nested_laplace()
accepts it on any block it builds; the weight's length is checked against the
observation count. Tests: test-block-row-weight.R, which pins the layer the
weight enters at by fitting the same model with the projector pre-scaled by
hand.
One centering rule for latent blocks, recorded on the fit (#458). Whether
a field is reported sum-to-zero (with the removed constant folded into an
intercept) or at its own mode differed by block type and, within car_proper,
by whether the block was a copy. The rule is now the prior's rank everywhere:
an intrinsic prior (ICAR, BYM2's structured component, RW1 / RW2) has a null
direction the data cannot identify against the intercept and is centered; a
full-rank one (proper CAR, AR1, IID) has none and keeps its mode. Centering a
full-rank field reports a (field, intercept) pair whose joint posterior
density is below the mode's -- the fold preserves eta, so the shift moves both
priors away from their argmax. block_centered on the returned list says
which convention each block took, so comparing two fits of the same field does
not silently compare two splits. log_marginal, the standard errors and eta
are unchanged: every driver evaluates them at the pre-centering iterate.
interval_gaussian no longer reports a floored plateau as a mode (#462).
The interval probability was differenced on the natural scale and floored at
1e-300. Once both pnorm tails underflow -- about 38 sigma out, which a
poorly scaled predictor or a bad warm start reaches transiently -- the kernel
returned a finite log-density, an exactly zero gradient and a floored positive
curvature. The line search accepts a finite objective and the convergence test
reads max|grad| == 0 as a mode, so the solve stopped there and built its
log-marginal from the floor constants. The probability is now formed in log
space (the difference taken in whichever tail keeps its exponent) and the
gradient and curvature as ratios of logs, so a far interval returns a gradient
of order (edge - eta) / sigma^2 pointing back at it. An empty interval
returns -Inf, which the line search backtracks off.
Rmath is gone from the family parallel path (#461). lgammafn, lchoose,
digamma, trigamma, psigamma and pnorm raise their domain and range
paths through R's warning() / error(), which touch R's global error state
and can longjmp -- out of an OpenMP structured block, which is
std::terminate. tulpa/portable_math.h now covers every routine the family
log-likelihood and curvature ladders reach, and neither header calls into
Rmath any more. test-portable-math.R scores each replacement against R's own
over the argument ranges the families use, including across the recurrence
switch where the asymptotic series is at its weakest, and fails if an R::
call reappears in either header.
An unregistered family, and tweedie without its variance power, are raised
on the calling thread (#459). Both stops sit in per-observation code that
runs inside a reduction. The family-enum entries hoisted theirs already; the
LikelihoodSpec route now checks the tweedie variance power in
BuiltinFamilyResponse::prepare(), which runs once, serially, before any
solve and is the first point that sees the family and phi2 together.
The observed-curvature second derivative declines instead of guessing
(#464). obs_curvature_deta2_for_family() summed the working curvature's
second eta-derivative with the observed-minus-working delta's. Where the
delta's second derivative is unregistered that term contributes zero and the
sum is the working answer wearing the observed one's name -- finite and
smooth, and wrong by exactly the delta, which is what the closed-form outer
Hessian, the hyperparameter standard errors and the grid weights are then
built from. It now returns NaN there, as curvature3_obs_for_family() already
did, and mixture_curvature_deriv2() declines in all five fields at once
rather than leaving one finite field readable. cpp_family_obs_curvature_deta2
exposes the value and the gate together.
CheckpointLog's readers take the mutex its writer takes (#460). save()
locked; has() and get() did not, so the class as documented invited a
reader on one worker to walk a bucket array another worker's emplace was
rehashing. The current consumer snapshots the log serially before its parallel
region, so nothing raced; the lock is what lets the next one do otherwise.
n_loaded() had no caller and is gone.
Dead surface removed from the nested-Laplace entries (#466).
force_sparse was an exported argument of cpp_nested_laplace_st_hsgp and
cpp_nested_laplace_st_nngp that both discarded -- an argument the generated
R wrappers advertised with a default while three sibling entries with the same
signature honoured it. Dropped from both. TULPA_COUPLING_FORCE_PARALLEL,
which takes the chunked coupling reduce on every cell so a small grid
exercises the parallel path, is documented at ?tulpa-envvars alongside
TULPA_GRID_WORKSTEAL and tested: test-coupling-force-parallel.R runs a
coupled fit in two subprocesses and asserts the two answers are identical.
Six pieces of the nested-Laplace kernels written twice or three times, now
written once (#465). The copy-spec parse (three copies, two of which had
lost validity checks), the unit-precision IID prior (three), the per-cell
coupled arm views (two), center_joint / log_prior_joint (two) and the
proper-CAR block construction (two, plus the joint branch's own copy /
non-copy pair) each meet at one definition: resolve_copy_arm_of_block(),
set_unit_precision_block_priors(), CoupledArmViews,
center_joint_blocks() / log_prior_joint_blocks() and
set_car_proper_block_priors(). The eleven grid entry-point tails are not
collapsed -- see the note in #465.
src/ comments describe the code, not the repository's history (#467).
src/ ships in the tarball, so its comments are public. Every issue-tracker
reference, planning-stage token (Stage 1.3, Change 2b, Layer B.1) and
account of what the code used to do is rewritten to state the current rule and
its reason. The stale claim that the cross-arm Hessian and the sparse coupled
twin had not landed is gone; both are present. DEFAULT_SIGMA_BETA joins
DEFAULT_TAU_BETA so the same weak fixed-effect prior has one definition in
the two parameterizations the kernels read it in, tied by a static assertion.
The next ten audit issues, and the one that was still a live drop
(#445, #447, #448, #450, #452, #453, #454, #455, #456, #457). Nine had their
src/ work already in the tree from earlier refactors and were missing the
arbiter their checklist named; #448 was not fixed at all.
cell_curvature3.hstates twice that a cell whose differenced Hessian comes back non-finite
takes the whole contraction to NaN, so a broken difference can never read
as an understated cubic term. The per-arm loop ten lines above the guard
did the opposite: a false return from `block_forms` dropped that arm and
let the rest of the cell sum to a finite number, and the `isfinite` check
below it can only see the terms that SURVIVED. `gamma_3` feeds the skew
correction, so understating it moves the correction toward zero -- the
direction that reads as "the Gaussian approximation was fine" -- and the
arm most likely to drop is the one where it was not. Both `block_forms`
exits now return NaN. `h <= 0` had collapsed two meanings into one skip:
an arm the probe direction does not move contributes exactly zero, which
is a value, while a moved arm whose step could not be sized is unreadable,
which is not; they are separated. The row-index reads that reach
`Rcpp::NumericVector::operator[]` are bounds-checked, since a cell-row
table disagreeing with the eta length was a read past the allocation.
pinned by a probe rather than by construction alone. `cpp_test_nngp_cond_cores`
drives `tulpa_nngp::cond_moments` and `tulpa_linalg::nngp_conditional_moments`
on one input: same moments at every jitter, both declining a covariance
that is not PD, and the jitter shown to be a NUGGET (it moves a
well-conditioned answer, which a pivot floor at the same value would not)
against R's own solve of `C + jitter I`.
cpp_link_ladderexposes `linkinv` / `mu_eta` / `mu_eta2` / `mu_eta3` so the ladder can be
held against `stats::binomial("cloglog")` from eta -50 up, where the
cancelling `1 - exp(-exp(eta))` had no correct digits left and returned
exactly 0 -- out of support for every consumer of the link. An absolute
tolerance passes on that value; a relative one does not.
of 600 to 5000 against `eta_c - logsumexp(0, eta)` computed without
forming a probability, plus the `cls` range refusal in both directions.
the per-arm index and weight closures own their own length bound. Both are
checked at block-build time, so `cpp_test_joint_pattern` reaches them with
no Newton iteration. The factories now verify the per-arm vector against
the arm's own row count where the vector is taken, rather than resting on
the R producer two files away.
not_converged on theskew and debias probes), #455 (the joint arm-spec views) and #457 (the
two function-local statics) were verified against the current tree and
already carry their fix and, where one is expressible from R, their test.
The next ten audit issues are the tests they asked for (#427, #428, #429,
#430, #432, #433, #435, #436, #437, #440). The src/ work for all ten had
landed with earlier refactors; what none of them had is the arbiter each
checklist named, so nothing held the fix in place. Each is now pinned against
something outside the function that produces it.
mu. The generic mu-space route floors mu beforedividing by it, and the density and the score used to floor at 1e-15 and
1e-7: in the band between them the two are derivatives of different
functions, and `tulpa_laplace()` line-searches on the first and steps along
the second. `test-family-mu-floor.R` finite-differences the score against
the density across that band for `binomial_probit` / `_cloglog` / `beta_*`,
straddles the analytic floor so the density's clamp point and the score's
are shown to be one number, and pins the worked example: at `eta = -6` a
probit binomial's score is `dnorm(-6) / pnorm(-6)`, which a 1e-7 floor
misses by a factor of 101. Both curvature ladders are held to the weight by
the same finite difference.
test finite-differences the gradient against its own value, so it passes
with 0.5 log|Q(theta)| dropped from both. test-implicit-diff.R now binds
the value to the same cell of cpp_nested_laplace_spde (1e-14 over six
cells) and the normalizer to 0.5 log|Q| of the R-assembled precision, and
records that the term spans 200 nats over that grid with an exact
d / d log_sigma of -n_mesh.
test-spde-fractional-marginal.Rrebuilds the whole marginal in R -- the determinant lemma on B, the
quadratic form through the Pl matvec, the stable binomial kernel -- and
matches it on the ordinary, weighted, confident-eta and zero-mass-node
branches, with the materialised-probability form kept as the negative
control that returns -Inf on the same fixture.
test-nngp-row-neighbours.R drives the SVCand GP density twins on a row with an interior zero and shows it is read as
its leading run, that the count-every-positive reading is a different number
on the same fixture, and that an out-of-range or negative entry ends the row
rather than resolving nn_order[-1].
nu. The shipped referencehardcoded the nu = 1 sigma map and ran only at nu = 1, so it passed
against either. test-spde-nu-general.R now reads .spde_range_sigma() and
runs at nu in {1, 2, 3}, with nu = 2 reachable from alpha = 3 with no
user action.
pg_nngp_field_conditional() is the moment pair extracted out of the sweep
so the probe and the draw read one derivation, and
test-pg-nngp-conditional.R asserts it is row i of
Lambda = (I - A)' D^-1 (I - A) assembled densely in R. On the fixture 18
of 20 locations carry child terms, so the parent-only reading was the rule.
test-nuts-nan-gradient.Rdrives a chain through LikelihoodSpec::gradient_fn whose log-posterior is
finite and whose gradient carries a NaN: every iteration is now reported
divergent, against a control arm that samples and diverges on nothing. The
two leaf predicates are driven directly.
test-log-det-signal.R hands thedispatch and the dense core an indefinite and an exactly singular matrix on
both backends and asserts the bool, and that the 1e-10 base ridge does not
rescue an indefinite H.
test-hsgp-warm-start-guard.Rbuilds the ModelData no fitted path can produce -- spatial_type = HSGP
with has_hsgp = FALSE -- and pins that the flag and the indices disagree
there while the diagonal comes back intact.
LaplaceResult gains pd_conditioned,recorded by both joint loops and emitted per cell, so a caller can tell an
export taken from a conditioned factorization apart from one that never
needed conditioning -- the sum-to-zero path's hessian_pd_at_mode reads the
direct factor of the pinned matrix and cannot answer that.
test-nested-laplace-joint-sparse-equivalence.R now compares the exported
fixed-effect block, the retained per-cell precisions and the standard errors
across the dense and sparse paths (icar / bym2 / car_proper, and with
TULPA_S2Z_DENSIFY_MAX = 0 forcing the rank-1 sum-to-zero storage), and
shows pd_conditioned is TRUE exactly at a conditioned factorization and
that nothing conditioned reaches a coefficient table.
The checkpoint payload carries pd_conditioned, so CheckpointLog::MAGIC is
bumped: a file written by the previous layout would be replayed field by field
into the new one and mis-parsed.
A tgmrf block's symbolic frame is the union over grid points (#472). The
SparseHessianBuilder is initialized once per fit, and the frame was read off
grid point 0 alone under an R-side convention that every grid point's Q shares
one pattern. An assembled Q loses entries at a parameter value that zeroes
them -- an AR1 Q at rho = 0 is diagonal, and Matrix drops an assigned zero
from the pattern -- so grid point 0 can be the sparser one, and on the SPARSE
path SparseHessianBuilder::add then discarded every off-diagonal the other
grid points carried. add_prior_pattern now emits the union, so a grid point
sparser than the frame writes into structural zeros and nothing is dropped.
The dense path assembles directly into the matrix and was never affected.
control$inner_factorization drives the dense joint inner Newton's
factorization backend (#471). "auto" (the default, the latent-dimension
threshold), "sparse" (CHOLMOD) and "dense" select what factorizes the
Hessian the dense joint driver has already assembled -- the three states the
single-arm loop's sparse_override carries. It is independent of
control$force_sparse, which chooses which driver ASSEMBLES the Hessian. The
new test-joint-inner-factorization.R runs one joint problem through both
backends and holds them to factorization noise. The control reaches the
multi-block joint path; a single-block fit that sets it now errors rather than
ignoring it.
An outer-grid cell that fell back to the PD-enforced log-determinant is
counted and reported (#601). On the sum-to-zero path the reported
-0.5 log|B| is read from a direct factor of the pinned matrix
B = H + sum_k coef_k 1_k 1_k'; where that factor cannot be formed, both
readers keep the PD-enforced value, which is a determinant of H + lambda I
after the LM escalation ladder. That value weights the outer hyperparameter
grid, so a cell that fell back is reweighted against its neighbours -- and
nothing said so. LaplaceResult now carries s2z_log_det_fallback, the grid
carries it per cell, and diagnostic_summary() reports the count once per fit
and raises the status to WARN. The checkpoint format carries the flag, so its
header magic is bumped to TLPACKP4 and an older log is rejected rather than
misread.
The sampler and tgmrf block builders check the dimensions they index R
arrays with (#472). build_sampler_model_inputs read nn from one spec
field and then indexed nn_idx / nn_dist against it with no check that the
tables carry that shape, and copied nn_neighbor_dist, nn_order,
nn_order_inv, X_svc, X_tvc, time_index, group_index and
svc_indices wholesale before indexing them at a length taken from elsewhere;
the random-effect branch read ncoefs / correlated / idx per term against
a count taken from ngroups. make_tgmrf_block checked p_k against
n_latent but never i_k / x_k against p_k[n_latent], nor that a row
index is in range -- and the dense prior scatter writes at that row index. All
of them now error naming the field that disagrees, and an empty per-grid list
is refused instead of being read at grid point 0.
Every CUDA host-to-device copy is checked (#474). One of seven call sites
in gpu_cuda.h read copy_to_device's return. A failed copy leaves the
allocation holding whatever the driver last placed there: for the payload
buffers that is a plausible wrong result returned as a success, and for the
three device pointer arrays it is cuSOLVER and cuBLAS dereferencing that
content as device pointers. The read-back path had the same split and is
checked too.
total_log_lik_spec is threaded (#471). The spec path's data
log-likelihood ran serially inside the objective closure -- once per
line-search trial -- while the eta assembly beside it and the family-enum
compute_total_log_lik both took n_threads. It now reads the same
n_threads through tulpa_parallel_sum. At more than one thread the
summation order differs from the serial one, which is the tolerance-level
invariant test-nested-laplace-joint-threading.R states.
Maintenance across the Laplace shard (#470, #471): 78 lines of GP covariance
helpers with no caller are deleted; the same-term random-effect cross block
drops two branches that cannot be reached (a term reads one group per
observation, so t == tp carries g == gp); used_sparse_factor is computed
once instead of twice; the tweedie parameter resolve, the Student-t degrees of
freedom and the truncated families' (q, p) derivatives are each one helper
instead of six, four and three copies; and the probability floor, the tweedie
mean floor and series width, the censored-arm curvature floor, the precision
denominator guard and the centering fold cutoff are named constants. The
backtracking line search's doc now states the acceptance test it actually
applies.
LikelihoodSpec::ll_fwd is removed, and the ABI is 42 (#493). The slot
was assigned at six sites and read at none: resolve_gradient_fn dispatches
on gradient_fn, then ll_arena, then the numerical fallback, and
AUTODIFF_FWD is an explicit alias for the arena path. Each assignment forced
a fwd::Dual instantiation of a likelihood kernel that never ran, and an
unexercised copy of a density is where a kernel falls silently out of step
with its siblings. A model package assigning it now gets a compile error at
the assignment site, which is the outcome the removal is for.
The PC prior's anchors are checked where they are set, and the density can
no longer emit a NaN (#499). lambda = -log(alpha) / U exists for U > 0
and alpha in (0, 1) only: at alpha = 1 the rate is 0 and log(rate) is
-Inf, so the prior is -Inf at every value of the scale and takes the whole
log-posterior with it; above 1 the rate is negative and the density is
improper and increasing. One predicate, pc_anchors_valid, now answers it,
and each door reports it in its own terms -- the R front doors name the
argument, the sampler entry names the spec, and the templated density, which
runs inside gradient loops and OpenMP regions where a throw is
std::terminate, falls back to a flat prior on sigma the way the SPDE
hyperprior already did.
The HSGP, HSGP-ST and TVC scale priors take settable anchors (#506). All
three hardcoded P(sigma > 1) = 0.01 inline. spatial_gp(approx = "hsgp")
and temporal_tvc() take sigma_prior_U / sigma_prior_alpha, defaulting to
exactly those values, and the densities read ModelData fields rather than
literals.
A scale or precision reaching a logarithm is validated at the entry point
(#522). nl_check_positive / nl_grid_axes_positive reject a zero or
negative sigma_re, tau_grid, sigma2_grid or lengthscale axis with a
message naming it, instead of letting -Inf or NaN reach the inner Newton
solve as a cell whose marginal is simply not finite. The NNGP marginal
fallback in the gradient path carries kGpVarFloor, the floor the main path
already passes into vecchia_cond_grad -- the unfloored convention was the
FAILURE path, where sigma2 is most likely to be extreme. The stored
non-centered draw and the log-posterior's own transform now take the same
bounded safe_exp, so the stored field is the same function of q as the
field the likelihood saw.
The integrator selection can be scoped, and its substep count is bounded
(#483). with_tulpa_integrator(name, expr) restores the previous selection on
error as well as on success; the selection is process-global, so a bare
tulpa_integrator() call leaves every later fit in the session on the new
scheme. set_integrator_scheme validates the name and the RESPA substep count
before any of its five globals moves, so a rejected call leaves the process on
the integrator it was already using rather than on a half-reset one.
Both scatter index caches key on the builder's pattern generation (#483).
They hold flat offsets into values resolved by lookup(row, col) and were
keyed on (builder pointer, nnz), so two different patterns installed into
one builder with equal nnz would satisfy the key and hand back offsets
pointing at other entries -- a silently wrong Hessian the pattern guard's drop
counter cannot see, because the offset is valid, just not the one meant.
TULPA_SCATTER_FORCE_PARALLEL is read per call rather than once per process,
so Sys.setenv() between two fits takes effect.
The AD likelihood ladder dispatches on an enum (#512). It compared the
family code against up to twelve std::strings per observation per
reverse-mode sweep, on a quantity fixed for the whole fit; prepare() now
resolves it once and the ladder is a switch. That classification IS the
coverage list builtin_family_has_ad() reports, so the gate and the branches
cannot fall out of step. The negative-binomial ICAR Gibbs kernel builds its
adjacency into flat CSR once at entry, the form the binomial kernels already
take, instead of constructing an Rcpp proxy per unit per sweep, and its
n_threads argument now drives the per-row work the non-spatial kernel
already parallelizes.
One beta density, one ICAR centring signature, one RE dimension pass
(#506, #507). The beta log-density was written three times (the generic family
kernel, the beta sampler, the SPDE sampler) and is now
tulpa::math::log_lik_beta_logit; the multiscale-GP PC prior reimplemented
log_prior_log_sigma2_pc inline, which is also how it bypassed the anchor
guard; icar_center_field no longer takes the component count it ignores, so
the call site answers "one direction, by design" without opening the callee;
and compute_re_prior stores each term's group and coefficient counts instead
of recomputing the same two ternaries in the next loop.
The centered and non-centered temporal GP are pinned to each other
(#499). cpp_test_temporal_gp_density drives the shipped density at both
parameterizations, and test-temporal-gp-parameterization.R asserts they
differ by exactly the forward transform's log-determinant -- including at a
(sigma, phi, dt) where the 1 - rho^2 floor binds at every step, which is
the configuration no single determinant reconciles if the two branches floor
different quantities.
test-nested-laplace-joint-box-mass.R,
-barycentre.R and -descriptor-plane.R each carried a byte-identical copy
of the same three-block gaussian fixture and let control$within_cell fall
through to the engine default, so flipping that default to box_uniform
re-targeted what their recorded numbers measure. All three were red under
NOT_CRAN=true; the issue saw only one because the other two are
skip_on_cran() and the run that found it was TULPA_FAST=1.The fixture is now ogd_fixture_sim() / ogd_fixture_fit() in
helper-outer-grid-dump.R, with within_cell an argument defaulting to the
shipped read, and test-outer-grid-dump.R asserts that default still equals
.NL_DIAG$within_cell -- so the next flip fails there, naming the files to
re-measure, rather than silently changing what they score.
Re-measured under the shipped read, the finding that moves is which PART of
the read a rule is shown to reach. The two constructions place the same mass
in the same cells and differ by half a cell in where inside one they place it,
which on these coarse grids is the scale the location is resolved at, so the
floor narrows on the widths (0.2158 -> 0.0882 at five levels) and widens on
the median (0.0024 -> 0.0184). The box-mass rule's reach moves from the
location to the spread; the barycentre placement's five-level median margin
falls from 22x the floor to 2.0x while its widths margin rises to 3.1x. The
rules are unchanged -- the box multiplier's per-cell values are identical
under both reads. Write-up: dev_notes/issue599/RESULTS599.md.
st_sum_to_zero_penalty() pinned the interaction along its row sums and its
column sums, S + T - 1 directions, which is exactly null(Q_s (x) Q_t)
under an RW1 temporal marginal. Under a non-cyclic RW2 one the kernel gains
the linear ramp and is T + 2S - 2, so the S - 1 site-specific time trends
summing to zero across sites carried no prior curvature at all. The engine
already held both numbers and they disagreed: the normalizer beside the
penalty reads rank_space * rank_time, which is 8 * 2 = 16 against
ST = 36 on the 3x3 / T = 4 fixture, so it was written for a
20-dimensional kernel while the penalty pinned 12.The penalty now carries a third family, each site's squared linear trend,
at the precision the s2z contract implies: s2z_precision(n) holds the field
MEAN -- the coefficient of the constant direction -- at sd = kappa, and
s2z_precision_weighted(u'u) holds the coefficient of an arbitrary direction
u' phi / u' u at the same sd, reducing to s2z_precision(n) exactly at
u = 1. One predicate, st_needs_trend_pin() (src/st_null_space.h),
answers where it applies, and the density, the sparse matrix form and the
precision-informed mass override all read it. TYPE_II takes the same term by
the same derivation; TYPE_III's kernel is already spanned by the column sums
and TYPE_I is proper. A CYCLIC RW2 gets nothing -- a ramp is not periodic, so
rw2_rank reports T - 1 there -- and an RW1 fit is unchanged bit for bit.
Measured against the numerical Hessian of the engine's own log posterior,
cond(M^-1 Q) under mass_matrix = "gmrf_margin" grew LINEARLY in tau
without bound (13.6 / 18.6 / 75.2 / 529 / 3900 over log_tau 0 to 6) where
RW1 saturated at 17.4; it now saturates at 25.4. On the #598 harness the
RW2 fixture's divergences fall from 43.5 to 8.4 per fit and, per EFFECTIVE
sample, from 5.48 to 0.627 against the adapted dense metric's 0.691 --
closing the one finding gcol33/tulpa#598 could not account for. What remains
is the tau funnel every arm shares. AUTO still does not select
gmrf_margin.
The low-rank mass storage (hmc_mass_lowrank.h) is now generic over WEIGHTED
group sums rather than indicator groups, since a trend group covers the same
coordinates a row group does and differs only in its weights.
Consumer-package paths only: nothing in tulpa sets
ModelData::has_spatiotemporal.
A cyclic Type-IV interaction is fitted under the cyclic Q_t (#596).
st_kronecker_temporal_quad() passed cyclic = false to the RW1 / RW2
kernels whatever SpatiotemporalData::temporal_cyclic said, while the GMRF
normalizer beside it read the flag through rw2_rank. A cyclic RW2
interaction therefore put rank_space * (T - 1) powers of tau_st in the
target against a quadratic form of rank rank_space * (T - 2). The flag now
reaches the diagonal and the cross term alike: rw1_cross_form /
rw2_cross_form take it and carry the wrap-around differences, and the
quadratic forms are those cross forms at a == b, so the wrap edge has one
definition instead of two. st_type_iv_precision() emits the matching wrap
rows, each difference row being one outer product, so the mass override
assembles the operator the density evaluates. Consumer-package paths only:
nothing in tulpa sets has_spatiotemporal, so the Type-IV fixture is the
entry point, and it now takes temporal_cyclic.
gp() no longer offers a linear solver (#542). solver, cg_tol and
cg_maxiter were accepted, validated, stored on the spec and printed, and
never reached GPData::solver_config, so every fit ran the Cholesky path
whatever was asked for. The choice is not worth wiring: it governs only the
k x k neighbour system with k = nn (15 by default), where a Cholesky is
both exact and faster than CG. The kernels and their equivalence test stay,
reached through GPSolverConfig.
An MCAR cell whose Sigma cannot be formed is infeasible, not silent
(#569). An extreme log-Cholesky diagonal made exp(ld) infinite, the forward
substitution returned 1/Inf = 0, and Sigma^-1 came back a zero matrix: the
MCAR prior contributed nothing to the gradient or the Hessian while
log|Sigma| stayed finite. mcar_sigma_inv_from_logchol() now reports
feasibility, one mcar_cell_sigma_inv() reader serves the four closures and a
new block.prep, and the driver skips the cell as it does for every other
block factory.
NlCellCache refuses a thread it has no slot for (#566). An
out-of-range thread number was clamped to slot 0, so two threads would claim
and publish the same slot silently -- the race the per-thread slots exist to
prevent. It throws, matching find().
One writing of nine duplicated computations (#523, #524, #536, #572).
The Cholesky elimination and its forward/back substitution, the CG and PCG
bodies and their copy-in branch, the adapted-diagonal clamp, the
fixed-effect prior scatter, the non-centered transform dispatch, the
options-request temporaries, the logistic derivative ladder and the Bell
coefficients of P0 each had one definition again. compute_log_lik_only()
takes skip_prior instead of evaluating the log-posterior twice and
subtracting, which also stops the prior cancelling in floating point.
matvec_add, compute_linear_predictors, sparse_matvec_csr and
make_se_kernel_matvec route through tulpa_parallel_for, so a team of one
skips libgomp as matvec already did.
SpdeNcTransform built Q = tau^2 K diag(1/C0) K while
SpdeQBuilder builds the operator-chain expansion
tau^2 (kappa^4 C0 + 2 kappa^2 G1 + G1 diag(1/C0) G1). Those are the same
matrix only where C0 diag(1/C0) = I, and the inverse mass is FLOORED to zero
at a zero-mass (orphan) mesh vertex, where the product drops the cross terms
kappa^2 (C0 D G1 + G1 D C0). On such a mesh the non-centered NUTS path and
the Laplace path described different fields, by exactly
tau^2 * 2 kappa^2 G1[j, ] on the orphan's row and column -- 6.2% of the
largest entry of Q, on 5 of 79 rows of the fixture -- which is the
disagreement the orphan ridge exists to prevent. The transform now assembles
the expansion, and the two agree to 5.8e-16 relative on the orphan mesh and
the healthy one alike.G1 diag(1/C0) G1 carries neither kappa nor tau, so init() builds it once
and each proposal costs a scaled add in place of two sparse products. The
rational path collapses the same way: its per-pole terms differ in two scalars
only, sum_k w_k (kappa^2 + r_k)^2 and sum_k w_k (kappa^2 + r_k). Both
hyper derivatives become exact rather than resting on C0 D = I:
d(kappa^4)/dlog_kappa = 4 kappa^4 and d(2 kappa^2)/dlog_kappa = 4 kappa^2
give dQ/dlog_kappa = 4 kappa^2 tau^2 K termwise, which is the closed form
backward() and forward_with_tangent() already applied.
tests/testthat/test-spde-guards.R reaches the orphan ridge on both
assemblies and on a Laplace fit and a non-centered adjoint over the same mesh,
and drives the four structural validators (spde_validate_csc / _fem /
_projector / _operators) through both R-callable SPDE entries with a
malformed length, column pointer and row index. Q is scored against an
independent R re-derivation and against SpdeQBuilder itself, with the
product form carried as a negative control: a regression back to it fails on
the orphan mesh and is silent on a healthy one. The indefinite-H guard in the
implicit-diff gradient is recorded there as deliberately untested, since no
valid input reaches it.mass_matrix = "gmrf_margin" is
the #585 precision-informed diagonal plus the block's two soft sum-to-zero
margins as an explicit rank-(S+T) term, M = D + lambda_row R'R +
lambda_col C'C, with both precisions fixed by S and T alone. Those
directions are 1_S (x) a and b (x) 1_T -- linear combinations, not
coordinates -- which is what #585 measured as the whole of the block's
stiffness and as the part no diagonal metric of any kind can rescale. Like
"gmrf" it resolves to a diagonal metric plus the term before any leapfrog
path reads it, and AUTO selects neither.The storage (src/hmc_mass_lowrank.h) is generic over GROUP-SUM directions
rather than over the Type-IV margins, so a single sum-to-zero on an ICAR /
RW1 / RW2 block is the one-group case of the same term. The inverse is
Woodbury on a k x k inner matrix, keeping the per-step cost
O(n + nnz(U) + k^2) against a dense metric's O(n^2), and the momentum draw
is a sum of two independent Gaussians rather than a factorization of M.
Scored on the #585 paired design -- 6 configurations x 8 seeds, arms sharing
the data and the chain seed -- at adapt_delta = 0.95: leapfrog steps per
effective sample come out at a pooled geometric-mean ratio of 0.040
against the adapted diagonal (46 of 48 pairs, sign test p = 8.4e-12), raw
sampling leapfrog steps at 0.053 on 48 of 48 (p = 7.1e-15), with ess_min
rising at the same time and max-treedepth saturation -- 26% to 98% of
iterations on the diagonal arms -- disappearing entirely. The "gmrf" arm
of the same run reproduces #585's null (pooled 1.08, p = 0.67).
Two limits are on the record rather than smoothed over. At
adapt_delta = 0.8 the non-centered configuration is unusable (250
divergences per 1000 iterations at an adapted step size of 2.06), which the
0.95 target repairs; and at 0.95 the three small 3x3 fixtures still show
divergences the diagonal does not, pois_3x3_T4_rw2 worst at 43.5 per fit,
while the two largest configurations are clean in every arm. Whether that is
new pathology or newly visible pathology in chains the diagonal metric was
keeping stuck is not settled here and is carried as #598. Write-up:
dev_notes/issue597/RESULTS597.md.
apply_drift is testable (#597). The fused drift q += c M^-1 p the
zero-allocation NUTS loop takes moved from hmc_nuts_optimized.cpp into
src/hmc_mass_drift.h. It and DenseMassMatrix::inv_mass_times_p are the
two places a metric meets a momentum, and they are now pinned to each other
by test on every metric the engine can build.
The Knorr-Held Type-IV fixture's scaffolding moved from the top of
test-st-iv-gmrf-mass.R into tests/testthat/helper-st-iv.R, so the
precision-informed diagonal and the margin metric are scored against one copy
of the numerical-Hessian arbiter rather than two.
mass_matrix = "gmrf" replaces the Welford-adapted variances over an
st_delta block with diag(Q^-1) of that block's own posterior precision
tau (Q_s (x) Q_t) + diag(h_lik) plus the two sum-to-zero margins, at each
warmup mass window. The metric stays diagonal -- the request resolves to
DIAG plus a flag before any leapfrog path reads it -- and AUTO does not
select it. The likelihood curvature comes through
LikelihoodSpec::eta_weights_fn, which is what #547 removed the half-wired
version for lacking; a spec shipping none declines with a reason on the fit
rather than silently.Scored on 96 paired NUTS fits (6 configurations x 8 seeds, the two arms
sharing the simulated data and the chain seed), leapfrog steps per effective
sample come out at a pooled geometric-mean ratio of 1.03 on the worst
parameter, 0.91 on the interaction block and 1.09 on the hyperparameters,
with every per-configuration sign test above p = 0.47 and per-pair ratios
spanning 0.18 to 9.2. The conditioning says
why, without a sampler: cond(Q) is 1.3e5 to 2.1e5 and neither diagonal read
moves it (diag(Q^-1) reaches 1.1e5, 1/diag(Q) changes nothing), while
deleting the S + T soft sum-to-zero margin directions takes the same matrices
to 11.5 to 51. Those directions are 1_S (x) a and b (x) 1_T, so no
diagonal metric reaches them. A mass carrying them as a rank-(S+T) term
reaches 6.8 to 18.6, which is #597.
ModelData::has_spatiotemporal and
spatiotemporal() errors at the R door, so no test could reach the
Knorr-Held sampler path at all. src/test_st_iv_fixture.cpp fills a Type-IV
ModelData directly, the way a consumer package does, and exposes its
layout, log-posterior, mass override and a NUTS fit. Every claim about the
new metric is scored against the numerical Hessian of the engine's own
log-posterior over the block.Behaviour changes, in the order they are most likely to affect a fit.
A failed sparse solve is no longer reported as a converged fit (#510).
SparseCholeskySolver::solve() was void and zero-filled on failure, and
every Newton caller tested only that the step was finite -- a zero step is
finite and satisfies max|delta| < tol, so a null CHOLMOD solve came back as
a converged fit with the mode left wherever the iteration happened to be.
solve() returns a status and fills NaN, and the four Newton call sites
consume it. refill_from_dense() no longer discovers its sparsity pattern
from the first Hessian's VALUES, which let a coincidental near-zero in the
first cell fix the pattern for every later one; an off-pattern entry now goes
to the pattern-drop counter. The two Takahashi entry points validate their
documented preconditions and refuse an unusable pivot rather than continuing
past it. Consumers linking against the C API: tulpa_sparse_chol_solve fills
NaN, not zero, on a bad handle or a failed solve; no signature or struct
layout changed.
An exact-NUTS GP, SVC or multiscale-GP fit is now reproducible at a fixed
seed (#587). nngp_nc_backward's range-gradient loop ran a dynamic OpenMP
schedule while writing into the slot the thread number names, so the
summation order -- and with it the last bits of every draw -- was a property
of the run rather than of the data. Two runs of one call at the same seed
differed by 5.2e-04. The rows are now cut into contiguous per-thread chunks
summed in chunk order. Draws from a multi-threaded fit move once, to a value
that no longer depends on the thread schedule.
control$epsilon is honoured on the SGHMC and SGLD backends (#576,
#577). SGHMC enabled its warmup step-size adapter unconditionally and SGLD
never disabled its polynomial decay, so a supplied step size was accepted and
then overwritten -- every value produced the same chain. Both adapters now
run only when the caller names no step size. A call supplying none is
unchanged.
control$n_threads is refused on the sampler route rather than accepted
and dropped (#587). OMP_NUM_THREADS remains the way to cap the team.
The negative-binomial Gibbs kernels evaluate an exact density (#495).
Three different eta clamps and two probability floors are gone, replaced by
the closed form y*eta - (y + r)*log(1 + exp(eta)), branch-stable on the
sign of eta; the clamps could not be made consistent, because the
augmentation was drawn at the clamped predictor while the coefficient update
solved the conditional the unclamped one defines. Three undocumented
magnitude boxes on the coefficients, random effects and their scale are
removed, so the chain targets the posterior rather than a truncation of it;
the documented dispersion bound stays. With store_eta = TRUE the saved
linear predictor is the end-of-sweep state, where it was previously one sweep
behind the coefficients beside it.
Both negative-binomial kernels require an intercept (#497), where they
previously shifted eta by a per-sweep constant on a design whose first
column is not all-ones, and wrote out of bounds on a zero-column design.
The SPDE implicit-difference gradient carries its total-derivative term
(#583). H reads the mode through the likelihood curvature, so log|H|
depends on the hyperparameters twice and the envelope theorem does not cancel
the mode's own derivative out of the determinant. It now agrees with its
finite difference; it was 5-10% away.
Fixed-hyperparameter and sampler-side entry points validate the indices and
lengths R hands them (#500, #503, #504, #516, #521, #522, #529, #532, #556,
#559, #562, #564, #566, #569, #570). An out-of-bounds read on the R heap does
not crash -- it returns a finite double and the fit stops being a function of
its data -- so these are silent-wrong-answer paths, not crashes. Among them:
a gradient buffer sized from the wrong argument (an out-of-bounds write), a
random-effect group index with no upper bound anywhere, a covariance packer
reaching atanh on a NaN that passed both of its one-sided clamps, and a
GLMM oracle that dropped an out-of-range group instead of rejecting it and
fitted a subset.
A malformed ModelData raises instead of returning a log posterior
(#563). One entry point returned 0 and another -INFINITY; a zero log
posterior is indistinguishable from a valid flat one.
Areal priors: the ICAR-BYM2 log and square-root sites are guarded, so the double path and its autodiff twin no longer describe different models at the mixing boundary (#485); the areal input validator is wired into the sampler-side ingestion, not only the Laplace entries (#501); and the multi-block BYM2 entry rejects a mixing weight outside the unit interval, as its two sibling entries already did (#532).
Temporal priors: one stationary floor behind every AR1 arm, including three gradient sites that used a raw factor and so scaled by a negative number past the boundary (#514); the temporal-GP centered and non-centered branches read one conditional variance (#499).
GPU: the batched triangular solves read the wrong triangle and computed only the diagonal inverse (#478); device information is enumerated rather than reported from placeholders (#480). Unverified at runtime -- no CUDA device was available -- and the affected entry points have no production caller.
SVC: one prior on the marginal standard deviation across both bases, where the Hilbert-space branch previously selected a different prior from the basis flag (#487).
Documentation: gcol33/tulpa#NNN references are removed from code comments
and roxygen (#575), and eight comments describing code that is no longer
there are corrected (#568, #574, #525, #537).
The NUTS metric is reachable from the sampler front door (#545).
select_and_init_mass_matrix resolved AUTO, detected the mass blocks and
built the block-diagonal MassBlock machinery on a path no caller could
reach: every entry point but the SPDE one pinned MassMatrixType::DIAG, and
the SPDE layout carries none of the structures the detector looks for.
tulpa_sample_glmm(control = list(mass_matrix = )) now selects it --
"diag" (the default, so an existing fit is unchanged to the bit),
"dense", "block_diag" or "auto". Under "auto" the correlated
hyperparameter groups get their own small dense blocks (the BYM2 and GP
(log sigma, phi) pairs, the multiscale-temporal variances, a correlated
random-slope term's Cholesky coordinates) and an ICAR or latent-factor model
inside the DENSE_MAX_PARAMS ceiling takes a full dense metric.
tulpa_hmc::parse_metric_type is the one string-to-metric map and rejects an
unrecognised name rather than defaulting to DIAG; the SPDE entry point took
its own integer code and now comes through the same map. A backend with no
mass matrix refuses a non-default value instead of ignoring it.
ModelData vectors are checked before the RE layout indexes them (#546).
compute_param_layout drove the RE-slopes loop off n_re_terms and indexed
re_n_coefs, re_correlated, re_n_chol and re_n_groups_multi by the term
counter with no length test, and read log_sigma_re_slopes[t][0] on a term
with no coefficient. ModelData ships under inst/include, so a LinkingTo
package reaches all five without passing through the R-side validation. Each
is now checked once, by name. The ICAR mass warm start guarded its CSR row
pointer on !empty() while reading adj_row_ptr[s + 1] across the whole
spatial block, and now requires the length that read needs.
build_sampler_model_inputs validates the multiscale temporal
time_index / group_index against n_times / n_groups: an out-of-range
entry used to contribute zero through compute_temporal_eta's per-read
guards, so a mismatched index vector showed up as a quietly weaker temporal
effect rather than an error. compute_temporal_eta also checks time_index
against the n_obs its caller declares, and
cpp_test_multiscale_temporal_log_lik rejects a non-stationary rho_short
the stationary AR1 factor would otherwise absorb into its floor.
The unreachable mass-block machinery is gone (#547). PrecisionBlock and
KroneckerBlock were never initialised, and SparseGMRFBlock was
initialised, never factorised, then deactivated at warmup end -- so
has_structured_blocks() was false on every path and the branches it guarded
in the kinetic energy, the momentum draw, inv_mass_times_p and the leapfrog
drift were dead, including three per-leapfrog-step std::vector
allocations inside the file that describes itself as the zero-allocation NUTS
core (#548). hmc_sampler_mass_blocks.h drops from 865 lines to 395. The
ST_IV precision-informed mass override the deactivation was waiting on is
tracked in #585 rather than left as a half-wired struct a reader cannot tell
from a live one. The 13 #if GP_DEBUG_BOUNDS blocks in
hmc_gp_log_lik.h are gone with them: hmc_gp.h defined the flag as false
unconditionally, so no build flag could reach them, and each block that
checked anything sat behind an always-on check that returned first. The
GP_AUTODIFF_DEBUG scaffold read a call_count declared under a different
flag, so turning it on did not compile; it is gone too.
The multiscale GP density stopped copying its inputs per gradient
(#548). multiscale_gp_log_lik_t transcribed MultiscaleGPData into two
temporary GPData objects on every call -- every neighbour array copied,
nn_neighbor_dist included at n_obs * nn * nn doubles -- because the
density took a GPData. It is now written against the six arrays it reads,
and both the single-field and the two-scale entry points pass their own
members straight in. The dense covariance accumulator
(WelfordCovStats, n_params^2 doubles per chain) is allocated only for a
metric that starts DENSE, instead of on every chain of every fit.
Every OpenMP team goes through tulpa_omp_team_size (#550). The indexed
joint scatter took its thread count from R (.tulpa_inner_threads, keyed on
the core count), and the three exported OpenMP probes took theirs raw, so
none of them saw OMP_NUM_THREADS, OMP_THREAD_LIMIT or the two-core cap
R CMD check sets through _R_CHECK_LIMIT_CORES_. All four now request their
team through the shared clamp, as does parallel_block_reduce.
Header hygiene in the HMC layer (#549). Ten UTF-8 BOMs stripped, and the
comments a non-UTF-8 round trip had turned into bare ? restored to what they
said. available_ram_bytes() deallocates the send right mach_host_self()
hands it on every macOS exit path, instead of leaking one per call.
hmc_gp_nc.h says at the top that it is a namespace-body fragment included
inside namespace tulpa_gp and must not be included directly. Refactor
history, dated benchmark tables and a stale "known heisenbug" note that told
the reader to avoid the only GP density there is are out of the code.
The correlation Cholesky covers the whole cone (#431, #443). The raw
parameters of a correlated random-effect term were mapped by
L[i, j] = tanh(raw) directly, which bounds each entry but not the row: at
n_coefs >= 3 a row's squared sum can exceed 1 and the map covers a strict
subset of the correlation-Cholesky cone. The three copies of that build then
disagreed about what happens outside it -- the HMC prior returned -Inf, the
LKJ helper returned false, and the Laplace spec solver clamped the squared
diagonal at 1e-12 and carried on, producing rows of norm above 1 and a
Sigma whose diagonal no longer matched the marginal SDs it was handed. Which
of the two a user got depended on the backend rather than on the model.
tulpa::build_L_from_raw is now the one build, templated over the scalar so
the AD paths differentiate what the double paths evaluate, and it takes
tanh(raw) as a canonical partial correlation scaled by the room the row has
left: every raw vector lands in the cone, so there is no support boundary left
to disagree about. tulpa::raw_from_L is its exact inverse and is what
pack_to_spec_re_params uses. The log-Jacobian gains the scaling term
0.5 log(s) alongside the tanh term; the LKJ exponent on log L[k,k] is
unchanged. At n_coefs == 2 the scaling factor is 1 and its Jacobian term is
0.5 log(1), so the two maps coincide term for term and a random
intercept-and-slope fit is unchanged.
Type-IV spatiotemporal rank is derived once (#438). The centered branch
counted the adjacency's connected components; the non-centered branch
hardcoded the spatial rank as S - 1. rank_space multiplies log(tau_st)
in the GMRF normalizer, so on a graph with more than one component -- islands,
or any unit its adjacency file leaves with no neighbours -- the two
parameterizations of the same model targeted different densities and reported
different tau_st posteriors, with nothing flagged. st_spatial_rank() is
now the one derivation, called from the non-centered Type IV, the centered
Type IV and Type III. The HSGP-ST branch also checks M * T_st against the
latent span it was sized from before indexing.
Areal entry points validate their adjacency (#441). adj_col_idx indexes
a WRITE into the dense Hessian (H[sp_idx][spatial_start + neighbor]) and
spatial_idx indexes the latent field, both through raw pointer arithmetic
with no bound, and neither was checked. tulpa::check_areal_inputs()
(src/areal_input_check.h) checks the CSR shape, monotonicity, tail and
column range plus the site index's length and range, and is called by
cpp_laplace_fit_spatial, cpp_laplace_fit_bym2, cpp_laplace_fit_car_proper
and the five nested areal / ST entries. An out-of-range value is now an R
error naming the entry point and the offending index.
Grid checkpoints fingerprint and serialize what they claim to (#442).
re_idx is an Rcpp::NumericVector and was folded at sizeof(int), so only
its leading half reached the hash: two RE assignments agreeing there shared a
fingerprint and a resume replayed one run's cells under the other's data.
Fingerprint::fold_rvec() derives the element size from the R type, so the
mismatch is not writable at a call site. The SPDE nested entry's offset is
now folded as well (make_nl_grid_checkpoint takes optional offset /
weights, and an absent one folds distinctly from a present zero). The
per-cell record now carries every LaplaceResult field rather than eleven of
them, so a resumed cell no longer comes back with an empty skew / importance /
debias / CILA payload that the grid reads as "did not compute it";
CheckpointLog::MAGIC is bumped accordingly. start_infeasible is carried
per cell through the grid merge and read by nl_grid_cell_to_result_list(),
which hardcoded false for every one-cell export.
Stochastic Lanczos quadrature: breakdown, orthogonality, bounds (#444). A
breakdown (beta_j below tolerance) is the case where the Krylov space is
exhausted and the quadrature is exact, and it was reported as
"matrix is not positive definite": alpha / beta stayed padded with zeros
out to n_lanczos, so the eigensolve returned Ritz values at 0 and the
definiteness guard fired on them. lanczos() now returns the steps actually
taken and resizes to them, and the guard skips a Ritz value carrying no
quadrature weight. The recurrence gains the full reorthogonalisation the cited
reference (Ubaru, Chen & Saad 2017) assumes, without which the basis loses
orthogonality within a few steps and log|H| picks up a bias on the
N > 100K path that has no Cholesky to cross-check against. The CSC triple
from R is validated before anything indexes off it.
Three AD primitives agree with their double counterparts (#446).
fwd::expm1 computed exp(x) - 1, the cancelling form, which is exactly what
log1m_exp_fn's small-argument branch routes through expm1 to avoid -- a
zero-truncated Poisson at small mu is both the regime where the truncation
correction matters and the branch that was taken. safe_log returned partial
1/1e-15 on the two reverse paths and 0 on the forward one in the clamped
region, where the value is a constant and the derivative is therefore 0; all
four now report 0 there. log_sum_exp gained the double core's non-finite
guard on all three AD types, so two mixture components that both underflow
give -Inf rather than a NaN in the adjoint buffer, and the six hand-written
copies of that reduction in autodiff_utils.h are one log_sum_exp_fn<T>().
fwd::pow(Dual, Dual) no longer forms 0 * log(0) for a constant exponent.
ad::Var gains the comparison operators its two sibling AD types already
carry, without which a templated helper that branches on its argument compiles
for three scalar types and not the fourth.
A failed checkpoint truncation is an error (#449). resize_file's
std::error_code was never read, and the append stream opens either way, so a
failed truncation left every record the run appended sitting past an orphaned
torn tail -- written, invisible to the next resume, and recomputed with no
message.
cpp_laplace_sample no longer ridges the caller's matrix (#451). Rcpp
binds a REALSXP argument without duplicating it, so the uniform ridge was
written into the R matrix the caller still held, and a second call on the same
matrix sampled from a precision carrying it twice.
Multi-chain stochastic-sampler results guard the log_lik copy against a
short vector the way the VI/ESS shim already did (#439).
The BYM2 outer grid checks its mixing-weight axis (#421).
cpp_nested_laplace_bym2 fixed the cell count from sigma_spatial_grid
alone and never looked at rho_grid again, where every sibling paired-grid
entry in the same file checks the pairing. A shorter rho_grid was read past
the end of the R vector for every cell beyond its length, and a rho outside
[0, 1] took sqrt(1 - rho) to NaN, which reached the inner Newton through
eta and returned a NaN cell instead of an error. All eight entry points now
go through one nl_grid_axes_length(), which names the axis that does not
pair and the length it has, and a mixing-weight axis is checked against its
own support. The 1e-10 inside the two square roots is tulpa::BYM2_RHO_EPS
in src/bym2_mixing.h, one value for the Laplace kernels and the
Polya-Gamma samplers, with what it costs at the endpoints written down.
The Polya-Gamma kernels size their draw matrices for the number of saves
the loop makes (#426). n_save = (n_iter - n_warmup) / thin truncates,
while the save condition (iter - n_warmup) %% thin == 0 fires
ceiling((n_iter - n_warmup) / thin) times, so a thin that does not divide
the post-warmup run wrote one row past the matrices. NumericMatrix's
operator() is unchecked and its offset is j * nrow + i, so the extra write
landed on the first saved draw of the next column for every column but the
last and one element past the allocation for the last -- a plausible finite
number either way. The two negative-binomial kernels now use the shared
pg_n_save(), which also refuses thin < 1 and n_iter < n_warmup, and
tulpa_gibbs() forwards thin to the spatial and temporal routes, which used
to drop it silently.
Tests for the seven audit items the accompanying src/ work had already
landed: the ICAR full conditional against a replica drawing from the same R
stream, with the zeroing sweep as a negative control (#423); batch_nngp_scatter's
entry contract and its left-packed neighbour scan (#425); the corrected
integrated Laplace on a parallel outer grid, pinned to the serial fit
(#424); the joint checkpoint fingerprint against a changed adjacency and a
promoted copy block (#419); the joint finite-difference gradient gate on one
block per contribution kind through the sparse scatter fits actually run, plus
dense-sparse equivalence on a pure INDEXED_SINGLE spec (#420); and the
posterior SCALE that SGHMC and SGLD sample, which is what the noise-scale
defect left wrong while every existing backend test read the mean (#422).
test-nngp-prior-scatter.R's conditional-variance reference conditions on
C + kNngpNugget * I, the matrix the kernel factorizes, read off the
scatter's own return rather than written into the test (#578).
One default fixed-effect prior, resolved at the front door (#408).
tulpa() substituted sd = 2.5, 10 or 100 on the fixed effects
depending on which backend mode = "auto" selected, so y ~ x and
y ~ x + (1 | g) on the same data were fitted under priors a factor of 40
apart, and nothing on the fit recorded which one ran. The default is now the
documented prior_normal(0, 2.5) everywhere, defined once in .TULPA_PRIOR
with a .PRIOR_CONSUMERS table naming each fitter that reads it, resolved
before backend dispatch and passed down. summary() and the fit itself
report the prior that was applied as $beta_prior. The nested-Laplace and
SPDE paths are unchanged: they hold their own field-conditional prior and
reject a supplied beta_prior.
The AD scalar primitives are guarded at the same boundaries as the value
path (#409). The templated log posterior is instantiated for double and
for two reverse-mode Var types, and the runtime gradient check compares the
double instantiation's central differences against the arena gradient as
though the two were one function. They were not: exp clamped its argument
on the value path and not on either AD path (so a log-scale hyperparameter
past 709 returned +Inf and took the backward pass with it), and sqrt gave
a +Inf partial at exactly zero where the value path gives a finite 0 -- an
HSGP spectral density underflows to zero at a long lengthscale and a high
basis index. arena::pow and arena::logit (and their tape twins) carry the
same treatment at a zero base and at the endpoints of the unit interval. The
clamp bounds are one named constant in tulpa::math.
verify_gradient_runtime() reports a non-finite gradient instead of
passing it (#410). Every comparison against a NaN is false, so the
worst-relative-difference scan stepped over exactly the failure the check
exists to catch and returned true; NUTS then ran on NaN momenta for the
whole fit. The finite-difference step is now scaled by each coordinate's own
magnitude, and the two duplicated difference loops are one helper.
A beta_prior a fitter cannot express is an error, not a silent swap
(#411). .beta_prior_ridge_sd() replaced a prior with no sd field -- a
prior_half_cauchy(), a prior_pc(), a misnamed list -- by its default, so
tulpa(..., mode = "ep", beta_prior = prior_half_cauchy(2.5)) fitted under
a different prior and reported nothing. Both resolvers now validate through
one .beta_prior_fields(), so a prior one accepts is a prior the other
accepts.
tulpa_variogram() checks coords against the residual length (#412).
A mismatch recycled a logical index and returned a well-formed semivariogram
built from squared differences of unrelated pairs. moran_i() and
tulpa_variogram() share one check, and coincident coordinates report the
empty distance axis rather than failing inside seq().
The generics tulpa borrows from bayesplot and rstantools dispatch under
either attach order (#413). pp_check, bayes_R2 and posterior_predict
are owned by those packages; tulpa's copies masked them, so
pp_check(y, yrep, fun) failed for the rest of the session once tulpa was
attached. .onLoad() now registers the tulpa_fit methods on the owners'
generics and borrows the owners' own .default methods onto tulpa's, so
neither direction depends on attach order.
The Polya-Gamma kernels validate every R-supplied index at entry (#415).
Group, spatial, time and neighbour indices were converted to 0-based and used
as raw subscripts, which Rcpp does not bounds-check, so an out-of-range
spatial_idx was a read or write past the end of an R allocation. All eight
entry points check length, NA and range through one helper, and the
neighbour list is checked against n_neighbors before either is walked.
The compiled test harness is reached (#416). 55 of the 238 generated
entry points had no caller, 48 of them in src/test_helpers.cpp -- including
every autodiff-primitive probe and all three serial-versus-parallel
equivalence helpers, so from the outside the package read as though that
coverage existed. They are now wired into test-autodiff-primitives.R,
test-cpp-kernels.R and test-parallel-equivalence.R, each against an
independent R computation. cpp_pg_negbin_negbin_gibbs and its driver are
removed: a numerator/denominator ratio sampler belongs to tulpaRatio, which
carries its own copy.
temporal_rtr() and spatiotemporal() / spatiotemporal_gp() refuse to
build a specification no fitter reads (#407, #414). Both returned an object
that flowed through tulpa() as an unrestricted model, or errored about an
unrelated field. fit_st_nested() is the spatiotemporal entry point.
First CRAN release. The engine's surface is unchanged from 0.0.198; this version number marks the release rather than a feature change.
1e-2 on the log scale (#403). At 5e-2 the step's truncation
error exceeds the reduction L-BFGS-B's line search chases near a flat
optimum, so the search can reach the correct mode and then abort, returning
a nonzero convergence code with the best point still in op$par.
bad_mode() reads that code as an unusable mode, and
fit_spde_nested_ccd() falls back to the rectangular grid, so the CCD
design silently does not engage.Measured on test-spde-ccd.R's analytic fixture across 1e-4 to 5e-2:
every step up to 2.5e-2 returns convergence 0 on Linux and Windows alike
and agrees bit for bit, while 5e-2 at factr = 1e5 is the one cell that
converges on one platform and aborts on the other, at a mode both platforms
agree on to 11 significant digits. On the real inner-Laplace marginal the
smaller step reaches the same mode in the same 13 evaluations at a lower
objective (505.675740 against 505.675808), so the wider step was costing
accuracy at the optimum.
control knob, and every L-BFGS-B stop
tolerance in the package is set in one table. fit_spde(control =
list(mode_find = list(factr =, ndeps =, maxit =))) overrides any subset of
the tuning for the (range, sigma) mode-find under method = "ccd"; an
unknown sub-knob or a non-positive value hard-errors rather than fitting the
default, matching what tulpa_check_control() gives the outer list.The values move to .NL_MODE_FIND (R/settings.R), which records why the
consumers differ, and the stats::optim() invocation itself moves to
.nl_lbfgsb_mode_find() (R/mode_find.R), so fit_spde_nested_ccd() and
fit_st_nested()'s auto-grid rescue share one call site instead of carrying
a copy each. tulpa_pathfinder() reads its factr from the same table
without sharing the helper: it is unbounded, takes an analytic gradient when
the caller supplies one, and carries maxit / pgtol as its own arguments.
Behaviour is unchanged on every path. The spatiotemporal rescue previously
inherited optim()'s own ndeps default, and test-mode-find.R pins that
the value now written out is that default by running both calls and comparing
par, value, convergence and hessian; the same file checks the helper
against the call it replaced for each consumer, and test-settings.R lints
the sources so a new factr or ndeps literal outside settings.R fails.
test-sbc-frontdoor.R opens its device with pdf() (#404). png() selects
whatever getOption("bitmapType") names, which resolves to the X11 driver
on a headless machine and cannot start there even where
capabilities("png") reports TRUE. Nothing in that block reads raster
output.
Renamed the local T bindings in spatiotemporal_effects() and
plot.tulpa_st_summary(), and the T field of the (internal, unread)
precision_structures list, to n_t / n_times. T and F are not
reserved words in R.
Intra-chain OpenMP teams now honour _R_CHECK_LIMIT_CORES_, capping at two
threads under R CMD check. The cap lives in tulpa_omp_team_size(), so
every team-size decision inherits it, and cpp_get_max_threads() reports the
size a region would actually get.
An outer axis whose grid does not contain its own posterior mode now says
so (#401). h / sd is the regime variable the reported hyperparameter
interval's position sensitivity is governed by, and its SD side is a 3-point
parabola at the axis's modal cell that withheld a bare NA on five
distinguishable conditions. One of them, the mode sitting on an END node, is
not a missing measurement: it says the grid brackets nothing on that axis.
Three consequences, all of them measured on the package's own four-axis joint
fixture at axes_at(5), where b1.alpha holds 57.7% of its axis mass on its
lowest node (2.88x the uniform share). The whole-grid verdict was read off the
axes that happened to score, so resolved could come back TRUE with an
unscored axis present; the note named the coarsest SCORED axis, which on that
fixture is a different, healthy one, and told the reader to add nodes there;
and .nl_railed_axes() -- which finds b1.alpha:lower instantly from stored
weights -- was attached only inside the registry grid rescue, which does not
run on a caller-pinned grid, so the placement the engine leaves alone by
construction was the one that never reported it.
Now: .NL_AXIS_SD_REASONS is a closed vocabulary (too_few_nodes,
mode_at_edge, coord_not_finite, stencil_degenerate,
curvature_not_negative) carried on the NA as an attribute, so every existing
is.finite() caller is unaffected; it is stamped through as
outer_grid_resolution_declined, per axis, beside the existing
theta_within_cell_declined / theta_cell_edge_declined; resolved requires
every axis to have scored; the rail report is attached wherever the resolution
is; and the note reports unscored axes and railing BEFORE the coarsest scored
one. A fit where nothing scored reports that nothing could be, rather than
returning NULL as if it carried no resolution at all.
_pkgdown.yml's figures: block is removed (#400). dev: svglite named a
package rather than a device function, and qualifying it as
svglite::svglite only moved the failure: pkgdown:::fig_save() sets
dev.args$bg <- NA unconditionally and svglite rejects a logical NA there.
The block's other two keys were already inert -- bg is a formal of
fig_save() that its body never reads. The default ragg::agg_png renders
the reference examples cleanly.
pkgdown/build_site.R is the pkgdown deploy entry point (#402).
pkgdown:::package_mds() renders every root-level *.md minus a hard-coded
exclusion list with no config knob to extend it, so CLAUDE.md, AGENTS.md,
todo.md and api.md were published verbatim and AGENTS.html was the first
entry in the site search index. The script holds them out of the root for the
duration of the build and asserts they reached neither docs/ nor
search.json.
tulpa_re_aghq() returns the mode/theta cross-Hessian block (#398).
Adds blup_cross (Bf) to the return value: the per-group -d^2 ell_g /
d theta db block at the mode, needed to draw a group's BLUP jointly with
theta instead of independently (Cinv %*% t(Bf) is the first-order
db_hat/dtheta correction) -- the same joint-draw contract
.tobs_community_em()-based tulpaObs families already expose via their own
Cinv/Bf. Computed via a cheap central finite difference of
theta_score over b (O(d) oracle calls, independent of n_theta).
Declines to NA (never a silent 0) when the oracle's theta_score has no
analytic implementation, as on the R-closure bridge (make_site/
make_group) -- REGroupOracle gains has_theta_score() to signal this.
Verified against the closed-form binomial-GLMM cross term and an
independent finite-difference re-solve of the group's mode under a
perturbed theta. Unblocks gcol33/tulpaObs#220 (ms_abun()'s posterior-SBC
registration).
tulpa_re_aghq() also returns the FULL per-group joint covariance
across RE terms (blup_cov_g, blup_cross_g). blup_var only ever
exposed the per-term diagonal of a group's posterior covariance; when a
group carries more than one RE term sharing the same grouping factor (e.g.
an abundance-arm term and a detection-arm term on the same species), the
group's mode is found jointly across every term's coefficients, so real
posterior covariance can exist BETWEEN terms -- cpp_aghq_blups() already
inverts the full joint Hessian to get it, it just discarded everything off
the diagonal before this. Drawing a group's terms independently would
repeat gcol33/tulpaObs#226 one level deeper (inside a group instead of
between theta and a group). Validated against a closed-form joint-Hessian
construction on a toy model with deliberately collinear RE terms; the new
diagonal is byte-identical to the pre-existing blup_var.
run_multi_block_nested_laplace_joint_batch's dense branch (small/medium
fields, n_x < SPARSE_THRESHOLD) solved every Newton step and the final
log-determinant through the fixed-ridge-only dispatch_factor_solve /
dispatch_factor_log_det, instead of the PD-escalating
joint_pd_step_solve_dense (the gcol33/tulpa#344 fix) the single-species
dense joint driver and the batched driver's own SPARSE branch already use.
A coupled likelihood whose observed Hessian is indefinite away from the mode
(the occupancy mixture's dark-cell term is not concave everywhere) could hit
a negative pivot at one outer-grid cell; with no ridge escalation the
Cholesky's sqrt produced NaN, silently dropping that cell's weight from
the fit and shifting every reported summary. Confirmed data-dependent (not
batch-slot-dependent): the same species diverged identically whether alone,
paired, or duplicated against itself, and grid cells before the affected one
matched the independent fit to machine precision. Fixed by routing both the
per-iteration dense solve and the final-pass log-determinant through
joint_pd_step_solve_dense with JointPDMode::LM, mirroring the
single-species driver and the batch driver's own sparse branch exactly.The simulation-based-calibration and goodness-of-fit entry points are S3
generics, so a downstream package registers a method on its own fit object
instead of exporting a second name for the same concept. sbc(),
pit_residuals(), test_dispersion(), test_outliers() and
test_zero_inflation() now dispatch; every existing body became the
default method, and sbc("prior_predictive", ...) / sbc("posterior", ...)
keep working unchanged through sbc.character(). A call passing the
experiment BY NAME (sbc(experiment = "posterior")) has to drop the name or
use object =, since the first argument is what dispatch reads.
dic() and cpo() are new generics over the same criteria layer
[tulpa_criteria()] computes, for the two criteria the loo package owns no
generic for. The default methods take a pointwise log-likelihood matrix. WAIC
and PSIS-LOO deliberately get none: loo::waic() and loo::loo() already
exist, and a same-named generic here would mask them.
test_uniformity() muffles only the ties warning a discrete response
produces, rather than every warning the Kolmogorov-Smirnov test can raise.
A reported hyperparameter bound that left the node range now says so
(gcol33/tulpa#390). An endpoint past the outermost NODE is produced by the
outside rule -- extend mirrors a half-cell beyond the outer coordinate --
so it is an extrapolation rather than a bound the design supports.
theta_ci_outside_nodes records it per axis ("lower" / "upper" /
"both" / NA), the same rule that makes a declined placement say so
(gcol33/tulpa#293). Recorded rather than corrected: the two used to be
indistinguishable on the fit.
The recentred axis's mode-SD ceiling is settled: max_sd_u = 3 is kept, on
evidence. gcol33/tulpa#387 could not score its VALUE because every ladder
came back flat or non-monotone. The reason was the LEVEL, not the fixture.
Over 48 (cap, span, node-count, clamp-policy) rungs on two ceiling-reaching
fixtures at 200 seeds, the reported bound leaves the node range on 56-90% of
fits at nominal 0.95 at EVERY setting and on 0% at nominal 0.50 -- so 0.50 is
the level whose bound the design supports, and 0.95 is where the earlier
ladders were measuring the extend rule instead.
At nominal 0.50: the cap is EXACTLY inert at nine nodes (1.5, 3 and 6 give
identical coverage to three decimals in 7 of 8 cells); at the shipped five
nodes, 3 is nearer nominal than 1.5 at span = 4 (0.450 / 0.415 against
0.370) and ties at span = 2.5; and 6 reaches 0.520 only on a doubled width.
Under the shipped sd_clamp_policy = "decline" a lower ceiling also is not
free -- dropping to 1.5 abandons the placement on 32-39% of fits. span and
n_pts are kept with it: nine nodes moves 50% coverage FURTHER from nominal
while costing proportionally more inner solves.
cuda_batched_cholesky and its siblings were defined TWICE, differently:
gpu_backend.h compiled stubs returning FALSE in the #else branch of
#ifdef TULPA_ENABLE_CUDA -- which neither Makevars ever defined -- while
gpu_nngp_laplace.h included gpu_cuda.h directly and compiled the real
ones. Two inline definitions of the same entity across translation units is
an ODR violation: the linker keeps one COMDAT and discards the rest, so
whether CUDA ran at all was decided by link order rather than by any switch,
and nothing in the package could report which had been built.gpu_cuda.h is now included from exactly one place, and the NNGP kernels
reach it through gpu_backend.h like everything else. Compiling it in needs
no CUDA SDK at build time and no GPU at run time: the driver, cuBLAS and
cuSOLVER entry points resolve dynamically and every entry returns FALSE
when absent, which is what makes "use CUDA if available" expressible as a
default. TULPA_DISABLE_CUDA builds the stubs instead, and that is now a
whole-program choice.
cpp_gpu_backend_kind() reports "cuda" or "stub", so which
implementation was compiled is observable rather than inferred -- a silent
either/or is what let this sit. It is deliberately separate from
cpp_gpu_available(), which asks whether a usable device is present at run
time.
Two closed investigations should be re-read in this light: gcol33/tulpa#283 fixed a column-major/row-major bug in this path, and gcol33/tulpa#389 ruled the GPU dispatch out as a cause -- a conclusion consistent with the stub having won the link. The per-matrix batch verification added in 0.0.192 (gcol33/tulpa#392) is what now guards the path on machines where it does run, and the gcol33/tulpa#389 determinism arbiter passes with the CUDA backend live on a device-equipped machine.
LatentBlock::d_fac is read through one accessor that carries its
contract (gcol33/tulpa#394). Twelve call sites read this grid-dependent eta
mixing coefficient and they disagreed: eight invoked it directly and four
guarded it with d_fac ? d_fac(k) : 1.0, on the SAME block vector. Only one
can be the contract, and it is the unguarded one -- all 38 block-construction
sites set d_fac unconditionally, and LatentBlock is not an exported
header, so no consumer package can construct one without it. d_fac_at() now
states that in one place and throws a named error instead of an uncaught
std::bad_function_call; a block that ever did omit it would previously have
been silently amplitude-1.0 down one path and a crash down another.
The obs_indices scratch buffer is cleared by the caller, once
(gcol33/tulpa#395). The contract asked IMPLEMENTATIONS to clear, and four of
eleven call sites cleared defensively anyway while seven relied on it. The
buffer is static thread_local and reused across the whole observation loop,
so an implementation that appended would have scattered every stale
(index, weight) pair into eta as though it belonged to the current row,
growing with position in the loop -- a silent wrong answer rather than a
crash. fill_obs_indices() clears and dispatches, so neither side can forget
and a new block kind cannot get it wrong.
CudaContext::initialize() is serialized (gcol33/tulpa#393). It was an
unsynchronized check-then-act: two threads both observing initialized_ ==
false would both load the CUDA libraries and function pointers into the same
members and call cuInit twice. The static's construction is thread-safe;
its initialization was not. Latent rather than live today -- every entry that
can build an NNGP block passes n_threads_outer = 1 as a hardcoded literal,
and the one entry taking it from R has no NNGP branch -- but the outer grid
is an OpenMP parallel-for and the block prep that reaches this deliberately
runs outside its critical section.
batch_nngp_scatter hands the neighbour covariances to
cuda_batched_cholesky() once the batch reaches 50 and then checked the
result by refactorizing ONE matrix on the CPU and comparing. That catches a
whole-batch failure -- which is what it was written for, after a cuSOLVER
column-major factor was read row-major (gcol33/tulpa#283) -- but not a
per-matrix one, and the call documents that it can fail partway. A probe at a
fixed index accepts every failure beginning after it, and since the matrices
have different effective sizes it is not representative of the batch either.Every matrix is now checked against the factorization's own defining identity
on the diagonal, sum_k L[j][k]^2 == C[j][j], plus a positive finite pivot.
That is O(n_nb^2) per matrix against the O(n_nb^3) the factorization
costs, so full coverage sits an order below the work it verifies. The exact
one-matrix CPU cross-check stays alongside it as the layout check.
gp(approx = "nngp") takes the coordinate dimension you give it
(gcol33/tulpa#391). 0.0.190 made the NNGP neighbour COVARIANCE read every
coordinate column; compute_nngp_neighbors() still selected neighbours over
the first two, so the selection and the covariance would have used different
metrics on anything wider, and a 1-D domain could not build a graph at all --
which put the dimension-general kernels out of reach from the front door. The
ordering, the candidate distances and the cached neighbour-pair block are all
read over ncol(coords) now, and the pair block comes from one dist() call
rather than a hand-rolled double loop. So a transect, a depth profile or a
time axis is a 1-D NNGP, and a depth-resolved domain is a 3-D one.The 2-D path is unchanged to the bit: the same neighbour indices and
distances, asserted at tolerance = 0 rather than approximately, and a
CONSTANT extra column reproduces the 2-D graph exactly.
spatial_gp(), spatial_multiscale() and spatial_svc() carried three
verbatim copies of the same formula/character parsing block.
.parse_coord_spec() is now the single body, with the arity as its one policy
argument -- and the policy is decided by what the spec ends up in rather than
by taste. NNGP reads any dimension >= 1; approx = "hsgp" and every sampler
mode take exactly 2, because both store coordinates at a fixed 2-D stride, and
they say so with the dimension they were given.laplace_core.cpp), the
batched builder (gpu_nngp_laplace.h) and the PG-Gibbs sweep
(pg_shared.h) -- each formed the neighbour-to-neighbour distance by hand
over coordinate columns 0 and 1, and read column 1 UNCONDITIONALLY. On an
n x 1 coordinate matrix that offset is 1 * nrow + i, which is n doubles
PAST the end of the matrix's own allocation, so the neighbour covariance was
built from whatever the R heap happened to hold behind it. Nothing crashed and
no value looked odd -- the read lands inside the heap and returns a finite
double -- so the fit simply stopped being a function of its data and moved
with the process's allocation history.MEASURED before the fix, the same seeds fitted twice in ONE process at
n_threads = 1: 10 of 20 fits differed at 49 locations with log_marginal
moving 4.09, and 20 of 20 at 50 and 60. Which SIZES broke moved between
sessions -- 30 and 60 broke and 120 was clean in a run where the original
report had 30 and 60 clean and 120 broken -- so the size-dependence the first
reading proposed was the allocation history, not a buffer edge. It was never
the batched-Cholesky dispatch either: 49 locations is on the CPU path and
broke.
The three loops now share one helper, tulpa_linalg::coords_dist()
(src/linalg_fast.h), which sums over EVERY column the coordinate matrix
carries. So the coordinate dimension is whatever the caller supplied, and a
1-column matrix is a 1-D domain -- a transect, a depth profile, a time axis --
rather than a 2-D one with a column missing. Three coordinate columns work on
the nested-Laplace path for the same reason. The arbiter is that a CONSTANT
extra column, which cancels in the distance, now reproduces the 1-D fit
bit-for-bit at two different constants.
GP field prediction reads the same metric the fit was built on.
cpp_gp_field_predict() computed both its distances over two columns while
the fitting path is now dimension-general; leaving it pinned would have made
prediction disagree with the fit on any other width, which is a silent metric
mismatch rather than an error. It goes through the same helper and requires
only that the prediction and fitted coordinates have the same number of
columns.
A path that cannot store a coordinate refuses it instead of misreading it.
Every sampler spec stores coordinates in a flat buffer at stride 2
(GPData::coords and its siblings), a layout shared with the samplers and the
ABI, so those sites are 2-D by construction and are not made general. They now
call tulpa_linalg::require_coords_2col() and error with the arity they were
given. The same guard covers the HSGP 2-D basis and the NNGP twin probes.
The R side stopped silently reshaping coordinates to two columns. Four
sampler specs and two prediction paths passed their coordinates through
matrix(as.numeric(x), n, 2), which does not CHECK the arity -- it IMPOSES
it. An n x 1 matrix is recycled so that column 2 equals column 1, putting
every location on the diagonal and scaling every distance by sqrt(2); an
n x 3 matrix is truncated to its first two columns. Both are a different
geometry accepted in silence. .coords_2col() now errors on either, and
.coords_plain() strips attributes without touching the shape for the paths
that read the dimension.
Measurements taken on a 1-D-coordinate NNGP fixture predate this. The
nngp_120 row of dev_notes/issue361/RESULTS361EXT.md and the mode-SD
ceiling evidence in gcol33/tulpa#387 were both taken on such a fixture and
were reading out-of-bounds heap. Neither conclusion rested on that row --
gcol33/tulpa#361's default is carried by five other reproducible
configurations, and gcol33/tulpa#387 kept max_sd_u = 3 explicitly because
that row is inadmissible -- but the row itself should be re-measured before it
is cited (gcol33/tulpa#390).
log axis mode +/- span * max_sd_u is exp(+/- 7.5), a factor of 1808
either side of the mode, and the reported interval is read off that span. The
fit now records outer_grid_recenter_declined = "sd_ceiling_unresolved",
which is the gcol33/tulpa#293 rule: a placement the engine declines to make
has to say so rather than be indistinguishable from one that was not needed.
control does not carry this; .NL_RECENTER$sd_clamp_policy is the single
place it is set, and "clamp" restores what shipped through 0.0.186.MEASURED over 200 fixed-truth seeds on each of six configurations x two
placement policies, arms paired seed by seed and differing only in this
setting. Summed |coverage - nominal| over nominal 0.95 / 0.80 / 0.50 at the
shipped placement, and the paired 95%-level record against "clamp":
"decline" 0.1393 against 0.1464, changing 35 trials, winning 7 and
losing none (sign test p = 0.0078) at an unchanged width (ratio 1.0000).
A third policy, "relative" -- cap the re-placed span by the INCOMING axis's
own span rather than by an absolute bound -- scores 0.1536 and loses 7 trials
to 1, and is kept as a selectable arm rather than shipped.
The FLOOR keeps substituting, and it is the same table that says so. A
clamped floor WIDENS a too-narrow axis, which is the direction that cannot
rail, so substituting there is right and declining costs 22 trials against 9
(sd_floor_policy = "clamp"). min_sd_u = 0.15 is a minimum of its ladder in
both directions -- dropping it to 0.05 loses 374 trials and wins none, raising
it to 0.30 loses 30 and wins none. The floor is also the bound that actually
binds: it engages on 3 of 7 rows and on every fit of those rows, where the
ceiling reaches 2 of 268 axis reads.
max_sd_u = 3 is KEPT, measured rather than left alone. Across a ladder
spanning a factor of 15 (0.4 to 6) the summed coverage deviation moves 0.2843
to 0.3071 and the mean 95% width 1.986 to 2.120, and 3 is the best rung on
calibration. Fixtures built to REACH the ceiling -- an iid design shrunk
until 27.5% of raw mode SDs pass 3 -- return a NON-MONOTONE coverage response
whose driver is not the cap: no axis rails at any rung, and the reported bound
lies outside the node range on ~89% of those fits, identically at 0.8 / 1.5 /
2 / 3. What moves across that ladder is where the outer-cell extrapolation
lands. The earlier reading that the ceiling produces 95% widths in the
hundreds came from the one row that reaches it, nngp_120, whose fits are not
reproducible (gcol33/tulpa#389).
Every recentred fit now says what its axes were laid from. Per moved axis,
outer_grid_recenter_sd_clamp ("none" / "floor" / "ceiling"),
outer_grid_recenter_sd_raw (what the stencil measured) and
_sd_used (what the axis was laid from) -- the raw SD is recorded even for an
axis the policy declined, since that is the reading which says whether
declining was right. diagnostics() names a clamped axis in its placement
note. The clamp is applied in exactly one function,
.nl_recenter_sd_clamp(), behind all four rescues -- the spatiotemporal
driver's hand-inlined copy of the same two bounds included.
A bound-decline is PER AXIS on the multi-axis drivers. The spatiotemporal
rescue re-places (tau_spatial, tau_temporal, rho) together, and declining
the whole pass because ONE axis hit a bound discards the placement of the axes
the mode-find did resolve -- on the ar1 fixture, both precision axes thrown
away because rho alone was unresolvable. Each axis now keeps its own
incoming nodes and outer_grid_recenter_sd_declined names which axes did so
and on which bound, so a partially re-placed grid is not read as a fully
re-placed one. With every free axis declined the pass reports the grid as the
fixed one it still is. A decline for any other reason is a failure of the
mode-find itself and still takes the pass down.
control$within_cell = "chord" restores the previous
report per fit, exactly; point estimates, moments, draws and weights are
untouched either way.The construction has been selectable since 0.0.175 and the default was held back by two things, both of which the placement work in 0.0.187 changed. Until gcol33/tulpa#361 the default axes were laid without reference to the posterior, so every earlier measurement of this choice was taken on a grid pinned coarser than any a user now gets. Re-measured at the placement the engine ships, summed |coverage - nominal| over nominal 0.95 / 0.80 / 0.50, chord against box-uniform: 0.2900 against 0.1233 on gcol33/tulpa#337's own pre-registered fixed-truth instrument (300 seeds), 0.2004 against 0.0361 over the 4680 truth-swept fits of the same fixture whose axis contained the truth, and 0.2467 against 0.1572 over nine (config, axis) rows spanning seven families at 200 seeds each. Box-uniform is nearer nominal on 6 of those 9 rows and at all three levels of the other two, at 0.69 to 1.08x the width.
The conditional-coverage sensitivity that held the default back is a property of ANY within-cell reconstruction, and at the shipped placement it no longer separates the two: box-uniform's realized 95% coverage swings 0.110 across a cell where it swung 0.415 on the coarse pinned grid it was measured on, and at nominal 0.50 the two reads swing 0.238 and 0.231. A resolution-conditional default was scored rather than assumed and is dominated by the fixed rule (best threshold 0.1733 against 0.1572).
The regime it is weakest in is recorded rather than left to be rediscovered: on a coarse grid PINNED by the caller with four crossed blocks and no placement pass, box-uniform wins one resolution and loses the next (per-axis summed deviation 0.5083 against the chord read's 0.6500 at four levels, 0.6875 against 0.5917 at five), because four axes give four independent box positions. That is the regime every measurement before 0.0.187 was taken in, and it is not what a default fit gets.
Two consequences worth knowing. A locally CCD-refined grid is not a partition
that tiles, so it declines the new default and reports the chord read with
theta_within_cell_declined = "support_mixed" -- a difference a performance
knob introduces, which is why it is recorded per axis rather than silent.
And .re_cov_derived_summary() is deliberately pinned to the chord read: its
values are derived quantities at the nodes rather than the design's own cell
coordinates on the reported axis, so a box partition of them would assert a
property nothing measured. The RE-covariance backends are unchanged.
The outer-grid placement pass reaches every registry family whose axes
carry a coordinate, and every one it does not reach says why
(gcol33/tulpa#361). .NL_REGISTRY_AXIS_FIELD named movable axes for icar
and bym2 alone; it now covers icar, rw1, rw2, iid, bym2, nngp,
hsgp and spde, and the field writer behind the finite-difference stencil
is one generic pass over that table instead of a hand-written branch per
family (which silently ignored its theta argument for everything else, so
the curvature was unusable there anyway). Membership follows one rule --
the transform registry names a coordinate for EVERY axis of the family's
grid, and its grid fields bind to those axes one for one -- and
test-nl-registry-axis-scope.R holds the table to that rule from both
sides, so a family added with a placeable grid and no entry fails rather
than being left out in silence. The seven families it does not cover are
covered by a stated reason on the fit: car_proper, ar1 and hsgp_mo
each carry a correlation axis on a support the registry will not guess and
decline with unguessable_axis: rho; mcar, miid and tgmrf hold their
axes in a single matrix field; lf carries no outer axis.
The same pass now runs on the MULTI-BLOCK tulpa_nested_laplace(prior =
list(...)) path, which previously stamped no placement at all -- not
"fixed", not a decline, not the railed-axis report. A moved block is
re-crossed on its own fields while its neighbours keep theirs, and the
finite-difference stencil re-evaluates the inner marginal through the
driver's own theta_grid_override, with each SPDE block's PC prior folded
back on so the stencil differences the target the grid integrates.
Behaviour change: control$auto_recenter = TRUE now re-places a grid that
does not RESOLVE its posterior, not only one that RAILS. An axis whose
median node spacing exceeds .NL_RECENTER$resolve_mult (2) posterior SDs in
its own coordinate triggers the pass alongside the rail test; both tests read
the weights the fit already stored, so a grid that brackets and resolves its
mode still costs nothing beyond them. Measured over 200 fixed-truth seeds on
each of six configurations (icar chain / icar lattice / rw1 / bym2 / iid /
nngp, eight scored axes), mean |coverage - nominal| against the rail-only
policy goes 0.043 -> 0.030 at the 95% level, 0.171 -> 0.084 at 80% and
0.243 -> 0.129 at 50%, at 0.63 times the 95% interval width and 0.76 times
the median bias, for 1.71 times the wall clock. The rail-only policy is
available as auto_recenter = "rail"; FALSE and "always" are unchanged.
Unconditional re-placement ("always") is not what shipped, and the reason
is cost rather than a clean calibration win. The two policies agree seed for
seed on five of the six measured configurations; they differ on the one whose
default axes already resolve their own posterior (NNGP, median h / sd 1.81
and 1.50), where the default fires on 39.5% of seeds against 97.5% and covers
0.530 / 0.500 at the 50% level against 0.135 / 0.385. Coverage arbitrates a
placement rule (gcol33/tulpa#331), so that row favours the default -- but the
reference read on a dense pinned axis that contains the posterior says
"always" is the nearest read of it there, and that the fixture's posterior
itself sits 0.62 / 0.73 above its own truth on the log scale, so a narrower
span covers better by cancellation. What separates the two cleanly is 1.71
against 2.04 times the wall clock. The threshold of 2 minimizes the 50%
deviation and the median bias over the measured family of thresholds and sits
0.0006 off the best 95% deviation.
The recentred axis's h / sd target of 1.25 (5 nodes over mode +/- 2.5
sd) is measured rather than asserted. Sweeping the node count at fixed reach
over 200 seeds on two configurations, mean |coverage - nominal| at 3 / 5 / 7
/ 9 nodes is 0.038 / 0.032 / 0.027 / 0.022 at 95%, 0.155 / 0.062 / 0.067 /
0.082 at 80% and 0.182 / 0.185 / 0.125 / 0.130 at 50%: 3 nodes is clearly
worse, and past 5 the gain is small and not uniform across levels. Cost is
linear in nodes per axis and multiplicative across crossed axes, so 7 and 9
nodes put the widest crossed default the engine lays (hsgp_mo, four axes)
at 2401 and 6561 cells, both past .NL_MULTI_GRID_HARD_CAP = 2048, where 5
nodes is 625.
tulpa_nested_laplace_joint() and fit_st_nested() refuse every per-axis
policy name rather than only "always": their rescues trigger on the whole
grid's collapsed-edge regime, so none of the three has a measured meaning
there.
control$skew_correct now defaults to TRUE
(gcol33/tulpa#364). summary() and confint() on a nested-Laplace fit report
Cornish-Fisher marginal quantiles at each coefficient's own inner-Laplace
gamma_3, about the centre gamma_1 + gamma_3 / 2 that Rue, Martino & Chopin
(2009) eq. (22) implies, wherever the combined inner band admits the
coefficient -- and the previous read wherever it does not.
control$skew_correct = FALSE restores the uncorrected report exactly.
Point estimates, standard errors, vcov(), draws, modes, weights and every
hyperparameter summary are untouched: the correction is post-processing on the
reported quantiles. skew_applied names the rows it was used on,
interval_source names the read, and $skew_correction$reason names the score
that declined any row.The three things a default-on had to be safe against, all measured on current main after the two changes in 0.0.185.
The flip survives the shipped gate. Scored by paired CRPS against the read
a default-OFF fit gives -- the gcol33/tulpa#336 grid mixture -- over 400
prior-predictive replicates each: the rare-event binomial-logit intercept
t = -1.895, and the small-group Bernoulli design's two coefficients
t = -3.765 and t = -3.201. The pre-gcol33/tulpa#354 read that had no
location term scored +3.54 / +6.12 / +4.64 on the same fixtures.
Coverage holds across model classes. Twelve configurations -- the six
built-in families on the single-block driver, a rare-event small-group
binomial, a small-group Poisson, the same data on the joint driver, and three
crossed groupings at outer dimension 3 -- read off ONE solve per seed by the
shipped recov_sweep(), so the corrected and uncorrected arms are paired and
differ only in the marginal read. Pooled over 960 trials at nominal 0.95:
0.9510 -> 0.9542, standard error 0.0070, with every configuration inside the
3-standard-error acceptance the shipped gates use and the gaussian family
identical to the bit. Two small-sample classes move, in opposite directions,
and are the whole of the movement: at 200 seeds and three levels the
correction takes the small-group Poisson design from 0.8950 / 0.7050 / 0.4200
to 0.9400 / 0.7950 / 0.4650 at nominal 0.95 / 0.80 / 0.50, and the rare-event
binomial from 0.9650 / 0.8050 / 0.4900 to 0.9175 / 0.7550 / 0.4700. Summed
distance from nominal over the nine cells: 0.295 uncorrected, 0.175 corrected.
The rare-event drop is the exact answer. Coverage at a FIXED truth cannot
say whether an interval moved away from the right one -- a credible interval
attains its nominal rate averaged over the prior, not at one parameter value.
Fixture A's posterior is exact by one-dimensional quadrature, so the same
rare-event regime runs at five fixed truths with the exact posterior as a third
arm (2000 fits): pooled, exact 0.9470 / 0.8650 / 0.5630, corrected
0.9290 / 0.8650 / 0.5630, Gaussian 0.9625 / 0.8210 / 0.4165. The corrected
interval reproduces what the exact posterior does at 0.80 and 0.50 and is 0.018
from it at 0.95, where the Gaussian is 0.044 and 0.147 away at the two lower
levels. At beta = -2 and level 0.50 the Gaussian interval contains the truth
on 0 of 400 replicates, the exact posterior's on 367, and the corrected one on
367.
The decline paths are no-ops. A coupled fit (every arm a multi-eta unit, so
no location term), a coefficient the importance k-hat flags, a coefficient past
the shape band and a non-nested fit all report bounds identical to the
correction-off fit, to 0.000e+00, while an eligible coefficient on the same
fit moves by 0.397. That required gcol33/tulpa#386 in 0.0.185; without it every
one of those classes moved.
.NL_DIAG$centre_unreliable = Inf, gcol33/tulpa#376). It shipped at 1.20 in
gcol33/tulpa#362 as the smallest cutoff that declined nothing the correction
was measured to help -- on four fixtures none of which could reach it. Three
that do reach it were then built on two sampling designs, and the cutoff
ladder over the enlarged set (6220 coefficient-seeds, 3600 of them admitted by
the other bands) is monotone and zero only past the largest admitted centre
measured: the per-fixture-coefficient costs sum to +0.30
at 1.00, +0.27 at the shipped 1.20, +0.19 at 2.00 and 0 only at 5.00.
No cutoff is protective anywhere -- all 13 fixture-coefficients score a
negative paired gain, and binned by |m| the correction gets BETTER as the
centre grows without turning.The mechanism is why. m_i = (1/2) sum_j c_j rho_ij and
gamma_3(i) = sum_j c_j rho_ij^3 are the same weighted sum at the first and
third powers, so a large centre carrying a small gamma_3 is uniformly WEAK
correlation -- the well-behaved incidental-parameter regime -- rather than a
strong direction being extrapolated. The band was anti-correlated with the
pathology it was imagined for.
Measured on those stored coefficient-seeds: 376 rows change and every one is
centre_unreliable -> eligible,
with the 2620 the shape band and the combined inner band decline untouched. On
those 376 the correction they now take scores -0.09665 against the Gaussian
they used to report (se 0.00676, t -14.31) where the exact posterior scores
-0.09721 -- 99.4% of the achievable gain. A live re-fit of the small-group
Poisson fixture agrees through the engine's own record: 21 of 80 rows move,
max admitted |m| 2.404, and every one still banded good by gamma_3 and
by the combined inner band.
Nothing else changes. cornish_fisher_in_band() is still the one predicate
behind both the eligibility record and the quantile path, centre_unreliable
is still in .SKEW_CORRECT_REASONS and in the precedence, and the cutoff is
still an argument -- .nl_skew_correction_attach(max_abs_centre =) now takes
it -- so a finite value restores the band on every path at once, which is what
the decline tests drive. gamma1_not_computable is what guards an unformed
location term and is untouched.
.nl_fixed_interval() branched on the whole fit, so
skew_correct = TRUE sent every reported coefficient down the MAP-cell path
and the ones the bands refused fell back past the gcol33/tulpa#336 mixture CDF
to mu +/- z sigma. The two reads cannot be composed on a CORRECTED
coefficient -- gamma_3 is retained at the MAP cell only -- but a declined one
has no MAP-cell read to preserve, so the fallback gave up the across-cell shape
for nothing. Worst on a fully coupled fit, where gamma_1 is unreachable and
every coefficient declines: enabling the correction there moved every bound and
corrected none. The base read is now computed first and only the rows
.nl_skew_marginal() applied to are overwritten; interval_source reports
"skew_map_cell" when every row was corrected, "skew_map_cell/mixture_cdf"
when both reads are in play, and the base source when none was.weights reaches the spatial mode-finding, not only the marginal precision
built at it (gcol33/tulpa#385). dispatch_laplace_spatial() carried no
weights argument at all, so a spatial tulpa_laplace() fit found the
UNWEIGHTED mode and then handed it to .marginal_H_beta_*(), which has always
consumed weights -- the reported mode and the reported precision described
two different models. On a 25-cell ICAR fixture at a 4:1 weight split the
slope moved 0.740 -> 0.880 (19%) once the weight reached the Newton loop, the
score of the weighted objective at the reported mode was 14.86 rather than 0,
and H_beta matched the WEIGHTED Schur at that unweighted point to 2.0e-16
while sitting 130% away from the unweighted one. Reachable from
tulpa_em_laplace(), which forwards a block's weights unconditionally,
including for a block that also carries spatial.The arithmetic already existed: BuiltinFamilyResponse::weights scales each
row's log-density, score and Fisher information alike, and the third-derivative
oracle behind gamma_3 reads it too. What was missing was the argument. All
six spatial kernels now take it -- cpp_laplace_fit_spatial / _bym2 /
_car_proper / _hsgp through build_spec_family_inputs(), and
cpp_laplace_fit_gp / _spde / _spde_precomputed through a weights field
on JointArm that make_single_arm() fills and the joint driver hands to its
built-in-family response. None needed the error fallback. The fractional-nu
SPDE marginal (cpp_spde_fractional_logmarginal) is weighted with them, so a
weighted mode is never paired with an unweighted log-marginal; its gaussian
branch uses the |I + K B| form, which stays finite at a zero weight where the
scaled-variance form does not.
Measured rather than asserted. The weighted mode reproduces an R Newton solve
on the penalized objective written from its definition to 8.5e-15, against
0.140 for the unweighted one; the weighted score at the reported mode falls
from 14.86 to 1.4e-06 (the loop's own step-based floor); H_beta and the
Schur at the reported mode now agree to 4.8e-17 with the same w. An integer
weight equals row replication on every kernel, which reads the weighted path
against the engine's own unweighted solver on a different data set. Nothing
moves at weights = NULL: 25 fits spanning every kernel through both the raw
C++ entry and the front door (including H_beta) are identical() to the
pre-fix build, worst numeric difference 0. No ABI change -- every struct
touched lives in src/, none in inst/include/tulpa/.
Evidence in dev_notes/issue385/.
An EM soft label travels on weights, not on a fractional y
(gcol33/tulpa#383). tulpa_em_laplace()'s own zero-inflated-Poisson example
encoded the occupancy arm as list(y = weights, ...), and the count-family
guard refused it -- the last R CMD check --run-donttest ERROR. The guard is
right and the encoding was wrong: the M-step maximizes the expected
complete-data log-likelihood, which for a Bernoulli latent is
sum_i [w_i log p_i + (1 - w_i) log(1 - p_i)], a WEIGHTED Bernoulli
likelihood carrying no binomial coefficient. A fractional y asks instead for
the exact binomial density at a non-integer response, whose lchoose(1, w)
normalizer is neither zero nor free of w -- a different objective that
shares a maximizer only once that term is dropped. The channel for the right
one already existed: BuiltinFamilyResponse::weights scales each row's
log-density, score and Fisher information alike, so the encoding is two rows
per unit, y = 1 at weight w and y = 0 at weight 1 - w. The example,
the m_step_encode contract and the block validator now say so, and a
fractional binomial y on an EM block errors naming the channel rather than
the density.
That claim is measured, not asserted (test-em-laplace-recovery.R). One
M-step's binomial arm reproduces glm(family = binomial, weights = ) to
3.4e-07 and the cbind(w, 1 - w) form to the same figure, its H_beta is
the weighted Fisher information to 1.2e-12 relative, and the Poisson arm
agrees with its own glm to 2.5e-08 -- the residual in each case being the
engine's weak built-in fixed-effect ridge, which glm does not carry.
Zero-inflated-Poisson recovery over 12 seeds at n = 2000 returns
(0.796, -0.903, 1.197, 0.501) against truth (0.8, -0.9, 1.2, 0.5), and the
observed-data log-likelihood -- written out independently of the fit -- rises
monotonically across undamped EM iterations, worst decrease -2.6e-11 against
a total climb of 78.8. test-em-laplace.R mocks the fitter throughout, so
until now the driver had 20 structural tests and no recovery.
tulpa_laplace(weights = ) is length-validated (gcol33/tulpa#384). The kernel
borrows the vector as a bare pointer indexed to N, so a short one was an
out-of-bounds read that returned the start vector as the mode with no error.
.validate_submodel_block() checks the same field on an EM block.
The EM occupancy block in test-recovery-assembled.R ran for the first time.
It carried the same fractional-y encoding, so the guard had made it
unrunnable since 0.0.30 and its premise -- that the raw EM leaves the
intercept biased and MI / Gibbs reduce that bias -- had never been measured.
Under the corrected encoding the raw M-step is already at the expected
complete-data maximum and there is no point bias to remove: median
|b0 - truth| is 0.0817 raw against 0.0863 under MI and 0.0856 under Gibbs.
What the corrections do add is the latent-state uncertainty the raw Laplace
conditions away, V_between > 0 and V_total > V_within, and that is what
the block now asserts.
sbc(experiment = "posterior") reports the pooling premise from the guard's
own state rather than asserting it (gcol33/tulpa#380). The driver calls
pool() for every simulation, so "verified" was always the true answer --
but writing it as a constant is the pattern gcol33/tulpa#293 is about, where a
field that cannot say "not reached" hides a path that was never taken. It now
reads state$checked, the same way fresh_groups already read its own.R CMD check is clean again (gcol33/tulpa#382). Three pre-existing defects,
none related to the code they sat next to, and each a one-line fix.
temporal_rtr()'s example called temporal_rw1() with no arguments while
time_var has no default, so checking examples was an ERROR on any machine;
it now passes "year", as every other call site in the package does.
.canonical_family()'s roxygen linked .family_or_stop(), which is internal
and has no Rd, giving a dangling cross-reference; the sentence naming it
stays, the link goes. And lmtest, numDeriv and spdep are used through
:: in the test suite and were not declared -- they are used TODAY, all three
are on CRAN, and this is a declaration rather than a new dependency. The
numDeriv test is now skip_if_not_installed()-guarded like the other two,
so it skips rather than errors where the package is absent.
The same hygiene one layer over: test-posterior-sbc.R's new front-door
equivalence assertion (gcol33/tulpa#380) compared the whole driver return,
including the fit_obs the posterior driver attaches, whose $timing is
wall clock and differs between two identical runs by construction. The
assertion now drops that attribute and keeps the experiment.
New: sbc(), one exported front door for simulation-based calibration
(gcol33/tulpa#380). The SBC machinery from gcol33/tulpa#335, gcol33/tulpa#339
and gcol33/tulpa#350 was complete and independently arbitrated, but it lived
only in tests/testthat/helper-sbc.R, so a user could not run SBC on their
own fit and a method claim citing it had no engine path a reader could
reproduce. Sections 1 to 6 of that file -- the predictive shapes, the PIT with
within-atom randomization and its folded read, the CRPS closed forms, the
exact simultaneous ECDF bands, and both drivers -- moved unchanged into
R/sbc.R. The tests now read the PACKAGE functions; nothing is duplicated.
sbc(experiment = ) selects the two experiments in which the PIT is uniform
and the CRPS is a proper posterior score: "prior_predictive" (ordinary SBC,
taking simulator / fitter) and "posterior" (calibration conditional on
an observed data set, Sailynoja et al. 2026 Algorithm 2, taking the six-callback
model). A fixed-truth sweep is not offered -- it is not an SBC experiment --
and the refusal sbc_crps_compare() enforces on one is unchanged underneath.
The result carries the per-(arm, quantity) PIT, the report table, the
calibrated band and what each guard concluded, and reads through print(),
summary(baseline = ) (which adds the paired proper-score ranking), plot()
(the ECDF difference against the band) and diagnostics(). Passing an sbc
result to diagnostics(fit, sbc = ) prints the calibration verdict under the
fit's own reliability band, which is a screen and not a verdict: the two are
measured to disagree in both directions.
The five predictive shapes sbc_mixture(), sbc_normal(), sbc_discrete(),
sbc_rank() and sbc_draws() are exported alongside it as the argument type
a fitter returns. They are the extension point, not alternative verbs.
Two guards, each on a premise that silently invalidates a result. The
prior-predictive path refuses a scored quantity whose truth does not move
across simulations -- what an improper prior looks like from outside, and what
the nested door's fixed effects are -- before spending the fits, naming
experiment = "posterior" as the way out; a location parameter admitted by a
structural argument goes through flat_prior, which is checked in both
directions and travels on the result. The posterior path refuses a pool()
returning no more than one of its two inputs (fitting the replicate alone is
ordinary SBC under a hand-made prior), and verifies disjoint group LABELS when
model$group_ids is supplied, recording the premise as unverified rather than
assumed when it is not. Whether the simulator drew those groups' effects
independently is not observable from outside the callback and is not claimed.
Inf as a
bound (gcol33/tulpa#381). .nl_wtd_quantile() delegates the interior to
stats::approx, whose linear form y0 + (y1 - y0) * t builds the DIFFERENCE
before scaling it, so two adjacent knots more than the double range apart take
it out of that range and the reported bound comes back non-finite at a
probability sitting strictly between two FINITE coordinates. This is the
interior read, not a cell edge: it reproduces with the partition finite and
its mirror standing, and on the sample support that forms no edge at all,
which is what separates it from gcol33/tulpa#377, gcol33/tulpa#378 and
gcol33/tulpa#379.The convex form (1 - t) y0 + t y1 cannot overflow there, but it is NOT
substituted wholesale: the two are not the same double, and over 840000
randomized reads 21.49% of them move between the forms, by up to 4.07e-10
relatively where the difference cancels. Three fixes on this path are pinned
on identical() against what approx returns. So .nl_interp_repair() is
reached only where the straight read already failed to return a double, and a
bracket whose knots are not both finite declines rather than inventing a
number. Measured over 1620000 reads (60000 node sets straddling the double
range in both signs x five declarations x three outside policies): non-finite
bounds 9504 -> 0, finite pre-fix reads that moved 0, repaired bounds outside
their own bracket 0, mis-ordered reads 0.
Inf or NaN
(gcol33/tulpa#379). .nl_cell_partition() mirrors the extreme cell's
half-spacing to place the outer edges; gcol33/tulpa#377 gave that mirror a
fallback on a DECLARED domain and left the surviving guess branch never
checking the edge it produced. On the LINEAR coordinate the mirror needs the
extreme coordinate plus half its own spacing to stay in the double range, and
at the top of that range it does not: c(1, 1e300, .Machine$double.xmax)
mirrored to Inf and the DEFAULT chord read reported Inf as a 97.5% bound
from a partition recording no reason, while c(-.Machine$double.xmax, 0,
.Machine$double.xmax) mirrored to -Inf and the interpolation between a
non-finite knot and a finite one reported NaN.Such a mirror now falls back to the extreme coordinates -- representable by
construction, and inside whatever support the axis has, since they are
coordinates of it -- and records mirrored_edge_not_representable, a fourth
entry in the closed .NL_EDGE_DECLINED vocabulary (gcol33/tulpa#293: a
silent-disable path needs a reason). It is a second name rather than
mirrored_edge_outside_domain because there is no domain here to be outside
of, and it takes precedence over a nodes_outside_declared_domain /
unknown_domain already in hand: those name a declaration that was set aside,
after which the guess ran, and this one names what the edges ARE.
The guess itself is not otherwise narrowed. An undeclared all-positive axis
whose log mirror underflows keeps its 0 lower edge, which is
gcol33/tulpa#377's boundary from the other side -- with no declaration there
is no support to measure an edge against, and car_proper's rho_car keeps
exactly the edge it has.
Measured over 595010 reads (59501 randomized node sets spanning the double
range in both signs x five declarations x both within-cell constructions):
non-finite partition edges 69930 -> 0, non-finite reported bounds attributable
to the mirror 251746 -> 0, mis-ordered bounds 0 -> 0, finite bounds outside a
declared containing support 0 -> 0, and no read gains a non-finite bound.
Byte-identical elsewhere: over 540 (node set, domain, weighting) cases and
4320 quantile comparisons, plus 118405 randomized node sets, everything whose
pre-fix partition had finite edges is identical() on the edges, the boxes,
the coordinate, the reason and every quantile read; 0 of 118405 moved despite
a finite pre-fix edge, and all 960 replaced partitions are finite, bracketing
and tile.
Only the linear coordinate reaches it -- log / qlogis / atanh land
inside about +/- 745 -- and no default .NL_GRID axis is near it; it is
reachable through a pinned grid and through
hyper_axis_spec(bounds = c(-Inf, Inf)). test-within-cell-box-uniform.R's
boxes_do_not_tile fixture moves for the second time, off the undeclared
c(1, 1e300, .Machine$double.xmax) -- which now tiles -- and onto the unit
axis at the subnormal floor, whose decline is a property of the node set
against its coordinate and which no edge guard changes.
.NL_EDGE_FALLBACK names the reasons whose edges ARE the extreme coordinates
(mirrored_edge_outside_domain, mirrored_edge_not_representable), so the
bound is conservative on that side; nodes_outside_declared_domain /
unknown_domain set a declaration aside, after which the guess ran and its
mirror stood, so the bound is a guessed edge. The single sentence
.tulpa_interval_read_note() fired for all of them reported the second pair as
running to the extreme coordinate, which is the opposite of what happened.a / 2 + b / 2 rather than
(a + b) / 2, so a partition whose every edge is a representable double is
built instead of declined (gcol33/tulpa#378). .nl_box_edges_from() bisects
each interior cell spacing in the axis's own coordinate; the sum was formed
before the halving, so two coordinates near the top of the double range took
it to Inf and the whole partition was refused with boxes_do_not_tile, the
axis falling back to the chord read. c(1, 1e300, .Machine$double.xmax)
declared unbounded is the reachable case: (1e300 + double.xmax) / 2 is
Inf while 1e300 / 2 + double.xmax / 2 is 8.988466e+307, and the boxes
that midpoint defines tile the axis.Only the LINEAR coordinate reaches it -- log / qlogis / atanh land
inside about +/- 745, where no sum of two coordinates can overflow -- so it is
the unbounded declaration and the undeclared axis whose values are not all
positive. No default .NL_GRID axis is near it; it is reachable through a
pinned grid and through hyper_axis_spec(bounds = c(-Inf, Inf)).
a / 2 + b / 2 and NOT a + (b - a) / 2, measured rather than
pattern-matched (dev_notes/issue378/midpoint378.R, 4e6 randomized pairs
spanning the double range in both signs). The offset form overflows on
b - a for opposite-sign extremes -- reachable, since the linear coordinate
is signed -- taking 1117 pairs to Inf that a / 2 + b / 2 returns finite,
and it rounds twice on ORDINARY operands, moving 52536 of the 3998914 pairs
where (a + b) / 2 is finite. a / 2 + b / 2 produced 0 non-finite results,
and stays inside [a, b] on 2e7 subnormal-heavy ordered pairs.
BYTE-identical wherever the sum form worked at all. Halving a normal double
decrements the exponent and leaves the significand alone, so a / 2 and
b / 2 are exact and their sum carries ONE rounding of the same real the sum
form rounds once -- rounding to nearest commutes with scaling by a power of
two. Asserted with identical() against the pre-fix file sourced from git
into its own environment over the namespace
(dev_notes/issue378/identity378.R): of 135 (node set, domain) cases over 4
weightings x 4 support kinds x 2 within-cell constructions x 11
probabilities, 133 are identical on the cell edges, the box partition and
every quantile read, and the 2 that move were both pre-fix DECLINES that now
tile. Over 197374 randomized node sets across all four domains, the fix
removes 24045 declines and introduces 0, with 172860 tiled under both and 0
of them moved; all 23913 recovered partitions are finite, strictly
increasing, of the right length, contain their own coordinates and stay
inside the declared support. The 3621143 pairs where both halvings are exact
carry 0 differences; the two forms part only where a halving is subnormal
(|x| below about 2.2e-308), by at most 1.0e-320.
The decline is kept, not removed. test-within-cell-box-uniform.R's
boxes_do_not_tile pin moves to the same node set UNDECLARED, where the
decline is a property of the node set rather than of the arithmetic: the
partition mirrors the extreme half-spacing in the value itself and
double.xmax + (double.xmax - 1e300) / 2 is past the double range in any
form, so there is no better midpoint to compute. A second pin covers a node
set the coordinate map cannot separate (a unit axis at the subnormal floor,
where plogis(qlogis(1e-320)) underflows to 0), alongside the zero-width
interior box already pinned. A new randomized identical() sweep over all
four domains holds the bisector to the sum form's own numbers.
.nl_cell_edges() the axis's domain and checked the mapped edge against that
domain's in_domain before accepting it. The chain did not stop at that
check: a rejected edge fell through to the all(v > 0) log guess, which
computes the SAME number on a positive axis and tests only is.finite(), so
the weaker test accepted one line later exactly what the stronger one had just
refused.Measured on the issue's own node set, c(1e-320, 1e-310, 1) declared
positive: exp(log(1e-320) - 0.5 (log(1e-310) - log(1e-320))) underflows to
exactly 0, and the fit reported a lower bound of 5.93e-323 for a quantity
whose declared support is x > 0. From the other end of the double range,
c(1, 1e300, .Machine$double.xmax) overflowed to Inf, fell past the log
guess to the LINEAR mirror, and reported -4.97e+299 and Inf for the same
declared support. It is gcol33/tulpa#369's defect one step down: there a
unit axis reported 1.0028 because the domain never reached the edge; here it
reaches it and is overruled.
THE PRECEDENCE, not a patch on the symptom. A declared domain that CONTAINS
every coordinate is authoritative: its mirrored edge is taken when finite and
in-domain, and otherwise the partition DECLINES to the extreme coordinates
themselves -- which that same containment test has already placed inside the
support, and which are the conservative answer -- recording
mirrored_edge_outside_domain. The guess is RESTRICTED, not removed: it runs
where there is no declaration to contradict (no domain named,
nodes_outside_declared_domain, unknown_domain), and car_proper's
rho_car on the adjacency eigenvalue interval keeps exactly the edge it had.
A node set the declaration does not contain cannot be honoured by any edge at
all -- the edges bracket the coordinates -- so that case is the guess's by
construction and says so rather than silently taking it.
The reason field is gcol33/tulpa#293's requirement and rides the hook
gcol33/tulpa#357 already built: .nl_cell_partition() reports the COORDINATE
it settled on, so it is also where the decline lives, and both readers take it
from the same return. It travels out through .nl_summary_quantile_read() and
.nl_axis_quantiles() to the fit as theta_cell_edge_coord /
theta_cell_edge_declined, one per axis, and diagnostics() says which axis
reports a conservative bound and why. .NL_EDGE_DECLINED is the closed
vocabulary.
WHAT MOVES AND WHAT DOES NOT, proven with identical() rather than a
tolerance (dev_notes/issue377/identity377.R, identity377.out): the pre-fix
R/nested_laplace_moments.R is sourced from git into its own environment over
the tulpa namespace, and 22 node sets x 5 domain declarations x 4 weightings x
4 support kinds x 2 within-cell constructions are compared call for call. Of
110 (node set, domain) cases, 100 are byte-identical on the edges, on the box
partition and on every quantile read; the 10 that move are exactly the ones
whose partition reports mirrored_edge_outside_domain, and no such case
arises on an undeclared axis. Over 10324 node sets inside their declared
domain -- the fixtures plus a 20000-draw randomized sweep across all four
domains -- 174 produced an edge outside that domain before the fix and 0
after.
The invariant asserted alongside gcol33/tulpa#369's unit / correlation
regressions in test-nl-interval-support.R is the general one: whenever every
coordinate is inside a declared domain, both edges are finite and inside it
too, for every entry of .NL_DOMAIN_TRANSFORM. test-within-cell-box-uniform.R's
boxes_do_not_tile fixture moves to the unbounded declaration, where an
interior midpoint's own sum still overflows before it is halved; declared
positive, that same node set now has a finite in-support partition and the
box read no longer declines on it.
support taxonomy, and the position sensitivity it carries is
reported rather than silently accepted (gcol33/tulpa#357). The outer grid's
cell weights say how much mass each cell holds; they do not say how it is
spread inside the cell, and a reported quantile needs both. "chord" -- the
shipped read -- puts the cumulative MID-mass at each cell COORDINATE and
interpolates between coordinates. "box_uniform" puts the cumulative FULL
mass at each cell EDGE and interpolates between edges: the same masses over
the same boxes with the knots moved half a cell.
control$within_cell = "box_uniform" selects it on tulpa_nested_laplace(),
tulpa_nested_laplace_joint() and the ST front door.THE DEFAULT DOES NOT MOVE, and that is shown rather than asserted: seven fits
spanning the single-block registry, the joint single-block driver, the
multi-block tensor, CCD and adaptive paths, a bounded bym2 rho axis and
the gcol33/tulpa#337 fixture were run against HEAD's R/ and against this
tree with the same DLL, data and control lists, and every reported
theta_median / theta_ci_lo / theta_ci_hi / theta_mean / theta_sd /
log_marginal / weights and per-block moment is identical()
(dev_notes/issue357/identity357.R, identity357.out).
It is a second FIELD on each .NL_SUPPORT entry rather than a fifth kind:
outside is a fact about what the producer left behind, within is a
caller's choice about how to read it, and the two are orthogonal. Only a
"density" support admits box-uniform -- a CCD design is a moment rule, a
locally refined grid's replacement clouds sit inside one base cell so a
Voronoi partition of the node set is not the design's own boxes, and a
posterior sample's values are order statistics. .nl_cell_edges() is now
.nl_cell_partition()$edges, which also reports the COORDINATE the edges came
out of, because the box read has to bisect the interior spacings in the same
coordinate the outer half-cells were mirrored in or the partition it tiles the
axis with is not one partition. That reconciles the construction with
gcol33/tulpa#369: on a unit axis the boxes are bisected in logit, so a BYM2
mixing weight with a node at 0.99972 (gcol33/tulpa#361 extended bym2_rho to
six nodes including 0.999) stays inside (0, 1) where the log guess reaches
1.0015. dev_notes/issue337/recon.R's slice_masses() STOPS when the boxes
do not tile; a reported interval cannot take that behaviour, so a partition
that is not finite and strictly increasing DECLINES with a recorded reason
("support_<kind>", "single_node", "boxes_do_not_tile",
"no_usable_node") and that axis falls back to the chord read on its own.
THE PARTITION COMES FROM THE GRID AND THE MASSES FROM THE WEIGHTS, which is
what gcol33/tulpa#337's own "keep the masses, tile the axis" says and what
recon.R does (axis_geometry() on the whole grid, slice_masses() on the
positive weights). A cell whose integration weight underflows to exactly 0
still SITS on the axis, and its coordinate is what fixes its neighbour's box
edge; filtering the coordinates by weight -- which is right for the chord
read, whose knots ARE the positive-weight coordinates -- shrinks that
neighbour's box to nothing. On the coarsest rung of the re-scored ladder
(2 cells at 400 groups) the softmax underflows one of two cells on 43 of 150
seeds, so that filter was the difference between measuring the construction
and measuring the chord read under another name. An empty interior box is a
FLAT segment of the CDF: the quantile is located on the cumulative mass and
evaluated inside the box it lands in, so an empty box is stepped over rather
than interpolated across.
Every nested path now stamps what its per-axis intervals were read off, not
only the multi-block driver (gcol33/tulpa#357). theta_interval_read /
theta_interval_design_mass were filled by .joint_dispatch_multi() alone,
so the single-block, joint single-block, registry and ST paths were
indistinguishable from a fit that does not record it --
helper-outer-grid-dump.R's .ogd_support() fell back to .nl_node_support()
for exactly that reason. .nl_posterior_moments() and its two multi-block
siblings stamp the pair, alongside the new within_cell_requested /
theta_within_cell / theta_within_cell_declined.
.tulpa_interval_read_note() had no consumer at all; it and the new
within-cell note are now attached by laplace_diagnostics() and printed.
A fit reports how coarse its own outer grid is:
outer_grid_cell_width / outer_grid_axis_sd / outer_grid_h_over_sd, per
axis, both in that axis's own coordinate (gcol33/tulpa#357). Every within-cell
construction resolves an interval endpoint to within one cell, so part of the
reported width and of its realized coverage is a property of where the grid
fell rather than of the posterior, and h / sd is what governs how much.
Below .nl_diag("grid_resolved") = 1 the cells are narrower than the
posterior they discretize and the two constructions converge; the
34-configuration census of the engine's own default axes puts every one of
them above it (minimum 1.01, median 4.25, maximum 18.06), so
.tulpa_grid_resolution_note() names the coarsest axis and its ratio.
.nl_laplace_at_mode_sd_axis() takes the coordinate as an argument and
return_log_sd becomes return_u_sd, the SD in whatever coordinate the
parabola was fit in -- the branch existed and had no caller in R/.
The re-score on current main, which is the open question gcol33/tulpa#357 was
left with. Neither the box-position sweep's blocker nor the ladder's verdict
moves. gcol33/tulpa#361 (default axis placement, bym2_rho node set), #369
(the outer cell edge's coordinate) and #375 (the rail guard) all landed since
the numbers were taken, but the fixture the comparison is measured on pins its
own grid, so none of them reaches it. Repeated on the working tree through the
SHIPPED .nl_axis_quantiles(within = ), 12 positions x 200 seeds at five and
seven levels, the conditional 95% coverage across one cell reproduces the
recorded numbers exactly: box-uniform 0.585 to 1.000 at five levels, SWING
0.4150, box-averaged 0.9033 against nominal 0.95; 0.895 to 0.995 at seven,
swing 0.1000, averaged 0.9508. The chord read's own conditional coverage at
nominal 0.50 runs 0.655 to 0.950 (swing 0.2950) and 0.720 to 0.855 (0.1350),
so both reads have the dependence and the chord read hides it behind width at
0.95. The promoted engine read reproduces the arm
dev_notes/issue337/recon.R scored to 1.8e-15 on every one of the 4800 fits.
The twelve-rung ladder transfers too: the paired CRPS favours box-uniform at
12 of 12 rungs, the folded PIT at 12 of 12, the 95% coverage is closer to
nominal at 11 of 12 (the twelfth an exact tie, 0.9933 and 0.9067 both 0.0433
from nominal) and raw KS at 10 of 12, the two exceptions the same levels = 2
pair the issue records.
The default flip stays the maintainer's call and the issue stays open on it.
Evidence: dev_notes/issue357/RESULTS.md, common357b.R, boxpos357b.R,
coarse357b.R, analyse357b.R, identity357.R, unit357.R.
.NL_DIAG$centre_unreliable = 1.20 was chosen in
gcol33/tulpa#362 as the smallest cutoff declining nothing the correction was
measured to help on, over four fixtures none of which could reach it.
Re-scored over SEVEN fixtures and 4224 admitted coefficient-seeds -- every one
gated by the SHIPPED combined inner band, which the earlier sweep did not
apply -- the paired cost of the band is monotone in the cutoff and reaches
zero only past the largest admitted centre measured (4.62): +0.302 at 1.00,
+0.270 at 1.20, +0.191 at 2.00, +0.089 at 3.00, 0 at 5.00. No cutoff
is protective on any fixture: all 13 fixture-coefficients score a negative
paired gain (t -1.65 to -9.83) and the correction gets BETTER as the centre
grows, -0.036 at |m| in (1.2, 1.5] through -0.249 at (3, 6]. The reason
is in the mechanism: m and gamma_3 are the same weighted sum at the first
and third powers of the per-observation correlation, so
rho_eff = sqrt(|gamma_3| / (2 |m|)) bounds the strongest single correlation
from below (exactly 1 on an intercept-only fit), and a large |m| with a small
|gamma_3| is uniformly WEAK correlation -- median rho_eff 0.72 to 0.84 on
the fixtures that stay under the band and 0.086 to 0.116 on the three that
reach past it. The constant is unchanged and the decline path is unchanged;
its justification in R/settings.R now records the enlarged measurement, the
derivation, and the price the band is being paid for..NL_DIAG$centre_unreliable = 1.20, gcol33/tulpa#362) had never fired: the
largest admitted centre on any measured fixture was 1.192 and the only test
producing centre_unreliable did it by overwriting inner_skew_gamma1. The
applied centre is m_i = cross / (2 sigma_i) -- the gamma_3 cancels exactly
between gamma_1 and gamma_3 / 2 -- so with rho_ij the Gaussian
correlation between eta_j and the probed coordinate, m_i and gamma_3(i)
are the same weighted sum at the first and third powers of rho, and reaching
the centre band inside the shape band needs many weakly correlated COHERENT
terms rather than a few strong ones. A small-group Poisson random-effect fit
supplies both: l''' = -mu is uniformly negative so every term adds, and many
small groups send |m| ~ sqrt(G) up while |gamma_3| ~ G^-1/2 falls. On 150
groups of 2 the band declines 178 of 300 intercept-seeds, every one with the
shape band and the combined inner band both admitting..nl_axis_rail() required the boundary node to carry at
least .NL_RECENTER$edge_mass = 0.5 of its axis's marginal weight, a per-NODE
share against a constant that does not know the node count -- so the same
posterior read at more nodes carried less on any one of them and a LONGER axis
became a WEAKER detector, exactly as #361 lengthened the default bym2_rho
span the rescue was meant to compose with. The reference is now what a FLAT
marginal would put on that node: edge_mass_mult = 2 against m * w[k],
which is 1 for a flat marginal at any node count and is the retired 0.5 share
at the four nodes it was tuned on. On one BYM2 posterior held past a fixed
span and read at 4 / 5 / 6 / 8 / 12 / 20 nodes, the rail now fires on
12 / 11 / 10 / 10 / 10 / 10 of 20 seeds -- the count whose marginal is maximal
at the top node at all -- against 12 / 10 / 6 / 3 / 0 / 0 before. A marginal
flat to within its own noise, and one that merely tilts 1.6x across the span,
still decline at every node count. Paired fixed-truth interval coverage over
240 fits at four truths, both arms the shipped engine of their own library:
41 replicates move and 199 are byte-identical, coverage gains 10 / 20 / 30 and
loses 0 / 0 / 1 at the 95 / 80 / 50% levels, and the 95% coverage at
rho = 0.9999 goes 0.000 to 0.167 where the detector had never fired at all.The default BYM2 mixing-weight axis reaches 0.999 (gcol33/tulpa#361). On a
BOUNDED axis a coarse span and a short span are not interchangeable: the
outside = "extend" read mirrors the outer cell edge in the axis's own logit
coordinate, so the four-node default could not report an upper bound above
plogis(logit(0.95) + 0.5 (logit(0.95) - logit(0.8))) = 0.97642 whatever the
data said, and no node count moves that. Measured over 800 fits per candidate
node set -- four fixed truths, 200 seeds, every arm the shipped engine with
only .NL_GRID$bym2_rho replaced -- 95% coverage at rho = 0.99 goes 0.555
to 0.955 and at 0.97 goes 0.820 to 0.950, mean |coverage - nominal| over the
four truths goes 0.140 to 0.011 at 95%, 0.189 to 0.088 at 80% and 0.081 to
0.099 at 50%, and mean |median bias| goes 0.058 to 0.041, at a mean 95%
width of 0.494 against 0.487. The 0.555 the four-node axis managed at
rho = 0.99 WAS the fraction the placement rescue moved; the six-node axis
contains its own mode on every one of those 800 fits, so it also spends no
refit. A five-node candidate stopping at 0.99 was scored too and is not what
shipped: it takes 95% coverage most of the way (0.900 at rho = 0.99) while
its 50% interval covers 0 of 200 there.
Three joint drivers restated a default axis's nodes as a literal instead of
reading .NL_GRID, so a change to the table did not reach them: bym2's
rho_grid in .joint_backends and .joint_multi_axis_grid, and ar1's
rho_grid in the latter. All three read .nl_grid_axis() now. The joint
paths' proper-CAR correlation nodes were duplicated between the same two
files and are a .NL_GRID entry of their own (joint_car_rho), separate
from car_rho because the joint drivers do not compute rho_bounds and lay
fixed nodes on (0, 1) rather than the adjacency eigenvalue interval. It is
deliberately not bound to a .NL_FAMILY_AXES field, since that table is what
.nl_axis_matches_default() reads and binding it would reclassify a caller's
identical nodes from a pin to a default.
control$auto_recenter takes "always" (gcol33/tulpa#361 checklist items 1
and 2): recentre every movable default axis on its posterior mode whatever
the fit did, rather than only an axis that rails. The recentred axis is
mode +/- 2.5 sd over 5 nodes, so it resolves the posterior at h / sd =
1.25 by construction against a census median of 3.9 on the fixed spans.
Measured over 200 fixed-truth seeds on each of a 100-region ICAR chain, a
144-cell ICAR lattice and a 100-region BYM2, mean |coverage - nominal| over
the four (config, axis) rows goes 0.036 to 0.026 at the 95% level, 0.176 to
0.078 at 80% and 0.268 to 0.160 at 50%, at 0.24 to 0.81 times the 95%
interval width -- the fixed arm covers exactly 1.000 at nominal 0.95 on three
of the four rows -- and median h / sd goes 3.09 / 2.92 / 1.99 to 1.29 /
1.29 / 1.02. It is NOT the default: it is a second full grid solve plus the
finite-difference mode/Hessian stencil on every fit, measured at 1.88x /
1.94x / 2.23x the wall clock, and .NL_REGISTRY_AXIS_FIELD names movable
axes for icar and bym2 only, so as a default it would move two families and
leave eleven on their fixed spans. tulpa_nested_laplace_joint() and
fit_st_nested() refuse the value rather than accept it and ignore it --
their rescues trigger on the whole grid's collapsed-edge regime, not on a
per-axis rail, so "always" has no measured meaning there.
reduction(+:) leaves the order its per-thread private
copies are combined in unspecified, and libgomp combines them as each thread
finishes; floating-point addition is not associative, so the data
log-likelihood landed an ulp or two apart from one run to the next and the
inner Newton loop's convergence test and line search amplified that into
~1e-13 on log_marginal. A capture of ten fits and 90 fields at
n_threads = 4, taken twice from the same build with every seed fixed, moved
15 of the 90.tulpa_parallel_sum() (src/omp_threads.h) now cuts the range into team
contiguous chunks by index arithmetic, sums each chunk left to right into its
own slot, and adds the slots in chunk order. Nothing about the answer is left
to the runtime: not the combination order, and not the partition either, so a
fit reproduces itself even where the runtime hands back a smaller team than
the one requested. The parallel loop runs over the chunks rather than the
observations, which keeps the combined parallel for construct. Because
gcol33/tulpa#373 had already funnelled every one of these sums through that
one helper, the change is one function and reaches the joint and non-joint
paths together.
What this buys is reproducibility AT a thread count, not across thread
counts: chunking the range imposes its own association, so a fit at
n_threads = 4 still agrees with the same fit at one thread only to
floating-point tolerance (measured at 6e-14 on log_marginal over four
families), which is the invariant ?tulpa_nested_laplace_joint now states.
The one-thread route is untouched: it keeps its plain loop and allocates
nothing, and the same 90 fields are identical() before and after the change.
The same capture taken twice at n_threads = 4 after it moves 0 of 90. Cost,
with the three reduction shapes alternated inside one process so they share
the machine's load: 1.00 / 1.03 / 1.02 times the old cost at 48 observations
with a team of 2 / 4 / 8, and indistinguishable at 960 and 12000
observations. Rows where all three arms run the same code move by up to 17% in
the same capture, so the 3% is at the edge of what the instrument resolves.
if(n_threads > 1) still enters libgomp when the clause is false, and that
the entry cost 7.6 microseconds per objective evaluation on the joint path;
the two loops fixed there each grew their own runtime branch. That policy is
now one pair of helpers in omp_threads.h, tulpa_parallel_for() and
tulpa_parallel_sum(), which take a plain loop at a team of one, and the two
#365 loops were moved onto them so there is a single place the rule lives.Sites moved onto the helpers: the single-arm spec eta assembly
(compute_eta_spec, which had no guard at all and so entered a one-thread
region unconditionally on every non-joint nested-Laplace objective
evaluation), both compute_total_log_lik overloads for built-in families,
tulpa_linalg::matvec (reached per leapfrog step through
precompute_generic_fixed_eta), the five per-observation regions in
pg_gibbs_core_step and the ten per-sweep regions across the four
Polya-Gamma samplers, and the NNGP non-centered reverse sweep
(nngp_nc_backward), whose per-thread workspace slot and accumulators are
now parameters of a single-sourced row body rather than captures.
A serialised region body runs in index order, which is what the plain loop
does, so both routes agree bit for bit, reduction(+:) included. Verified
over 292 captured fields and 68968 doubles -- non-joint nested Laplace at
three families and three field types, a corrected-integrated-Laplace fit,
cpp_laplace_fit, six Polya-Gamma samplers, generic NUTS, NNGP exact NUTS
and single-arm joint fits -- at one thread and at four, and again under
OMP_NUM_THREADS=1. Every field is identical() before and after except a
handful on the joint fitter at n_threads = 4, which a control run showed is
not reproducible run to run under a FIXED build in either direction: a
num_threads(4) reduction combines its per-thread partial sums in an
unspecified order.
Cost off a corrected-integrated-Laplace M ladder on the non-joint fitter
(7-cell sigma grid, one thread, variant = "qmc", two runs per arm): the
per-objective-evaluation fixed cost falls 3.39 -> 1.31 us binomial, 3.66 ->
1.34 us poisson and 3.25 -> 1.39 us gaussian, and the per-observation cost
falls about 8% with it, since a plain loop keeps the eta assembly in the
caller's frame instead of an outlined region body. Per auxiliary draw at
N = 48: 9.96 -> 7.27, 8.97 -> 6.58 and 9.91 -> 7.77 us.
R::lchoose(n, y), three lgammafn calls, inside the
density; poisson spent lgammafn(y + 1); gaussian matched no special branch
and fell through the ladder to parse_family_link, which builds a
FamilyLink carrying two std::strings.BuiltinFamilyResponse::prepare() (and FamilyLogLik::prepare()) resolves
the family code once into a FamilyKind plus a parsed FamilyLink, and
fills a per-observation vector of the constant. log_lik_for_family and
grad_hess_for_family gain an entry point taking both, so the objective adds
a looked-up double where it called lgamma. Each ladder is written once, in
a core that the std::string overload forwards into after an
allocation-free classification, so a caller still passing a string does
exactly the work it did before and no path regresses. A response that was
never prepared reports so and takes the string route, so a construction site
that forgets the call loses the saving, never the constant.
The split densities are kernel + const in that order at every call site, so
a caller holding a precomputed constant reproduces them bit for bit. Measured
over 23 fits (ten families x single-arm and joint, plus three
corrected-integrated-Laplace fits) and 253 captured fields -- log-marginals,
modes, per-cell modes and Hessians, grid weights, coefficients, intervals,
vcov, logLik -- every field identical() before and after, no tolerance.
Cost, read off a corrected-integrated-Laplace M ladder (7-cell sigma grid,
one thread, variant = "qmc"), the two arms alternated so they share the
machine's load: per-observation cost of the objective 0.190 -> 0.149 us
binomial, 0.230 -> 0.138 us gaussian, 0.189 -> 0.118 us poisson. Per
auxiliary draw at N = 48 that is 9.9 -> 6.4, 11.3 -> 6.4 and 10.3 -> 5.7 us.
laplace_newton_solve_ll applied
center_effects_fn(x) after the loop converged and then re-evaluated eta,
the Hessian, the data log-likelihood and the log-prior at the shifted point,
so what it reported was a Laplace expansion at a non-stationary point. The
fold each caller applies preserves eta, which is why the data term never
moved; the log-prior is what does. A proper field prior is not
shift-invariant, and even an intrinsic one moves the beta ridge through the
coefficient the fold lands in, so the size of the error tracks how far the
field is from shift-invariant: score_max at the reported point measured
1.6e-05 on an RW1 fixture and 5.7e-02 on an AR1 one, both ~1e-14 now. That
propagated into every cell weight, since the outer grid weights cells by
softmax(log_marginal). The ordering is now the joint loops': log-marginal
and every inner-layer probe at the uncentered iterate, center_effects_fn
last and over the reported mode alone, with the correction presenting its
draws through the fold. Measured on the RW1 fixture, the reported
log-marginal moves 2.6e-09 and the weights 7.4e-11.tests/testthat/test-cila.R holds the non-joint gaussian arbiter to the
joint file's 1e-9, where it read 3.4e-07 before, and pins score_max.outside = "extend", so the interval reaches the outer CELL EDGE rather than
clamping at the extreme node, and .nl_cell_edges() mirrored the extreme
half-spacing in a coordinate GUESSED from the values -- log whenever they were
all positive, which a proportion is. A BYM2 mixing weight whose top default
node is 0.95 therefore got an upper edge of 1.0353, and the fit reported a
97.5% bound above 1 for a quantity that lives in (0, 1) and is singular at
both ends: 1.0028 on the default axis, 1.0012 after gcol33/tulpa#361 moves it.
The edge is now mirrored in the quantity's OWN coordinate, and that coordinate
is not a new notion of a domain: .NL_DOMAIN_TRANSFORM already defines
positive / unit / correlation / unbounded, and .joint_axis_domains()
already maps every outer axis onto one of them from the SAME per-axis registry
the outer Pareto-k unconstrains with -- it was simply consulted only on the
moment-rule read. It is now supplied whatever the support, on the registry,
multi-block, spatial-field and axis-summary paths; hyper_axis_spec()'s
declared bounds name the domain on the generic tulpa_hyper_grid() door.
positive reproduces the log mirroring exactly and unbounded the linear
one, so a scale axis is bit-identical; an axis whose support the registry will
not name (car_proper's rho_car on the adjacency eigenvalue interval) keeps
the guess rather than acquiring an invented support, and a node set that
contradicts the domain the caller named falls back to it too.car_proper fit -- whose rho_car support the transform
registry will not guess, which blocks placement for every axis of the fit
including its ordinary positive-scale tau -- was indistinguishable from a
fit the rescue never applied to. The rail REPORT ($outer_grid_railed_axes,
gcol33/tulpa#361) is now taken before anything can decline, since it reads
stored weights and needs neither curvature nor a scope entry, and the decline
names the blocking axis ("unguessable_axis: rho_car") or says the family
carries none the rescue is built for ("family_out_of_scope"). Both are READ
BACK rather than only recorded, per gcol33/tulpa#346: diagnostics() carries
grid_railed_axes / grid_placement / grid_recentred_axes /
grid_placement_declined, and print() shows a one-line reading beside the
gcol33/tulpa#276 regime note.A corrected fit carries ONE grid weighting (gcol33/tulpa#367). The corrected
per-cell masses are what fit$draws is pooled from, so they are the fit's
grid posterior and are now reported as fit$weights / fit$log_marginal,
with fit$weights_source naming the read. The pre-correction pair is kept as
fit$cila$laplace$weights / $log_marginal, a name that cannot be mistaken
for the reported one, and fit$cila$cell_weights / $cell_log_marginal are
the adopted vectors THEMSELVES -- identical(), not equal to a tolerance --
so a caller pairing the two cannot pair them wrongly. Each fitter's own
hyperparameter-summary tail is re-run on the adopted read, so theta_mean
and the axis quantiles describe the weights the fit reports rather than the
ones it replaced. A grid whose cells did not all produce a usable particle
set is read conditional on the cells that remain, the gcol33/tulpa#342
convention, with $cila$retained_mass recording the original share.
The correction reaches tulpa_nested_laplace(), the single-block non-joint
fitter (gcol33/tulpa#368). control$cila threads through
laplace_newton_solve_ll, spec_inner_solve,
run_multi_block_nested_laplace and every cpp_nested_laplace_* entry, so
icar / bym2 / car_proper / nngp / hsgp / rw1 / rw2 / ar1 / the ST variants and
the multi-block driver all carry it. That loop centres BEFORE the correction
runs and re-evaluates its factor at the centred point, so it proposes from
result.mode and presents its draws unchanged where the joint loops apply a
centring fold.
A sparsely factorized cell is corrected instead of declining
(gcol33/tulpa#366). A draw with covariance A^-1 is P' L^-T eps, which the
live CHOLMOD factor supplies directly through its own Lt and Pt solves --
no refactorization, no dense triangle, and a whole chunk of draws per call.
This opens the correction to SPDE, NNGP, ICAR at scale, MCAR, HSGP-MO and
latent factors, and to any fit above the sparse dispatch threshold. Only an
LL' factor carries that square root; a simplicial LDL' fallback declines by
name ("sparse_factor_not_ll") rather than drawing from the wrong covariance.
Every objective evaluation in the joint Newton loop was paying about 7.6
microseconds of OpenMP region entry it could not use (gcol33/tulpa#365). The
eta assembly and the data log-likelihood both guarded their parallel region
with if(n_threads > 1), which serialises the body but still enters libgomp;
at one thread they now take a plain loop, in the same index order, so the
numbers are unchanged. Measured on a 48-observation binomial joint fit: the
correction's marginal cost per auxiliary draw falls from 26.80 to 19.23
microseconds, and the line search pays the same saving on every fit. What
remains is 1.9 microseconds of the correction's own machinery (Sobol net,
normal quantile, triangular solve, bookkeeping), flat in the data size, plus
0.345 microseconds per observation inside the shared per-observation
likelihood -- 93% of the cost at that size, and not reachable by batching the
density call.
mu_i + sigma_i {gamma_1 + gamma_3 / 2 + w(z_p; gamma_3)}, so the correction
RELOCATES the marginal by that centre and cornish_fisher_eligible() bounded
only the reshaping: |gamma_3| was banded and the centre was accepted at any
size as long as it was finite. Since gcol33/tulpa#354 gave the centre a
location term the admitted displacement reached about one standard error on
the engine's own fixtures with nothing noticing. One predicate now owns the
whole band decision (cornish_fisher_in_band()), a coefficient past
centre_unreliable reports the Gaussian quantiles and records
centre_unreliable, and the eligibility record summary() / confint()
consume is READ from that same predicate rather than re-derived beside it.var(eta_j | x_i) and so gamma_1 nonzero -- because the existing fixtures
cannot reach a large centre (dev_notes/issue362/RESULTS.md).control$skew_correct still defaults to FALSE. Under the banded gate the
gcol33/tulpa#354 flip is unchanged: the centre band declines nothing on either
fixture the default-on decision rests on (paired CRPS t -1.895 on the
rare-event intercept, -3.77 / -3.20 on the small-group RE fit, against
gcol33/tulpa#302's +3.54 / +6.12 / +4.64).ess_grid over the tensor -- so on a crossed grid a second axis carrying
spread hid an axis hard against its own boundary; it also named one movable
axis per family and placed only in the log coordinate. A BYM2 rho railed
against its 0.95 ceiling was therefore detected (pareto_k_grid_edge_axes
names it) and then left there, with the fit recording grid_not_collapsed on
a grid that had collapsed. An axis is now railed when its OWN marginal -- the
one .nl_axis_quantiles() reports that axis's median and interval from -- is
maximal at an endpoint and that endpoint carries at least half the marginal
weight, which for a unimodal marginal is exactly the statement that the mode
is at or beyond the boundary. Every axis a family lists in
.NL_REGISTRY_AXIS_FIELD is movable on its own rail, in whichever coordinate
the engine's own transform registry gives it (log for a scale, logit01
for a mixing weight, mapped back into the OPEN interval). Every registry fit
records $outer_grid_railed_axes whether or not a rescue could move the
axis, so a span the engine may not touch -- a user pin above all -- says so
instead of silently integrating a tail. Provenance is unchanged and still
decides: a pinned axis is reported and never moved. MEASURED over
gcol33/tulpa#357's 34-configuration census, read as the shipped engine
reports it: railed axes 2 -> 0, recentred fits 2 -> 6, reported median and
interval moving on 3 rows. On a fixed-truth BYM2 sweep (100 regions, rho in
{0.70, 0.90, 0.97, 0.99}, 60 seeds, both arms the shipped engine of their own
build) mean absolute deviation from nominal coverage goes 0.229 -> 0.075 at
the 50% level and 0.046 -> 0.038 at 95%, the median's bias shrinks at every
truth, and the 95% interval narrows 0.548 -> 0.508 at equal coverage; at
rho = 0.99 the 50% interval went from containing the truth on 3 of 60
replicates to 32 of 60. Against a reference read on an axis that contains the
posterior, the moved axis is nearer on 84 of 84 replicates whose read
changed. dev_notes/issue361/RESULTS.md.n_points points from the WHOLE inner
Gaussian, weights each by the exact joint density it came from, and reports
the weighted particles; the cell marginal and the latent posterior both
converge to the exact ones as the effort grows, so n_points is the only
dial. control$cila on tulpa_nested_laplace_joint(), off by default.MEASURED on gcol33/tulpa#341's fixture B -- a small-group Bernoulli GLMM over
a 7-point hyperparameter grid, which is the shape of a real consumer fit --
at 400 prior-predictive replicates scored by tests/testthat/helper-sbc.R
against a machine-accurate quadrature. Paired CRPS against that quadrature on
the intercept:
| arm | CRPS - exact | t | SBC p_unif | |---|---|---|---| | shipped nested read | +0.0447 | 4.44 | 1.6e-03 | | + subspace debias | +0.0191 | 3.17 | 0.017 | | + cila, M = 1024 | +0.00090 | 1.10 | 0.15 | | + cila, M = 4096 | +0.000033 | 0.05 | 0.80 |
The slope behaves the same way (subspace +0.0138 at t = 2.38, cila at
M = 1024 +0.00027 at t = 0.30), and the hyperparameter, already correct
before, stays correct. All three variants sit inside k < 0.7 on every one of
the 400 replicates, median 0.42 / 0.51 / 0.51 for qmc / is / rqmc.
COST on the same fixture, medians over 8 seeds at 3 repetitions: M = 1024 is 10.2x a plain fit and M = 4096 is 37.6x, against the subspace debias's 26.1x. The correction is one extra pass over the settled grid plus M evaluations of the Newton loop's own penalized objective per cell, and that closure takes one latent vector at a time, so the M evaluations dominate and are what a batched density interface would address.
The auxiliary point set is a NATIVE Sobol' net -- Joe & Kuo direction numbers
for 1024 dimensions compiled in, Gray-code recurrence, no new dependency
(src/sobol.h). Its low-discrepancy property is asserted against Monte Carlo
rather than assumed: on a smooth integrand the QMC error slope beats the plain
MC slope by 0.51 / 0.51 / 0.35 / 0.21 at d = 1 / 2 / 14 / 18, and the first
2^m - 1 points fill every elementary interval of every one-dimensional
projection exactly once. Past the tabulated dimension the request falls back
to iid draws and records fallback = "sobol_dim_exceeded", so a fit never
reports a variant it did not run.
The correction's randomness is ENGINE-OWNED, not R's. Requesting it leaves
every other posterior quantity on the fit bit-for-bit unchanged, the outer
grid stays parallel-integrable (the subspace debias forces a serial grid
precisely because it draws from R), and the reported k-hat does not flap with
the seed; control$cila$seed selects an independent realization of the
randomized-QMC shifts.
Below 512 points per cell the request is REFUSED rather than served. Proposition 5's recovered posterior is literally a weighted particle set, so a truth outside the particle range gets PIT exactly 0 or 1 -- gcol33/tulpa#341 measured every variant leaving the simultaneous calibration band at M = 64 and the iid one still leaving it at M = 256.
Every decline says why. A cell whose inner solve factorized SPARSELY reports
"sparse_factor_unavailable", since drawing from the inner Gaussian needs a
triangular solve against the factor and the sparse solver exposes only the
full solve; the R attach reports "no_fixed_effects", "no_grid_weights",
"redispatch_failed", or the first cell's own reason where nothing usable
came back.
auto_grid() coerces in place instead of through as.numeric(), so a
MATRIX-valued axis survives the mark (gcol33/tulpa#360). Two families store
their axis as a matrix of pre-paired coordinates -- mcar / miid's
logchol_grid, tgmrf's theta_grid_built -- and flattening one destroys
the axis it is declaring, which left those families with no way to declare a
default at all: a wrapper package computing its own coarse logchol_grid
had to choose between handing the driver a flattened vector and leaving the
axis undeclared, where the provenance predicate reads it as a user pin.
dim() and dimnames() now come through, and the marked matrix is carried
by every consumer of the mark -- .nl_grid_provenance(), the pinned/default
predicate, the gcol33/tulpa#352 consumption check and .nl_block_axis_grid().
A grid the COPY SPEC carries and the copy resolver does not read is refused
or recorded rather than dropped in silence (gcol33/tulpaObs#192). gcol33/tulpa#352
walks the prior blocks; the copy spec is the other object the multi-block
joint driver resolves an axis off, and nothing walked it.
.resolve_one_copy_spec() reads arm, block and alpha_grid, and a
numeric field beyond those now takes the same provenance split every unread
axis takes: PINNED is refused, naming the field and the (sigma, alpha)
parameterization the copy path integrates; an auto_grid()-marked or
engine-default value is dropped and recorded on $axis_fields_dropped. The
reported case was a sigma_pos_grid from the retired (sigma_occ, sigma_pos)
parameterization, pinned by the user and answered with a bit-identical
log_marginal. .NL_COPY_SPEC_FIELDS is the read-field set, held against
the resolver's own body by a source lint.
A node set that is an MCMC SAMPLE is named as one, and no longer borrows a
cell partition's outer half-cell (gcol33/tulpa#358). support = "density"
covered two geometries: a tensor grid, whose values are cell representatives
of a partition with known spacing and whose extreme cell's mass reaches half a
spacing past its coordinate, and tulpa_re_cov_gibbs()'s equal-weight
posterior draws, whose values are order statistics with an unknown tail beyond
the largest of them. gcol33/tulpa#353's outside = "extend" is derived for the
first and only harmless on the second, and a caller reading the tag could not
tell which it had. The kinds are now one table, .NL_SUPPORT, naming each
one's outer-edge policy exactly once -- density and mixed extend,
sample clamps, and moment_rule carries none because it never reaches the
quantile read -- with .NL_SUPPORT_KINDS the single vocabulary every formal
reads, so adding a kind is one entry rather than four hand-written lists. The
producer names itself: .nl_node_support() maps integration = "sample" to
the new kind, the Gibbs fit carries that tag, and tulpa_re_cov_nested()'s
hand-written copy of the same producer-to-kind decision now calls the shared
reader. Nothing reported moves -- the policies differ only outside
[1 / (2 n), 1 - 1 / (2 n)] and the backends report 0.025 / 0.5 / 0.975 --
which tests/testthat/test-support-sample.R shows rather than asserts, on the
dispatcher, on the whole derived-Sigma summary table, and on a real Gibbs fit
whose reported posterior is identical to the pre-change read recomputed on its
own draws. A misspelled kind hard-errors instead of being translated.
The tulpa.kdiag.capture aperture publishes the importance log-ratios the
REPORTED outer Pareto-k-hat was fitted on (gcol33/tulpa#356). It was written
by .nested_is_pareto_k(), the scoring primitive, which runs once per
CANDIDATE proposal -- and the joint dispatch scores several per reported
k-hat (each moment-matching pass, then the grid mixture, then the skew-normal
rescue), keeping whichever gives the lowest shape. A candidate scored after
the winner left its ratios in the aperture, so an external check refitting the
GPD on cap$lr landed on a different number from the fit's own: a spatial
occu_cover() fit reported 0.7714820579 against 1.197156391 recomputed, with
loo::psis() agreeing with the recomputation. The reported value was the
correct one throughout -- it is exactly tulpa_psis() on the grid-moment
pass's ratios, the proposal the dispatch selected, while the second
moment-matching pass scored 1.197156391, lost the comparison, and was the last
thing written. The primitive now returns its ratios on lr and publishes
nothing; each backend publishes once, from the proposal it selected, through
the single .kdiag_capture() helper, alongside the tail_points request that
resolves the GPD tail size. So
tulpa_psis(cap$lr, tail_points = cap$tail_points)$pareto_k reproduces
fit$pareto_k exactly on every backend that reports an outer k-hat --
tulpa_nested_laplace(), tulpa_nested_laplace_joint() single-block and
multi-block, fit_spde(), tulpa_re_cov_nested() -- which
tests/testthat/test-kdiag-capture.R pins. The defect was confined to the
joint path, the only one that scores a proposal it can discard. No reported
k-hat moves: the aperture consumes no RNG and the scoring is untouched.
The inner-Laplace LOCATION term gamma_1 is computed, and the skew
correction is applied about the centre it defines (gcol33/tulpa#354). Rue,
Martino & Chopin (2009) Sec 3.2.3 carry two terms; this engine had only the
cubic one, and src/inner_laplace_skew.h's SCOPE note called the other
blocked because its likelihood-curvature perturbation is diagonal only in the
paper's augmented representation. It is not blocked. Expanding the denominator
log-determinant along the same conditional-mean curve gives
gamma_1(i) = (1/2)[(1/sigma_i) sum_j l'''_j s_j u_{i,j} - gamma_3(i)], whose
one new quantity is s_j = [A Sigma A']_jj, the marginal variance of the
linear predictor -- index-independent, so one pass per fit, and read off exact
affine eta differences plus the full solves the live factor already serves
rather than by assembling A. Verified against a central difference of the
log-determinant on an independently written model (3e-11 relative) and against
the paper's own eq. (21) line 1 at A = I over a coupled GMRF prior (1e-15,
term for term). Bounded by INNER_ETA_VAR_MAX_SOLVES; a coupled or
multi-process unit declines with multi_eta_unit, never with a zero.
The control$skew_correct marginal is now a NET GAIN over the uncorrected
Laplace, which gcol33/tulpa#346 measured it not to be. The cause was the
centre, not the location term alone: z + (g/6)(z^2 - 1) is the quantile
function of a MEAN-ZERO variate, while RMC eq. (22) has mean
gamma_1 + gamma_3 / 2, so placing the reshaped variate at the Laplace mode
asserted gamma_1 = -gamma_3/2. On the #346 fixture gamma_1 is identically
0 and the whole missing centre was the mean the cubic term itself induces.
Measured on the same 400 prior-predictive replicates: paired CRPS
-0.01643 at t = -1.89 against the uncorrected Laplace, essentially all of
the -0.01662 the exact posterior achieves, where the previous read scored
+0.00775 at t = +3.54; SBC uniformity 0.0833 -> 0.0329 against an exact
0.0290, with the PIT re-entering the simultaneous band; endpoint error
reduction 56.6% -> 77.4%. The correction also beats the shift only control
arm, so the cubic term earns its place once the centre is right. The gate's
pin moved from expect_gt(dlt("skew_cf"), 0) to expect_lt(..., 0), and the
old read is kept as a no centre control that reproduces the loss exactly.
gamma_1 is required: a coefficient without one declines with
gamma1_not_computable rather than being corrected about an assumed centre.
The default stays FALSE.
$axis_fields_dropped when the axis was an engine default or an
auto_grid() the resolved path does not read; nothing read it back, so such a
fit reported exactly as one whose axis was integrated. diagnostic_summary()
now carries the record and one sentence per dropped field -- the field, the
block, the path named with the same label the #352 refusal uses, and the axis
that path integrated instead; print() carries the compact unused axis
fields line; and summary() carries the record as an axis_fields_dropped
attribute alongside interval_source / retained_mass. All three go through
one reader (.tulpa_axis_dropped()) and one formatter, so a fit that used
every axis it was given is silent in all three and the ordinary fit is
unchanged. This is the read half of the gcol33/tulpa#293 rule that a
silent-disable path needs a reason field.The hyperparameter-axis interval reaches the outer cells' own edges instead of
stopping at the extreme grid coordinate (gcol33/tulpa#353).
.nl_wtd_quantile() places the cumulative mid-mass cumsum(w) - w/2 at each
cell coordinate, which leaves w_1 / 2 of the mass below the first coordinate
and w_n / 2 above the last; the clamp had nowhere to put it and returned the
extreme coordinate for every probability there. A grid's values are cell
representatives with known spacing, not order statistics, so the missing
support is the outer cells' own half-width: outside = "extend" mirrors it,
in log on a positive axis and in the value otherwise, and a density
support dispatches to it. Nothing inside [values[1], values[n]] moves, so a
fit whose reported probabilities never reached the clamp is byte-identical.
Measured over a prior-predictive experiment whose grid tiles the prior, the
shipped read covered every truth inside its support and none outside it
(cover95 + pit_extreme = 1.0000 exactly at 4, 5 and 7 levels): 95% coverage
goes from 0.743 to 0.989 against nominal 0.95, the PIT atom at 0 and 1 from
0.257 to 0, the paired CRPS improves at every resolution, and the read's order
of convergence to the exact posterior rises from 1 to 2. It does not make the
read calibrated -- the clamp was partly cancelling a separate interior
over-width -- and dev_notes/issue353/RESULTS.md reports the instruments that
come out against the change alongside those that support it.
The coupled joint Newton takes a step from an indefinite start Hessian, and a
fit that stalls reports the cause that stopped it (gcol33/tulpa#344). The
occupancy mixture's dark-cell term log(psi (1-p)^J + 1 - psi) is not concave
in (eta_occ, eta_det), so a data set with few detections has a negative
curvature direction at the Newton start x = 0. The SPARSE joint loop
conditions the Hessian there (joint_pd_step_solve); the DENSE one -- which
every small coupled fit runs on, n_x below the sparse threshold -- did not,
and pd_mode never even reached it, so control$hessian = "psd" was inert on
that path. A plain Cholesky of an indefinite matrix returns a non-finite step,
the line search accepts nothing, and the loop reported its START VECTOR as the
mode: on the engine's own coupled fixture, 32.5% of prior-predictive draws at
prior_sd = 2, n_visits = 4 returned (0, 0) with score_max after 5000
iterations equal to its value after 1, and neither the budget nor the
tolerance was binding.
The two PD policies now live in one place (src/joint_pd_step.h) and both
loops are backends for them: pd_lm_escalate() is the smallest diagonal load
making the factorization succeed (Nocedal & Wright, Numerical Optimization
2nd ed., Alg. 3.3), pd_eigen_clamp_solve() the clamped spectrum. With H
already PD the first attempt succeeds, no load is added, and the step IS the
plain Newton step, so every fit that factorized before is unchanged -- the
dense and sparse loops agree to 4e-16 on the mode and exactly on
log_marginal for both an indefinite and a PD start. Measured over the
issue's own 80-replicate sweep, non-convergence goes from 32.5 / 22.5 / 13.8 /
7.5 / 3.8% (the five affected configurations) to 0% in all ten, each converged
fit landing within 1.9e-06 of an independently optimized mode in a median of
4-7 Newton iterations.
Three reports named the wrong layer for it, the gcol33/tulpa#293 shape one
layer over. inner_skew_declined said backend_unsupported, which is false
-- the same backend computes gamma_3 on every converged solve of the same
fixture; grid_fixed_declined said block_not_extracted, a retention step
downstream of the real cause. Both now say not_converged, which is also a
reason in the shared Pareto-k vocabulary, and the three probe drivers settle
it in one place (.inner_skew_attach_probe()) rather than each writing its
own tail. $modes is kept -- it is the solver's record of where it stopped,
and the warm-start chain and the refinement passes read it -- but the
per-cell-mode AVERAGE every coefficient report falls back on now reads only
cells that reached a mode, so a stalled fit reports NA with
interval_declined = "not_converged" instead of its own start vector as an
estimate. A non-PD Hessian at the returned point also withholds the stored
precision and the fixed-effect block, whose inverse is not a covariance there.
New test-coupled-indefinite-start.R carries the sparse-detection fixture the
suite lacked: every coupled fixture sat at b_occ = 0.2, b_det = -0.5, where
the start Hessian is positive definite, which is why nothing reached the path.
It establishes that it reaches the regime -- lambda_min < -5 read off the R
log posterior, with the old setting asserted PD as the control -- rather than
reaching it by luck.
sigma_grid on an icar block reached
the multi-block driver, which integrates tau_grid, and the field neither took
effect nor produced any diagnostic -- so a fixture believing it pinned a 2-node
axis per block ran a 9-node default on a different parameterization, at an
order of magnitude more cells. The two paths keep their parameterizations: the
registry integrates the intrinsic families over the PRECISION and the joint
areal backends over the field SD, which .NL_FAMILY_AXES already recorded as
deliberate. What changes is that the binding table now also says which fields
each PATH reads (.NL_PATH_AXES, R/settings.R), so the check is one pass
over the whole table rather than a rule per family, and a family whose drivers
agree needs no entry at all. Both nested-Laplace front doors run it on the
incoming prior, resolving the path per block (registry / single-block joint
areal / copy). The verdict splits on the same provenance question every
auto-recenter rescue asks: a PINNED unread axis is an error naming the field,
the block, the path, the axis that path integrates and -- for the one pair the
engine itself converts, icar's tau = 1 / sigma^2 -- how to write the same
grid in the axis that path reads; an axis that IS an engine default carries
nothing a pin would add, so it is dropped and the drop is recorded on the fit
as $axis_fields_dropped rather than left invisible (gcol33/tulpa#293). The
refusal is symmetric: a tau_grid handed to the single-block joint areal
backend was ignored the same way and is refused too. Six fixtures in this
repo were in that position and now pin the axis they meant
(test-inner-skew.R, test-checkpoint-universal.R,
test-nested-laplace-spatial-recovery.R, test-nested-laplace-joint-svc-areal.R,
test-svc-fold.R, test-audit-228-239.R). New test-nl-axis-consumption.R,
which measures the declared conversion against a fit rather than asserting it
from the source.recov_posterior_sbc() derives two seeds and hands
draw_theta one and simulate the other, so the obvious set.seed(seed) at
the top of each callback is the CORRECT fixture and none of them carries an
offset any more. Under the previous contract both callbacks got the same seed,
and a fixture writing the obvious thing drew the replicate's group effects and
residuals from the very uniforms that produced theta' -- not p(y | theta'),
and a non-uniform PIT with nothing wrong in the inference under test. The
driver applies the 660000L offset every fixture used to apply itself, so the
seeds a fixture sees did not move, and the #339 measurement in
dev_notes/issue339/ was re-run under it.sigma (gamma_3 / 6) (z_p^2 - 1) takes the same value at both
ends, so there the correction is a pure location shift of the interval with
its width unchanged. Section 4 of test-inner-skew-correction.R adds a
whole-marginal gate on the gcol33/tulpa#335 harness (recov_sbc(),
sbc_report(), sbc_crps_compare()), with a shift only control arm that
separates the shift from the reshaping. Measured over 400 prior-predictive
replicates read off one solve per seed: the endpoint score improves 56.6%
while the paired CRPS against the exact posterior is a net loss of +0.00775
at t = +3.54, and the shift alone scores -0.0145, which is what a Gaussian at
the exact mean and standard deviation achieves. The correction stays off by
default; the missing piece is the location term, not the cubic one.
.nl_skew_correction_attach() now gates each coefficient on the combined
inner band -- the worse of gamma_3's band and the inner importance k-hat's,
the resolution gcol33/tulpa#304's selector already reads -- and records the
k-hat, the combined band, a reason from a closed vocabulary and the
whole-fit reliability verdict alongside the eligibility.The per-cell fixed-effect retention on the joint tier indexes the same grid
as the weights, on every grid (gcol33/tulpa#345). .nested_fixed_moments()
reads $weights, $grid_modes and $grid_hessians as three views of one
grid, and a caller pairing weights[k] with grid_modes[[k]] makes the same
assumption with no guard at all. .joint_attach_grid_fixed() built the pair
cell by cell and wrote NULL into the slot of a cell whose inner solve
returned no usable block -- which on an R list REMOVES the element instead of
leaving it empty. An interior empty cell was invisible, since the next write
re-extended the list; a TRAILING one left the pair one shorter than the grid,
so the marginalizer's length check declined and every fixed-effect estimate,
standard error and interval on the fit came back NA. The slot is now
skipped rather than assigned, which is what the marginalizer's zero-weight
skip and the gcol33/tulpa#342 renormalization were already written for.
Adaptive refinement is what makes the trailing case reachable: it appends
cells at the end of the grid, and an appended cell whose solve returns a
non-finite marginal carries zero weight and no block.
A joint fit with no coefficient table says why it has none
(gcol33/tulpa#293's lesson applied here). A retention holding no cell the
weights put mass on -- every block unusable, or an all-NA weight vector --
now records grid_fixed_declined = "no_weighted_cell_block" instead of
handing back a pair .nested_fixed_moments() silently returns NULL on with
grid_fixed_declined reporting NA.
tulpa_posterior_draws() serves a single-block nested-Laplace fit
(gcol33/tulpa#347). The posterior of such a fit IS the outer-grid Gaussian
mixture .nested_fixed_moments() returns, so a draw is the same two steps the
joint backend already runs -- pick a cell by weight, then draw that cell's
Gaussian -- and the two backends now meet at one allocator,
.nl_mixture_draw(). What a draw covers differs by what the backend retained
and travels on the result as a scope attribute: the joint fit keeps each
cell's sparse precision and so samples the full latent vector ("latent"),
while the single-block fit inverts each cell's precision into the marginal
fixed-effect block and releases the precision, so it samples that block
("fixed"). A grid that dropped a positive-weight cell samples the cells that
remain and carries the same retained_mass provenance confint() does
(gcol33/tulpa#342). Measured against the retained mixture at 4e5 draws: the
sampled mean and covariance reproduce .nested_fixed_moments() to 2.9e-04 and
3.1e-03 marginal-SD units, and the sampled 2.5% / 97.5% quantiles reproduce
confint()'s mixture-CDF bounds to 3.3e-03 marginal SDs.diagnostics() reports the reliability band on a fit that carries no draws
(gcol33/tulpa#348). Draws are what the per-parameter mean / sd / ESS / rhat
columns and n_draws are computed from; the outer PSIS k-hat and its regime,
the grid quadrature ESS, the inner-Laplace gamma_3 and importance k-hat, and
the combined verdict are all read off the fit and need none. The guard was on
the wrong quantity, so a default single-block nested fit whose pareto_k had
cleared the escalation threshold reported NULL. The band is now returned with
an empty body, n_draws = NA, and a param_table_declined reason that points
at tulpa_posterior_draws(); print() shows the band and says the columns
need a posterior sample. A fit with neither draws nor any band quantity (a
plain Laplace fit, no outer grid to score) still returns NULL, now saying
both halves of why.posterior_sample() and tulpa_draws_array() name what a draws-less fit
carries instead of returning a bare NULL (gcol33/tulpa#349). The note gives
the backend, the posterior representation on the fit -- the outer-grid mixture
over the latent vector or over the fixed effects, per-cell modes with no
retained covariance, a Laplace mode and precision, or posterior moments -- and
the accessor that samples it where one exists. The internal probes that read
draws with a designed fallback (the posterior interop conversion, k-fold,
power-scaling) go through the silent accessors, so a fallback does not narrate
itself.control$max_grid_cells sets the multi-block outer-grid cell ceiling
(gcol33/tulpa#343), defaulting to the 2048 that was hard-wired. The cap is a
cost guard -- each cell is one inner Newton solve -- and a good default, but a
converged tensor REFERENCE grid is the case a caller knowingly wants the
expensive run: 4 axes at 7 levels is 2401 cells, so no cubic reference above 6
levels per axis could be built on a 4-axis fit, and the outer-grid line's
reference fits had to lift the package internal in-process. Both enforcement
sites -- the multi-block nested-Laplace dispatch and the joint multi-block
dispatch, until now independent copies of the same comparison -- go through
one .nl_check_grid_cap() reading one resolved value, carried for the fit on
a scoped option so every re-dispatch inside it (the k-hat re-evaluations, the
refinement passes) enforces the caller's ceiling. The error keeps each site's
remedy for the accidental blow-up and names the override for the deliberate
run; the >50 cell warning is unchanged.
Posterior SBC joins the prior-predictive harness: calibration checked
CONDITIONAL on an observed data set rather than averaged over the prior
(gcol33/tulpa#339, after Sailynoja, Schmitt, Buerkner & Vehtari, Statistics
and Computing 36:78 2026). The truth is drawn from pi(theta | y_obs), a
replicate from pi(y | theta'), and the PIT taken under the augmented
posterior pi(theta | y, y_obs) -- ordinary SBC with the posterior at y_obs
in the role of the prior, so #335's simultaneous band, folded read, randomized
discrete PIT and CRPS closed forms all carry over and truth =
"posterior_draw" is a proper-score experiment. It also scores what prior SBC
on this engine cannot: the nested door puts no prior on the fixed effects, and
an improper prior cannot be drawn from.
test-posterior-sbc.R). Run
with the exact posterior at both stages on a gaussian random-intercept fixture
it must be uniform, and both premises have negative controls that fail when
broken -- fitting the replicate alone instead of pooling takes the
hyperparameter outside the band, and re-observing the observed regions with
their own effects takes the intercept, the hyperparameter and the joint
log-likelihood outside it while the slope survives.dev_notes/issue339/: 15 configurations at N = 1000 including
occu_cover, a pre-registered family-wise verdict rule, and a power curve
(80% power at roughly a 10% over-dispersion or a 0.14-SD location bias;
measured false-positive rate 0.0117 against a nominal 0.05). The shipped
reliability band and the calibration check disagree in BOTH directions:
pois_40, which the band calls reliable (both layers good) on an outer
k-hat of 0.196, fails at p = 2.3e-13; binom_30, whose outer k-hat of 1.413
is well past the escalation threshold, passes at p = 0.17. The band is a
screen, not a verdict.recov_draw_y() in the recovery sweep now
delegates to sbc_draw_y(), which sits beside its own log density, so a family
cannot be simulated under one parameterization and scored under another.
Bit-identical across all six families.recov_sbc() draws the truth from the prior instead of fixing it and reads
the whole PIT ECDF; sbc_crps() scores calibration and sharpness together.
Both live in tests/testthat/helper-sbc.R, test-side and tier 3, alongside
the sweeps rather than replacing them.P(all U <= t) = t^n), brute-force simulation, measured
simultaneous coverage, and the published Kolmogorov critical value, which its
constant-width member reproduces to four figures.rank / n_ref against a continuous uniform is a silent miscalibration and is
kept as a negative control.recov_sbc() records which experiment
produced a result and sbc_crps_compare() refuses to rank a fixed-truth one,
where the CRPS-optimal forecast is a point mass at the truth.dev_notes/issue335/RESULTS.md..nested_fixed_moments() renormalizes the grid weights over the cells that
retained a fixed-effect block (gcol33/tulpa#342). It normalized over the whole
grid and then summed over the retained cells only, so a cell that carried
positive weight and no retained block left the mean shrunk toward the origin
by exactly the dropped mass, and the covariance misstated to match. A dropped
cell now gives the moments of a grid that never held it.mass on the moments, and a
retained_mass attribute on confint() / summary(), give the original
retained share of the grid weight -- 1 on a complete grid, below 1 on a
repaired one. A reader tells the two apart from the fit alone.interval_source reports "mixture_cdf". The reachability is
unchanged from #342: a healthy fit retains every cell, so no reported number
moves on one.sum_k w_k N(mu_kj, V_kjj), instead
of being read off the single Gaussian matching that mixture's first two
moments (gcol33/tulpa#336). A mean and a variance are linear functionals and
survive the collapse; a quantile is not and does not. estimate,
std.error, vcov() and the debias selectors are the same moments they
were -- only the quantile step changed, so any difference in an interval is
attributable to the marginal read and to nothing else..nl_gauss_mixture_summary()
already inverted this CDF by bisection, and ranef() already reported the
per-group posterior with it; the fixed effects on the same fit collapsed
first. .nested_fixed_moments() now returns the components alongside the two
moments and .nl_fixed_interval() chooses the read, so there is one mixture
summarizer rather than a second one written for the fixed effects.tulpa_posterior_draws(), which samples a cell by weight and then that
cell's Gaussian and so realizes the same mixture; and the reduction cases,
where a Gaussian-equivalent mixture (one cell, or several with identical
component means and variances at any weights) reproduces the previous
interval exactly. A merely symmetric mixture is not one of those cases and is
free to differ in width -- 0.5 N(-2, 1) + 0.5 N(2, 1) has a 95% interval
near +/- 3.64 where matching its moments gives +/- 1.96 sqrt(5).gamma_3 is computed by re-dispatching the kernel at the
fitted MAP cell, so a fit retains one value per coefficient and not one per
cell, and the composed marginal sum_k w_k F^CF_kj is not identified by
retained state. The two address different non-Gaussianities: this one across
cells, #302 within the MAP cell. confint() and summary() carry
interval_source ("mixture_cdf" / "gaussian_moment" / "skew_map_cell")
and interval_declined, so a fit says which read produced its bounds and why.recov_sweep() scores all three reads off one solve per seed, so the
collapsed Gaussian remains the baseline the #302 correction is judged against
and the mixture read is a third paired arm. Measured over 60 seeds at 2
coefficients each: coverage does not move on the six-family configurations
(binomial 114/120 both arms, poisson 111/120 both, 0 discordant seeds of 120
in each), and on the skewed small-group fixture the mixture arm covers 119/120
against 118/120 on 1 discordant seed. Coverage therefore cannot separate the
two reads at any affordable seed count here, the same power limit
gcol33/tulpa#331 hit on the fixed effects, so the width is what is measurable:
the mixture interval is wider by 4.0e-04 and 4.2e-04 on average on those two
configurations and by 5.4e-03 on the skewed one, positive on every seed. This
lands as a consistency correction whose measured effect is small, not as a
calibration gain -- the collapsed Gaussian is a lossy compression of a
posterior the engine had already computed, and the two agreed closely wherever
the grid was near-Gaussian in the fixed-effect marginal.sigma_1 width ratio is 0.4776 / 0.5840 / 0.4465, flat in resolution rather
than decaying, because each atom moves a share of its own box and that share
does not shrink as the boxes do -- and covers 128 / 129 / 118 of 200 against
the shipped 200 / 200 / 200. The mass rule never moves a sigma_1 trial and
widens that interval by 4.1% to 9.8%.boundary, no_factor, cancellation and, for the
barycentre, out_of_box, tallied per cell on both sides of the refinement
gate. n_axes_declined is unchanged and rides alongside. The reasons are
taken at the same constants that already gate the refusals rather than
re-derived afterwards, and each one is triggered by a real fixture rather than
shipped as an unobserved string -- out_of_box turning out to be unreachable
through the closed form by construction, since its error bound sits under the
in-box slack.recov_draw_y() drew rnorm(N, 0, sqrt(phi)) --
the residual-VARIANCE convention tulpa() takes and converts at the boundary
-- while every fitter in the file goes to a direct door, which reads phi as
the residual SD. RESID_SD now names the gaussian configurations in the door's
own parameterization, leaving the data-generating process byte-identical, and a
new test pins the convention against the engine through the closed form
se = phi / sqrt(Sxx) at three phi values across both doors, so a future
crossing fails loudly instead of silently rescaling. A suite-wide sweep found
this fixture was the only crossing, in either direction.recov_fit_joint_phi_crossed() keeps the
defect runnable and reproduces the old table exactly, so the new floor is
demonstrated to catch it: a 40-seed draw at the crossed rate fails with
probability 0.714, against 0.0010 spurious at the corrected rate.gamma_3 being exactly 0 on every one of 300 gaussian fits (the
log-likelihood is quadratic in eta, so the inner Laplace IS the conditional
posterior) at realized importance efficiency 1.0000.pnorm differences, verified to 6.1e-14 against
integrate() and an independent Simpson rule. The decline is per AXIS rather
than per cell -- the integral over a bounded box is finite at either sign of
the curvature -- so a cell convex on one axis keeps the exact factor on the
others.integrate(). Hyperparameter axes
only: a fixed effect is marginalized from each cell's own inner solve, so
moving that cell asks what beta would have been at the moved point.|M_1 - M_0| factorizes as
w_c |exp(r_c) - 1| and the two factors are separated by orders of magnitude,
so the product is ordered by the integration weight before any predictor is
consulted: the incumbent ranking captures 0.8173 of the realized correction in
its top decile against an oracle's 0.8231, and weighting it by the predictor
moves it away. The two readings of "realized correction" order cells oppositely
(Spearman -0.700). Recorded with the derivation, and with the scale hierarchy
it is conditional on. The cells where the predictor fails are not the cells
misfit flags (correlation -0.015), so shape and mass remain orthogonal.outer_grid_dump() carries the per-cell modes and
Hessians and outer_grid_rebuild_fixed() hands them to
.nested_fixed_moments() itself, so a candidate weight rule is scored against
a coefficient's interval and not only a hyperparameter's, at zero fits and
through the engine's own marginalizer. A fit that declined retention says so
rather than returning NULL.outer_grid_weight_report() scores every part of a read against the noise
floor (gcol33/tulpa#330). The verdict set is derived from the same part list
the difference and the floor are built from, so a part cannot gain a difference
and a floor without gaining a verdict -- which is how the median, the one part
the gcol33/tulpa#326 rule moves, had none.log_mass_ratio as the per-cell reading, and records what
the asymmetry costs in coverage rather than in grid-internal metrics.iid block, so it has one latent axis and
.joint_local_ccd_engage() -- which requires four -- declined silently: every
number measured on this path over four rounds came from grid dumps rather than
from a coverage sweep. recov_fit_joint_local_ccd() is that sweep at four
crossed groupings on a deliberately coarse four-level base grid, run twice on
the same seeds with the refinement on and off, and a companion test asserts the
refinement actually engaged rather than trusting the settings.sigma_1 interval is more than
fourfold sharper (0.2330 against 1.0590) with half the median bias (0.0245
against 0.0592) while still covering 149 of 150 against a nominal 0.95.
These rates are superseded: they were measured on a fixture that fitted its
gaussian arms at a quarter of their own residual variance (gcol33/tulpa#332).
The conclusion held on re-measurement; the numbers did not. See 0.0.151.recov_sweep() takes the nominal level to judge at (z / level, defaulting
to the 95% every existing gate runs at) and reports mean interval width, so a
second level and the width effect are read through the shared harness rather
than a parallel one.Delta_c exp(ell_c) the base grid gave it against the refined cloud
Delta_c sum_j delta_j exp(ell_j) its own nodes give it, as
log_mass_ratio = logSumExp_j(log delta_j + ell_j - ell_c) over the full node
set including the centre. That comparison is the embedded-rule local error
indicator classical adaptive cubature uses to decide whether a subdivided
region's estimate is comparable to its unrefined siblings', and this grid
evaluates both rules already, so it costs no inner solve. Recorded per cell on
local_ccd_info with the two masses it is formed from and max_node_weight,
the share the single largest node takes of its own cell's refined mass, for
refined and declined cells alike.mode_gain = 0.5 g' (-H)^-1 g is the nats the quadratic model predicts the
log-density gains by moving the expansion centre to the cell's own fitted
peak. The plain gradient norm offset says nothing about how sharply the
log-marginal falls away in the direction it points, so two cells with the same
norm and curvature an order of magnitude apart are displaced by very different
amounts; the scaled form is comparable across cells and the unscaled one is
kept for the cases the scaled one cannot be formed in. mode_gain declines to
NA where -H is not positive definite: a cell whose fitted quadratic is not
concave has no interior peak to be displaced from.misfit is
non-quadraticity, offset / mode_gain are off-centring, log_mass_ratio is
mass correction, and skew_max reads misfit and nothing else.tests/testthat/helper-outer-grid-dump.R, gcol33/tulpa#322). A candidate
construction for the outer integration weights is pure post-processing of a
fit that already ran, so outer_grid_dump() writes the grid state
(joint_grid, log_marginal, dnode, weight_kind, the axis tags and
domains, the support the read was taken off, and the summary the fit shipped)
and outer_grid_rebuild() re-reads the per-axis summary under any weight
vector. The read goes through the engine's own .nl_axis_quantiles() ->
.nl_summary_quantile(), never a second copy of it, and the round-trip
assertion in test-outer-grid-dump.R -- rebuild-with-own-weights equals the
shipped read -- is what makes an offline difference attributable to the
weights alone. It holds exactly (0.000e+00) on a tensor grid, a global CCD and
a locally refined grid.outer_grid_noise_floor() estimates the scale below which a difference
between two reads is not resolved by the grid, as the spread of the read under
a weight-preserving coarsening of each axis's own atom set (consecutive atoms
merged at their weighted mean carrying their summed weight). Total mass and
each group's first moment are exactly preserved, so only resolution is
removed. On a one-axis dump with a Gaussian outer log-marginal the floor
bounds the read's true error against the closed-form quantiles at every
resolution from 9 to 81 levels.dnode on the fit beside the integration
weights it was folded into. Recovering it afterwards is a division by
exp(log_marginal), which loses the scale and is undefined on a cell whose
inner solve returned no finite marginal.offset / offset_declined on local_ccd_info (gcol33/tulpa#321). The score
puts the linear term in its own design columns, so a cell whose outer
log-marginal is a perfectly good quadratic that simply is not centred on the
cell fits exactly and scores near zero however steep the gradient across it:
passing it certifies that the design can represent the cell, not that the
cell's coordinate is a representative point of it. offset is the
standardized displacement of the cell's own peak from the cell's coordinate,
in units of the marginal spread the whitening used, and nothing gates on it --
a gradient across the cell is a cross-cell estimator question, orthogonal to
the local shape skew_max reads.The mechanism is the cell's own non-quadraticity, and it is measurable from the
design rather than inferred. A central composite design identifies a full
quadratic exactly, so the least-squares residual of the nodes' measured
log-marginals against intercept + gradient + Hessian in the whitened offset is
the part of the cell the design cannot represent; the nodes are evaluated
whatever the residual says, so the score costs no inner solve.
.joint_local_ccd_misfit() reports it as a standardized cubic magnitude on the
same convention the inner-Laplace gamma_3 uses, and on the Gaussian target it
is identically zero in all 48 configurations while on every skewed family it
exceeds 0.08.
The threshold is .NL_DIAG$gamma3_ok (0.5), one number for the inner band and
this gate because both are a standardized third-order departure from the
Gaussian the approximation was placed from. Where it belongs was measured: on
an eight-family ladder (the Gaussian target plus Gaussian copulas with
Gamma(1), (2), (4), (8), (16), (32) and (64) marginals, 48 configurations each
bar 32 for Gamma(1)), 0.5 is the only threshold on the ladder 0.01 to 2 that
improves or ties every family. Per family, gated against refining
unconditionally: 7.3118 / 7.3118 on the Gaussian, then 36.3139 / 37.2185,
40.8376 / 42.8578, 31.9596 / 34.3650, 34.1830 / 36.8361, 44.4462 / 47.2740,
61.8421 / 63.1718 and 83.2492 / 83.2492 down the ladder; pooled 340.1435
against 352.2843, with 365.0831 for never refining. Lower thresholds score
better pooled (0.175 gives 337.4142) by regressing on the two least skewed
families. 399 of 1196 candidate cells are declined across the ladder, none of
them on the Gaussian target.
On the four-axis two-block fixture the gate is measurably neutral, which is
what it has to be: over the 20 distinct refined configurations the summed
absolute endpoint error against the converged m = 13 reference is 2.11091
gated against 2.12847 unconditional and 3.12915 unrefined, and the largest
single-configuration movement is 0.00696 against that reference's own 0.01716
endpoint noise floor. Its per-cell scores there run 0.053 to 3.822, and the
design-dominated m = 3 configuration reads 0.126, so it keeps its cloud.
control$local_ccd$skew_max overrides the threshold; $local_ccd_info gains
misfit, skew_max, cells_declined, misfit_declined and
n_cells_declined, and a refinement whose every candidate declined leaves a
plain tensor grid that reports theta_interval_read = "density".
integration = "grid", so .nl_node_support() read it as a
homogeneous density grid and nothing downstream could tell. The support is now
named "mixed", the fit carries theta_interval_read and
theta_interval_design_mass, and diagnostic_summary() surfaces the pair.The reported numbers are unchanged, on the measurement. Three replacement
reads were scored against the converged m = 13 tensor reference (28561
cells) of the four-axis two-block fixture, whose own noise floor is 0.01716 on
the endpoints and 0.03853 on the widths: splitting the read into a mass CDF
plus a per-cell moment-matched Gaussian, collapsing each refined cell's design
block to one atom at its own weighted mean, and the gcol33/tulpa#308 moment
read. Summed absolute endpoint error over seven base grids (m = 3 ... 9,
design_mass 0.930 down to 0.092) is 0.63446 for the shipped weighted
quantile, 0.73159 for the collapse, 0.74464 for the split and 1.20402 for the
moment read, against 1.00289 for not refining at all; over fourteen further
configurations reached by varying local_ccd$max_cells on the same fixture,
1.65810 / 1.86035 / 1.89038 / 2.82062. The collapse is ahead at the single
design_mass = 0.930 grid (0.12022 against 0.16409) and behind at every lower
one, which is the pattern a fix has to avoid. On analytic outer targets whose
axis quantiles are closed form the same ordering holds: at design_mass >= 0.5
on an equicorrelated Gaussian (19 configurations) 3.18024 for the quantile
against 15.72792 for the collapse and 17.80607 for not refining. The moment
read wins on that target because a Gaussian moment match is its exact family
there; on a Gaussian copula with Gamma marginals, correlated the same way, it
is the worst of the four (39.93958 against 26.24674 over 27 configurations,
both measured with the refinement engaged unconditionally; under the
gcol33/tulpa#318 gate above the quantile scores 24.59641 there).
tulpa_re_cov_nested() reports the median and 95% interval of every derived
covariance quantity (sigma_i, rho_ij, Sigma_ij, in every block) from the
moments its integration design reproduces, instead of from a discrete weighted
quantile over the design's node positions (gcol33/tulpa#308). A central
composite design is a moment rule: its nodes sit where they reproduce the
integrand's first two moments and carry no probability mass of their own, so
the cumulative design weight across them is not a CDF. The discrete quantile
clamped an out-of-support probability to the extreme node, which made every
reported interval exactly the design's own extent -- at k covariance
coordinates, theta_hat +/- 1.1 sqrt(k) posterior SDs, whose coverage is
capped at 2 Phi(1.1 sqrt(k)) - 1 = 0.729 / 0.880 / 0.943 / 0.972 for
k = 1 ... 4 no matter how much data the model is given. The interval is now
moment-matched on each quantity's own coordinate (log for a scale or a
variance, atanh for a correlation, the identity for a covariance) and mapped
back, so scale intervals stay positive and asymmetric and correlation
intervals stay inside (-1, 1). Measured sigma_1 coverage over 200 seeds
per arm (poisson, 60 groups x 40 observations, nominal 0.95), varying only the
random-effect block: k = 1 0.735 -> 0.950, k = 2 0.880 -> 0.945, k = 3
0.950 -> 0.945 (binomial SE 0.015 - 0.031). The mean and sd columns are
unchanged, tulpa_re_cov_gibbs() was never affected, and the tensor-grid
layout (control$integration = "grid"), whose uniform cells do discretize the
density, keeps the weighted quantile.
.nl_wtd_quantile() takes an explicit outside policy for a probability
beyond the support's own cumulative range. The default "clamp" is the
cumulative-mass convention a sample uses and is unchanged; "na" withholds
the number where the support is a quadrature design rather than posterior
mass, so the clamp is a stated choice instead of a silent one.
The nested random-effect-covariance recovery gate
(test-re-cov-recovery.R) is raised from 75% to the 85% its Gibbs sibling is
held to. The 75% was the defect above, measured and accepted rather than
diagnosed.
Every remaining consumer that reads a median and interval off a CCD-integrated
outer grid now takes them from the moments the design reproduces, the same way
tulpa_re_cov_nested() does since gcol33/tulpa#308. The three are the joint
multi-block per-axis hyperparameter summary (theta_median / theta_ci_lo /
theta_ci_hi, gcol33/tulpa#309), the inline spatial() bar field's per-block
sigma / rho and its MCAR Sigma summary (gcol33/tulpa#310), and
spatial_range() / temporal_corr() (gcol33/tulpa#312). Confirmed before
changing anything: on a real three-block joint CCD fit the reported upper
endpoint equalled the node maximum on every axis at 0.000e+00, and on a real
MCAR fit four of the six derived quantities had BOTH endpoints exactly on the
node extent.
Measured coverage of the reported interval, gaussian response, 40 groups per
factor and 1200 observations, varying ONLY the number of crossed random-effect
blocks (which is the outer dimension d), 200 seeds per arm, nominal 0.95:
d = 3 0.9350 -> 0.9450, d = 4 0.9900 -> 0.9700, d = 5 0.9650 -> 0.9300
(binomial SE 0.007 - 0.018). Before, the mean interval width grew with the
design -- 0.3131 / 0.3560 / 0.3899 across the three arms, in a data regime that
did not change -- because the endpoints were the design's 1.1 sqrt(d)
whitened-SD reach rather than a property of the posterior; after, it is
0.3241 / 0.3251 / 0.3261, flat to 0.6%.
On the MCAR bar field, whose simulated Sigma gives an explicit truth (8 x 8
lattice, 20 observations per cell, 150 seeds, nominal 0.95), every derived
quantity's coverage improves: sigma_1 0.9067 -> 0.9133, sigma_2
0.7467 -> 0.8333, rho_12 0.7533 -> 0.8400, Sigma_12 0.8667 -> 0.9267,
pooled 0.8211 -> 0.8767 over 900 trials. It does not reach nominal; the
residual is the inner Laplace's attenuation of a spatial covariance at 64
units plus the outer Gaussian grid's fit to a skewed log-Cholesky posterior,
which this does not address.
Each nested-Laplace hyperparameter axis names the DOMAIN its interval is
formed on, read from the same per-axis registry the outer Pareto-k
unconstrains with (.joint_axis_domains()): a positive scale on log, the
BYM2 mixing weight on logit, an unconstrained coordinate (a copy alpha, an
MCAR log-Cholesky entry) on the identity. A proper-CAR rho_car, whose support
is the adjacency's eigenvalue interval, has no domain the engine will guess, so
its interval is withheld rather than reported as the design's extent.
ranef() on a tulpa_re_cov_nested() fit reports the SAMPLED values for a
random-effect coordinate the subspace debias selected, instead of the Gaussian
mixture the fit had stopped using (gcol33/tulpa#314). The node mixture is
reused rather than redrawn, so the group effects and the fixed effects
marginalize one weighted node set, and a source column says per row which
construction produced it ("sampled", "mixture" or "mode"). A fit whose
S contains no random effect is untouched, consumes no random number, and
reports what it reported before.
The inner-k-hat identity test strips the $skew_correction record alongside
$inner_* (gcol33/tulpa#313). Every entry of that record is derived from
gamma_3 or from the switch itself, so it necessarily differs with
control$diagnose_skew on and off; the invariant the test asserts -- that the
diagnostic consumes no randomness and changes no non-diagnostic field -- is
unchanged.
Every joint outer grid reports weight_kind, one entry per cell, saying
whether that cell carries the mass of its own cell or an in-cell design weight
(gcol33/tulpa#311). A fit integrated by one rule reports one value throughout;
a locally CCD-refined grid is the one support carrying both, and it now says so
per cell instead of leaving a consumer to read one kind off integration.
local_ccd_info gains n_design_nodes and design_mass, the share of the
integration weight sitting on design-weighted nodes.
The per-axis median and interval keep the weighted quantile on a refined grid,
which is a measurement rather than an omission. Scored against outer targets
whose exact axis quantiles are known in closed form, the refined grid's
quantile beats the unrefined grid's own in 8 of 8 configurations on a
diagonal-Gaussian target (d = 4/5, 5/7 levels per axis, two peak
sharpnesses), so declining it would withhold a number strictly better than the
one the same summary reports one refinement earlier. On a target the moment
rule cannot fit by construction (each axis carrying a Gamma(k, 1) marginal on
a log-tagged axis, 18 configurations over d = 4/5, 5/7/9 levels,
k = 1/2/8), the moment read beats the quantile as often on an unrefined grid
(12 of 18) as on a refined one (11 of 18), and refining improves the moment
read in only 2 of 18 -- so that advantage belongs to coarse grids and
near-lognormal targets, not to the mixed weights, and moving a refined fit onto
it is a separate question about density supports. The mixing is bounded by the
refinement itself: each node cloud is clamped to its cell's Voronoi half-box,
so a refined cell's mass is redistributed only inside the cell the unrefined
grid had collapsed onto one point.
1 / sqrt(-d2_j) is the
spread along axis j with every other axis held at the cell; the per-axis
summary reports marginal spreads, and on a correlated outer posterior -- a
sigma-alpha copy ridge is one -- the two differ by sqrt(H_jj (H^-1)_jj). The
stencil now also differences the cell's CORNER grid neighbours, which a cell
interior on every axis always has and which the tensor base already evaluated,
so sqrt(diag((-H)^-1)) costs no extra inner solve. A cell whose corners are
missing (a grid a previous pass spliced nodes into) or whose local -H is not
positive definite keeps the conditional scale.Only the scale changes. The design stays axis-aligned, the per-axis shrink to
the Voronoi half-box and the node clamp are untouched, and so is the
weight-conservation argument. That is deliberate: the summary reads a weighted
quantile over the refined grid and on design weights a cumulative sum is not a
CDF, so what it returns is close to the design's own per-axis EXTENT. An
axis-aligned design puts an axial node at f_0 sd_j on coordinate j; a
design rotated by the Cholesky factor of the same covariance puts it at
f_0 L[j, k], and measured on an equicorrelated target with unit marginal SDs
at rho = 0.8 the per-axis extents are 2.200 / 1.760 / 1.765 / 1.918 times the
SD -- so a rotated design reports an interval that depends on the arbitrary
order of the axes. Rotating was built and measured and is not what shipped.
Scored against an equicorrelated Gaussian outer target on identity axes, whose
axis marginals are standard normal whatever the correlation is, at four axes
and three levels per axis (design_mass above 0.99), reported 95% width as a
fraction of the exact 3.91993:
rho 0.0 0.5 0.7 0.8 0.9
before 1.0052 0.8587 0.6748 0.5543 0.3942
after 1.0052 0.9740 0.9540 0.9527 0.9569
Mean absolute endpoint error at rho = 0.8 goes 0.58240 -> 0.06182. The target
is quadratic, so the stencil is exact there: the recovered marginal SDs match
sqrt(diag(Sigma)) to 1.1e-15 and the conditional ones 1 / sqrt(diag(Q))
exactly.
On the package's own 4-axis multi-block fixture, refit on axis ranges that
bracket its posterior and scored against a converged m = 13 tensor grid
(28561 cells, 583 s, refinement off), mean absolute endpoint error per base
grid, with the reference's own m = 11 against m = 13 movement as the noise
floor (0.01716 on endpoints, 0.03853 on widths):
levels m 3 4 5 6 7 8 total
design_mass 0.931 0.289 0.222 0.254 0.169 0.121
before 0.1797 0.1533 0.1219 0.0616 0.0537 0.0469 0.6170
after 0.1641 0.1518 0.1219 0.0616 0.0537 0.0469 0.6000
The improvement is concentrated where #316 is: at three levels per axis the
reported mean width goes 0.67781 -> 0.71103 against a reference 1.09089, and
since both sit below the reference that direction does not depend on the
reference's exact value. It is modest there because that fixture's outer
correlations are weak (the refined cell's local correlations are -0.460 to
+0.158) and because at four of the six base grids the estimated -H is
indefinite, so the conditional scale is kept. Four other candidates were built
and measured against the same reference -- removing the shrink and clamp and
deleting absorbed cells, calibrating the scale to the cell's own mass share,
collapsing each cloud back to a mass atom, and rotating by the Cholesky factor
-- and none of them beats this across the sweep.
Also measured: the shrink to the Voronoi box, which #316 names as the
mechanism, does not bind on that fixture at three levels per axis. The
conditional scale there is (0.2120, 0.1034, 0.1954, 0.0388) against a
half / node_reach of (0.3197, 0.1818, 0.3197, 0.1420), and a candidate that
removes the shrink and the clamp entirely returns bit-identical numbers.
local_ccd_info gains cell_share, the share of the base grid's integration
weight each refined cell held before any node was placed. It is a different
number from design_mass, which is the share the refined region holds after:
the replacement nodes sit nearer the peak than the cell's own coordinate did,
so refining raises it. Reading both separates how concentrated the base grid
already was from how much the refinement concentrated it.
A joint multi-block fit records which outer integrator the caller ASKED for
and why the CCD did not run (gcol33/tulpa#315). $integration names the
integrator that ran, and .nl_node_support() keys the interval construction
off it, so "grid" could not distinguish a tensor grid the caller chose from
one a declined CCD fell back to; the reason existed and was thrown away
outside a verbose message. $integration_requested carries the request and
$integration_declined the reason -- NA_character_ when nothing was
declined, otherwise "axis_count", "unguessable_axis", "degenerate_axis",
"modefind_ridge", "modefind_boundary", "modefind_degenerate",
"modefind_failed", "hessian_singular" or "hessian_not_pd". The
cell-count warning drops its "set control$integration = \"ccd\"" advice on a
fit that already asked for one and was turned down, and names the decline
instead.
control$subspace_debias is now
accepted by tulpa_nested_laplace() -- both the single-block kernels
(icar / bym2 / car_proper / rw1 / rw2 / ar1) and the multi-block driver -- and
by tulpa_nested_laplace_joint() on both its single-block and multi-block
paths, with the same settings and the same meaning as on
tulpa_re_cov_nested().The selector costs nothing new here. control$diagnose_skew (on by default)
already re-dispatches the kernel at the fitted MAP cell and attaches both
inner scores, so .subspace_bands() reads the per-index gamma_3 and inner
Pareto-k-hat off the fit instead of paying for a probe of its own -- the one
thing it had to learn is to prefer the stored per-index k-hat over re-fitting
the raw importance curve, which a nested fit does not retain. The correction
itself re-runs the settled grid once with the sampler on, because the
corrected shape is a property of each cell and which cell is the MAP is only
known after the first pass. A corrected fit then reports $draws -- each
cell's Metropolis sample for the selected coordinates, the rest from the
Gaussian conditional given them, mixed by the grid weights through the same
.re_cov_nested_beta_draws() the RE-covariance backend uses -- instead of the
Gaussian-mixture moments, and every coefficient-facing method reads them
through the accessor it already used.
Measured against the exact conditional posterior, computed by two-dimensional quadrature outside the engine. Both fixtures put the latent block where no observation reaches it, so the conditional posterior factorises into the fixed-effect target and an independent Gaussian and the quadrature is exact for the reported coefficients.
Grid backend, rare-event binomial logit (n = 120): total absolute interval-endpoint error 0.5229 -> 0.1883 (mean of 5 seeds, sd 0.0672), a 64.0% cut; the reported centre moves -2.526 -> -2.63 against an exact posterior mean of -2.635. S is the intercept, selected on an inner k-hat of 0.705 with gamma_3 = -0.375 against an exact skewness of -0.388.
Joint multi-block, the #300 coupled occupancy fixture: 0.6189 -> 0.163 / 0.341 / 0.313 over three seeds, the centre -0.240 -> about -0.15 against an exact -0.1437 and the scale 0.389 -> about 0.447 against an exact 0.479.
Coverage, with the exact posterior's OWN coverage as the reference rather than the nominal level -- the correction targets that posterior, so reproducing its coverage is the success condition and matching nominal is not. Grid backend, 400 seeds, per coefficient:
| level | coef | exact | plain | corrected | |-------|-------|----------------|----------------|----------------| | 0.95 | beta0 | 0.9050 (.0147) | 0.9750 (.0078) | 0.9150 (.0139) | | 0.95 | beta1 | 0.9300 (.0128) | 0.9550 (.0104) | 0.9225 (.0134) | | 0.80 | beta0 | 0.7600 (.0214) | 0.8225 (.0191) | 0.7575 (.0214) | | 0.80 | beta1 | 0.7950 (.0202) | 0.8375 (.0184) | 0.7800 (.0207) |
The corrected rate is within 0.52 SE of the exact posterior's on all four; the plain Laplace sits 2.4 to 4.8 SE above it. Its apparently better agreement with the nominal level is an over-wide, mis-shaped Gaussian, not accuracy. Joint multi-block, 200 seeds with S pinned to both coefficients, same reference: 0.9500 / 0.9600 / 0.7900 / 0.7400 exact against 0.9598 / 0.9598 / 0.7940 / 0.7387 corrected and 0.9749 / 0.9548 / 0.8191 / 0.7739 plain (one seed's fit reported an NA bound, so n = 199 for the fits and 200 for the reference).
Cost, over those sweeps: 0.0324 s (SE 0.0005) -> 0.1765 s (SE 0.0011) on the grid backend at mean |S| = 1.060, and 0.0662 s (SE 0.0012) -> 0.4019 s (SE 0.0045) on the joint one at |S| = 2, i.e. 5.45x and 6.07x. Roughly half of that is the second grid pass and the rest the sweeps themselves.
An EMPTY selection leaves every backend bit-for-bit identical to the plain
path, asserted per backend (log-marginal, weights, modes, the per-cell
fixed-effect pieces, summary() and vcov()) in the new
tests/testthat/test-subspace-debias-backends.R.
The band selector under-flags on the coupled fixture, measured. On the
#300 coupled occupancy fit both coefficients band good on both inner scores
-- gamma_3 reads 0.256 and -0.126, the inner k-hat 0.378 and 0.329 -- while
the exact skewness of the occupancy coefficient is 1.198, which is
unreliable. gamma_3 recovers 0.21 of it there, below the 0.564 to 0.943
range #304 measured on the separable fixtures, and the inner k-hat over 256
one-dimensional draws does not separate that target from a Gaussian either. So
on that fit the selector takes nothing and control$subspace_debias$idx is
what pins the set; the joint numbers above are the pinned run. The ok band
floor is unchanged -- #304 measured it, and one fixture where a lower-bound
estimator undershoots harder than usual is a fact about the estimator, not a
reason to move a threshold that was itself set on measurement.
Threading. The sampler draws from R's RNG, so a debiased outer grid is
integrated serially whatever n_threads_outer asked for, on both the
single-arm driver and the sparse joint one. The cheap warm-start screen never
runs the correction, and a joint cell whose inner solve took the s2z rank-1 or
the PSD eigen-clamp path carries no usable factor to build the surface from
and is left uncorrected -- the same two paths diagnose_skew declines on, for
the same reason.
One request, one unwrap, one assignment point. The debias request travels
through every nested kernel entry as ONE R list (idx plus the sweep budget)
rather than four parallel arguments; unwrap_debias() / DebiasRequest
(src/laplace_spec_fit.h) turn it into the solver's options at each entry,
and run_subspace_debias() (src/subspace_debias.h) guards, runs and records
the outcome for all three Newton loops -- the single-arm spec loop and both
joint loops -- so the "empty index set is a no-op" contract and the result
mapping are written once. cpp_laplace_fit_multi_re()'s four #304 arguments
are folded into the same list; pre-release, no shim.
control$subspace_debias is left refused at n_quad > 1 on
tulpa_re_cov_nested(), which is the honest answer rather than a gap. The
adaptive Gauss-Hermite inner marginal integrates each group's random effects
out, so at the fitted point there is no conditional latent field: no joint
precision to take Sigma e_i from, hence no Gaussian-conditional-mean surface
and nothing for the sampler to move along. What could be corrected there is
the outer optimum's own Laplace approximation, which is a different
construction on a different density and would not share this machinery.
Reporting the SAMPLED values per group for a random effect the closure pulled
into S is split out as gcol33/tulpa#314: the draws exist per node but are not
recorded on the fit, so ranef() still reports the Gaussian mixture there.
Fixed: tests/testthat/test-inner-pareto-k.R asserted that a fit is
bit-for-bit identical with control$diagnose_skew on and off after stripping
inner_* and timing, which stopped holding when #302 added the
gamma_3-derived $skew_correction record two commits later. The record is
diagnostic-derived, so it is stripped alongside them (gcol33/tulpa#313).
$grid_modes / $grid_hessians on a joint
fit (#305) read the block off the cell precision, so the joint kernels ran with
store_Q internally and the whole outer grid's precision was alive at once
between the kernel call and the extraction. Both joint Newton loops now take
the request -- the leading block size plus the field sum-to-zero groups, both
fixed by the latent layout before the first solve -- and return the block on the
LaplaceResult re_cov contract, which the grid driver emits per cell. The
dense loop builds one cell's CSC, extracts, and releases it; the sparse loop
reads the builder's own CSC, so there the block costs no precision copy at all.
store_Q is once again the caller's own knob, passed straight through.Measured on an ICAR chain fixture (n_fixed = 8, 40-cell grid, R heap
high-water over three paired runs): peak 49.9 vs 51.5 MB at n_x = 408,
53.6 vs 58.2 MB at 2008, and 62.1 vs 71.9 MB at 6008 -- the saving tracking the
grid's precision (1.63 / 4.07 / 8.54 MB, i.e. 41.7 / 104.2 / 218.7 KB per cell)
and growing with the field where the retained block does not (868 bytes per
cell at every size). Fit time is unchanged: 0.27 / 0.73 / 1.73 s against
0.31 / 0.72 / 1.75 s.
The blocks are byte-identical to what cpp_joint_inner_vcov_blocks() returns
for the same cells -- same bytes in, same routine -- so summary(),
confint() and vcov() report exactly what they did in 0.0.142, and draws,
modes, weights and log_marginal are untouched.
Local-CCD refinement no longer costs a joint fit its intervals
(gcol33/tulpa#307). The node solves carry their own fixed-effect block through
the splice alongside the inner modes, so a refined grid reports instead of
recording grid_fixed_declined = "local_ccd_refined". Refinement itself is
unchanged.
One extraction algebra behind all of it: src/inv_block_extract.h holds the
conditioning-by-kriging constraint correction and the diagonal-block
extraction, both templated on a solve oracle. laplace_newton.h's
inv_block_layout path drives it against the live Newton factor;
extract_inner_vcov_block_cell() drives it against a factorized cell with the
constraint; the joint loops go through the latter.
summary(), confint() and vcov() on a tulpa_nested_laplace_joint() fit
reported the point estimates and NA for every standard error and both
bounds, on both the single-block and the multi-block path. The grid
marginalizer .nested_fixed_moments() reads one representation --
$grid_modes and $grid_hessians, the per-cell fixed-effect mode and
marginal precision -- and the joint driver stored neither, so a joint fit
could not put an interval on any coefficient.Both joint drivers now fill that same pair through one shared helper,
.joint_attach_grid_fixed(), so the two tiers reach the one marginalizer
rather than growing a second one. Both joint layouts stack every arm's
coefficients as a contiguous prefix of the latent vector, so the extraction
is arm-aware by construction: it takes the whole 1:n_fixed block in one
pass and reports each arm's coefficients under its own name. The per-cell
block comes from cpp_joint_inner_vcov_blocks(), the joint tier's existing
per-cell inner-covariance extraction, so the reported covariance is the
field-constrained one the fit's own tulpa_posterior_draws() mixture is
generated from. The reported covariance is the law of total variance over the
outer grid and so carries both the within-cell curvature and the between-cell
hyperparameter spread.
Measured against references outside the engine. On the #300 coupled fixture,
whose log posterior is written independently in R, the reported covariance
matches the inverse numerical Hessian of that density at the mode to 1.4e-09
relative. A one-arm joint fit and the single-block fit of the same model --
the same data, block and grid -- now agree to 5e-08 on every coefficient and
standard error, where the joint side previously produced NA; that
equivalence is asserted for poisson, binomial and gaussian. Over a
multi-cell ICAR grid the mixture matches the independent R implementation of
the same law-of-total-covariance (.joint_mixture_moments()) to 8e-17, and
200000 draws of the fit's own posterior mixture reproduce its standard errors
to 0.1%. CI coverage is judged by the existing recovery sweep with the joint
fitter substituted rather than by a second harness.
Retention is control$keep_grid_hessians (default TRUE), and costs
O(n_fixed^2) per cell: 860 bytes per cell at n_fixed = 8, unchanged as
the field grows from n_x = 408 to n_x = 6008, with no measurable fit-time
overhead (-0.1% over three paired runs). Reading the block needs the cell
precision, which the kernels now keep during the fit and drop again unless
control$store_Q asked for it; that transient is the existing store_Q
peak, ~64 bytes per latent per cell, and gcol33/tulpa#307 tracks removing it
by extracting the block inside the joint Newton loop.
Draws, modes, weights, log_marginal and the hyperparameter moments are
bit-for-bit identical with the retention on and off -- this adds reporting,
not inference. Where the retention cannot be trusted it declines with a
reason on $grid_fixed_declined instead of going quiet: "not_requested"
when switched off, and "local_ccd_refined" when local-CCD refinement
rewrote the outer grid after the cells were stored (local-CCD keeps
precedence, so no existing fit changes).
.nested_fixed_moments() skips a grid cell that carries no integration
weight. A pruned cell with no retained block previously turned the whole
marginalized covariance into NA.tulpa_re_cov_nested -> tulpa_re_cov_gibbs, or a grid
refinement), with nothing saying which directions needed exact treatment, so
every coordinate paid the sampler's price including the ones the Gaussian
already fits. The per-index inner diagnostics are already a map of exactly
that. control$subspace_debias (default FALSE) bands every probed index,
takes S = the misfit set, and corrects only x_S by Metropolis with
x_{-S} carried at its Gaussian conditional.The sampled surface is x(u) = mode + V Sigma_SS^{-1} L u with
V = Sigma E_S -- the q-dimensional generalization of the one-dimensional
conditional-mean curve both inner diagnostics already walk, reusing
inner_probe_column() rather than a second solve. The Gaussian restricted to
it is exactly N(0, I) in u, so the walk is spherical and the Laplace
shaping lives in the coordinates. Random-walk Metropolis, not NUTS: each
evaluation is one call of the Newton loop's own penalized objective (O(N), no
factorization, no derivative), so a gradient sampler would buy nothing and
would need a derivative the loop does not expose along the surface.
Selection is at the ok band, one step below the unreliable band the
reporting layer flags on, because gamma_3 is a LOWER bound on the true
skewness (0.564-0.943 of the exact value across the engine's own fixtures), so
selecting at the reported boundary would leave genuinely misfit coordinates
uncorrected. The inner importance k-hat (#303), which needs no derivative and
does not undershoot the same way, is folded in as the worse of the two -- on
the rare-event sweep below it is what bands 225 of 400 intercepts
unreliable where the mean gamma_3 is only -0.597.
MEASURED against an exact reference (Bernoulli random intercept at fixed RE
SD, the group intercepts integrated out by Gauss-Hermite and p(beta | y)
marginalized on a grid): the exact intercept marginal is
mean -4.0050, 95% (-6.5082, -2.2898); the Laplace Gaussian gives
mean -3.4025, (-5.3153, -1.4897), total endpoint error 1.9930; the
correction on S = {intercept} gives mean -3.7675, (-6.1446, -2.0611),
endpoint error 0.5923 -- a 70.3% reduction. Residual bias remains, as expected
from a lower-bound skewness estimate and a Gaussian conditional.
MEASURED against correcting EVERY coordinate, which is the question of whether
a subspace is enough (200 seeds x 2 coefficients, Bernoulli random intercept,
60 groups of 3, at the true RE SD, against the exact quadrature marginal,
nominal 0.95): plain Laplace 0.9050 (se 0.0147), subspace debias 0.9275
(0.0130) at a mean |S| of 0.945 coordinates, full-S debias over all 62
latent coordinates 0.9225 (0.0134). Correcting about one coordinate recovers
what correcting all 62 recovers, and costs 0.313 s against 0.461 s.
MEASURED against the full Gibbs debias (400 seeds, rare-event binomial-logit
with a random intercept, pooled over both coefficients): at nominal 0.95,
plain Laplace 0.9738 (se 0.0057), subspace 0.8662 (0.0120), full Gibbs 0.8888
(0.0111) -- subspace within 1.4 standard errors of the full debias; at nominal
0.80, plain 0.8738, subspace 0.7175 (0.0159), full Gibbs 0.7037 (0.0161),
within 0.6 standard errors. Cost 0.468 s against the full debias's 1.287 s,
2.75x cheaper. On the small-group binary RE fixture every probed
coordinate bands good (max |gamma_3| 0.236 over all 122 latent
coordinates), S is empty, and the fit is the plain one.
A second whole-fit sweep on a denser fixture -- Bernoulli random intercept, 60
groups of 3 (n = 180), beta = (-2.5, 1), sigma_u = 1, 200 seeds, all three
backends at their defaults on the same data -- does NOT reproduce that match,
and the reason is worth stating rather than averaging away. Beta coverage
pooled over both coefficients at nominal 0.95: plain nested 356/400 = 0.8900
(se 0.0156), subspace 359/400 = 0.8975 (0.0152), full Gibbs 376/400 = 0.9400
(0.0119) -- a 2.2 standard-error gap, at 1.074 s against 8.123 s. The
correction is not what falls short there. On the SAME 200 seeds its own layer,
conditional coverage at the true sigma, goes 0.9050 plain -> 0.9275 subspace
against 0.9225 for correcting every one of the 62 latent coordinates, so one
coordinate recovers what all 62 do, at 0.313 s against 0.461 s. What is left
is outer: the nested path's sigma_1 interval covers 150/200 against the
Gibbs sampler's 199/200 and its intercept interval is 26% narrower, and
turning the correction on moves neither number. That is a different layer,
and the two backends are not even integrating the same hyperprior (the
conjugate Sigma | b draw cannot take the flat default); gcol33/tulpa#308
separates it.
Against exact Gauss-Hermite quadrature on that fixture (24 seeds x 2
coefficients, max grid tail mass 8.4e-14) total absolute endpoint error is
plain 24.2040, subspace 21.2363 (-12.3%), every-coordinate 7.7709 (-67.9%).
So the band-selected subspace recovers the full correction's COVERAGE while
recovering about a fifth of its endpoint accuracy: it puts the interval in the
right place without fully fixing its shape. Closing that remainder is what the
coupling closure below would do, and only by growing S to nearly the whole
coupled block.
The COUPLING CLOSURE (grow S by the precision-graph neighbours whose partial
correlation with a member exceeds a threshold) is implemented and was measured
both ways rather than assumed, which is what the issue asked for. At the
default threshold it changes nothing: against the exact marginal it moves the
endpoint error 0.5923 -> 0.5651, a difference of 0.027 against a combined seed
standard error of 0.038, and across the 400-seed sweep it fires on 163 seeds
yet leaves coverage identical on 1572 of 1600 seed-coefficient-levels. The
reason is that the partial correlations between a fixed effect and the random
effects only run about 0.09 to 0.25 on these models, so a threshold in the
usual "strong coupling" range never bites.
Lowering it far enough to bite does move the finer metric, and that is worth
stating precisely rather than glossing: on a 14-coordinate fixture the total
endpoint error against the exact marginal falls 4.43 -> 1.21 only once the
threshold reaches 0.05, at which point |S| has grown to 13.4 of 14 -- the
full debias wearing a different name rather than a subspace one. So
conditioning x_{-S} on the Gaussian does NOT reproduce the exact marginal
endpoint for endpoint; it removes about 70% of the Gaussian's endpoint error
at |S| = 1 and the rest is not cheaply recoverable by growing S.
On the arbiter the issue actually names -- interval coverage -- that residual
does not show: at 200 seeds the |S| = 1 correction and the all-62-coordinate
correction cover 0.9275 and 0.9225, indistinguishable. Coverage is the coarser
of the two metrics, and the closure is off by default because nothing measured
here asks for it. It stays available as closure = TRUE or an explicit
threshold.
S is recorded on the fit as subspace_debias (selected indices, the
per-index band table they were read from, what the closure added, and the
per-node acceptance), so the escalation is auditable rather than implicit. An
empty S is not a special case of anything: the sampler is never entered, no
random number is consumed, and the fit is bit-for-bit the plain Laplace fit --
asserted on both the solver and the front door.
src/rwmh.h). The
starting scale 2.4 / sqrt(d), the Roberts-Gelman-Gilks target acceptance,
the Robbins-Monro burn-in adaptation and the accept test were written out
inline in the covariance Gibbs sweep and would have been written out again for
the subspace debias. They are now one set of primitives both consume.
rw_accept() draws its uniform unconditionally so a sweep consumes exactly
one uniform per test whatever the ratio is, which is what keeps the migrated
Gibbs sweep's RNG stream unchanged.gamma_3 is now consumed, not only graded: the inner-Laplace marginals can
be skew-corrected (gcol33/tulpa#302). The cubic term was computed, banded
and printed, and nothing read it -- so the inner layer was nested
approximation with no debias, the position this engine is designed against,
one layer in from where that argument is usually made. summary() and
confint() on a nested-Laplace fit run with control$skew_correct = TRUE now
report Cornish-Fisher marginal quantiles at each coefficient's own gamma_3,
gated to the good / ok bands, and the Gaussian quantiles everywhere else.
$skew_correction records the per-coefficient gamma_3, band and
eligibility; a skew_applied attribute on summary() / confint() records
what was used at the requested level. Wired through tulpa_nested_laplace()
and both tulpa_nested_laplace_joint() paths. (At this release a joint fit
recorded the correction without showing it, because the joint driver retained
no per-cell fixed-effect Hessians for the grid-marginalized covariance;
gcol33/tulpa#305 supplies them in 0.0.142 and the correction applies there.)Rue, Martino & Chopin (2009) Sec 3.2.3 fit a skew normal here, under three
constraints -- mean gamma^(1), variance 1, third log-density derivative at
the mode gamma^(3). Two of those inputs exist in this engine and one does
not: gamma^(1) comes from their denominator expansion, which is diagonal
only in their augmented x_j == eta_j representation
(src/inner_laplace_skew.h carries the reason). A skew normal fitted on the
cubic term alone is therefore a different construction from theirs, and its
attainable skewness saturates at |skewness| ~ 0.995 with the shape parameter
diverging as that bound is approached -- inside the very band the correction
is gated to. The Cornish-Fisher expansion is the quantile-side inverse of the
same Edgeworth series gamma_3 is the leading term of, is linear in
gamma_3 so it does not saturate, and returns quantiles directly.
The correction is skewness-only and therefore partial, which is measured
rather than asserted. Against exact quadrature quantiles of rare-event
binomial-logit posteriors it cuts total absolute endpoint error from 2.4931 to
1.3837 (44.5%), improving both endpoints in every case. On CI coverage over a
small-group Bernoulli random-effect fixture (N = 48, 200 seeds x 2
coefficients) it is directionally right and immaterial: nominal 0.95, Gaussian
0.9650, corrected 0.9600; nominal 0.80, 0.8050 vs 0.8075; nominal 0.50, 0.4950
vs 0.5000 -- every difference inside one standard error. Two reasons the
coverage gain is smaller than the marginal gain: gamma_3 is a lower bound on
the true skewness (0.875-0.943 of it on the cases above), and a biased Laplace
mode stays biased because the location term is not computed. The correction
is therefore OFF by default (.NL_DIAG$skew_correct); the coverage
measurement does not justify defaulting it on. Draws, modes, weights and every
other field the solve produced are bit-for-bit unchanged either way -- this is
post-processing on the reported quantiles.
New: .nl_skew_marginal(), .nl_skew_by_fixed(), .nl_skew_correction_attach()
(R/laplace_diagnostics.R), src/cornish_fisher.h / .cpp,
tests/testthat/test-inner-skew-correction.R, and a paired
corrected-vs-Gaussian coverage gate in test-nested-laplace-recovery.R.
gamma_3 now scores coupled multi-predictor likelihoods instead of
declining on them (gcol33/tulpa#301). The cubic Edgeworth term assumed a
log-likelihood that is a separable sum of one-eta terms, so every unit reading
several linear predictors at once -- a zero-inflation mixture's (count, zi)
pair, a CellCouplingSpec cell's arms (tulpaObs's occu_cover) -- had no
per-eta third derivative and came back NaN for good. The expansion is
unchanged; only the contraction widens, to
sum_units sum_{a,b,c} T^{abc} u^a u^b u^c with T the unit's third
derivative in its linear predictors and u the eta response to Sigma e_i.
The separable case is the one-coordinate special case of it.T is never materialised (src/curvature3_contract.h). Partition the unit's
coordinates into K blocks and the contraction equals
sum_a d/ds [u' L''(e + s u^(a)) u] at s = 0, because moving along block
a's slice of the direction differentiates exactly that block's coordinates.
Each term is one central difference of the Hessian the likelihood already
returns for the Newton solve, so the whole tensor costs 2K extra evaluations
per unit and no storage, at any block sizes. For a CellCouplingSpec that
Hessian is the analytic CellDerivs block, so this is one finite-difference
layer on an exact quantity, not a difference of a difference.
The step is scaled PER BLOCK off that block's own eta magnitude, matching the
eta-space step the scalar working-weight fallback takes. Measured against a
five-point third derivative of the cell log-density: identical to a single
global step while the arms share an eta scale, and 1.8x more accurate once one
arm's |eta| is 67x the other's. The contraction is symmetrised over index
permutations; for this block decomposition that is algebraically the plain sum
(the three relabelings coincide), so it buys robustness at a block whose own
quotient could not be formed rather than variance reduction.
Verified against the exact posterior, not asserted: on the coupled two-arm
occupancy fixture the engine's gamma_3 reproduces the same quantity computed
independently in R -- the third derivative of the exact log posterior along
the same conditional-mean curve -- to 8e-4 relative, and the zero-inflated
Poisson to 5e-4. Held against the two-dimensional quadrature of the same
posteriors it has the right sign and undershoots, closely where the skewness
is small (0.86 and 0.93 of the exact value at |skew| ~ 0.11-0.13) and by
about half where it is moderate (0.299 of an exact 0.530). That last case is
pinned in the suite because it has a consequence: gamma_3 is a LOWER BOUND
on the skewness, and there it bands "good" where the exact value bands "ok".
A coupled Gaussian cell, whose Hessian is constant, reads exactly 0.
The scalar single-coordinate path is untouched: byte-identical across seven
fixtures on both the family-enum and the spec entry (identical(), max
absolute difference exactly 0), verified against a build of the preceding
commit.
"coupled_likelihood" is retired from the inner-skew decline vocabulary and
from .INNER_SKEW_STRUCTURAL -- coupling several processes in one likelihood
no longer describes anything permanently unscorable. What remains is
"curvature3_unavailable" (a spec that ships no way to reach a third
derivative) and "coupled_arm" (a coupled fit for which no cell tensor could
be built at all). Every decline still returns NaN; one unreadable cell takes
the whole contraction to NaN rather than silently understating the sum.
New cpp_cell_coupling_curvature3() exposes the contraction at one cell,
outside any solve, so a registered spec's tensor can be checked against a
direct numerical third derivative of its own log-density and the step policy
measured rather than asserted (tests/testthat/test-cell-curvature3.R).
gamma_3 scores the inner Gaussian by expanding the joint
log density along the Gaussian conditional-mean curve at a probed latent
index, which needs a per-observation third derivative -- so a coupled
multi-process likelihood (a ZI mixture, tulpaObs's occu_cover) declines
permanently and the fit has only the outer k-hat, which scores a different
layer. inner_pareto_k walks the SAME curve and simply evaluates the joint
density along it: the inner Gaussian is an importance proposal for the exact
conditional posterior, and the Pareto-smoothed shape of that ratio scores the
approximation directly. No likelihood derivative anywhere, so it answers
wherever a mode was found.It runs on the probed subspace, not the field. Importance sampling degrades
with dimension on its own, so a k-hat over all n_x coordinates would report
n_x rather than the approximation; one dimension per probed index keeps
every sampling problem 1-D and makes the number directly comparable to the
gamma_3 for the same index. The engine returns the draws and the joint log
density at them (src/inner_laplace_is.h); the Pareto fit is the existing
shared .nested_is_pareto_k() core, which now accepts an injected draw
matrix, so there is one importance-sampling k-hat in the package rather than
two. The conditional-curve solve v_i = Sigma e_i is extracted to
src/inner_laplace_probe.h and shared with the cubic term; neither
refactorizes.
A Pareto shape index is scale-free -- it describes the SHAPE of the
importance-weight tail and says nothing about its size. Measured on the
engine's own fixtures at 256 draws: a gaussian-family coefficient, where the
inner Laplace is EXACT and gamma_3 is exactly 0, reads k-hat 0.19 / 0.26 at
importance efficiency 1.000, and a balanced binomial intercept (N = 500,
S = 230, gamma_3 = -0.007) reads 0.640 at efficiency 0.99998. Both are noise
on a proposal that needs no correction. The k-hat is therefore banded only on
probed indices whose realized efficiency falls below
.NL_DIAG$inner_k_material_ess (0.995); the raw shape is reported either way,
and inner_pareto_k_uniform records that no index carried a correction worth
describing.
The two inner scores agree where both compute. Across a binomial-intercept
skewness ladder ((N, S) = (500, 230), (500, 60), (100, 3), (20, 2), (15, 1)),
|gamma_3| runs 0.007 to 0.897 and the importance efficiency falls
monotonically with it (0.99998, 0.9962, 0.850, 0.807, 0.634 -- Spearman 1.00);
the tail shape follows at Spearman 0.90, and the band verdicts agree rung by
rung. On the coupled fixture, where gamma_3 is NaN for every index, the arm
with the larger exact posterior skewness (0.53 vs 0.13 by direct quadrature)
is the arm with the lower efficiency (0.983 vs 0.997).
.tulpa_combined_reliability() folds the inner layer's two scores into one
band -- the worse of them where both computed, the one that did where only one
did -- so a fully coupled fit reads "reliable (both layers good)" instead of
"inner Laplace not assessed". Reported through diagnostics(),
print.laplace_diagnostics() and diagnostic_summary(); declines carry a
reason from the same closed vocabulary the outer k-hat uses.
The draws are engine-owned and deterministic rather than taken from R's
stream, so requesting the diagnostic leaves a fit bit-for-bit unchanged and
the reported k-hat does not flap with the seed. Cost is one joint-density
evaluation per draw per probed index -- no factorization -- which is why the
budget is a fixed engine constant rather than the outer diagnostic's
k_samples, whose draws each cost a full inner Laplace solve.
.tulpa_inner_k_reliability() reads its fields with [[. On a
declined fit the only field carrying the inner_pareto_k prefix is the reason
string, which $ would partial-match into the k-hat.The engine can now test its own coupled likelihood paths (gcol33/tulpa#300).
CellCouplingSpec has been virtual-dispatched per cell since the joint driver
gained a coupled branch, but every genuinely non-separable implementation lived
downstream in tulpaObs, so the cross-arm scatter, the dense-pair allocation and
the per-cell derivative contract were only ever exercised by a consumer. A
minimal coupled likelihood is now registered here as a test fixture:
test_occupancy_mixture (src/test_cell_coupling_occupancy_mixture.h), a
two-arm occupancy mixture whose cell density is
psi prod_v Bern(y_v | p_v) + (1 - psi) 1{no detection}. A cell with a
detection factorises; a cell with none puts the occupancy state and every visit
inside one logarithm, so d^2 log p_cell / d eta_occ d eta_det and the
cross-visit second derivatives are nonzero. It writes both dense cross blocks
((occ, det) and the (det, det) self block) rather than taking the rank-1
self-cross shortcut, so a third-derivative tensor has an explicit Hessian to
difference, and it declares those two through dense_cross_pairs() while
omitting the one-row occupancy self block.
cpp_cell_coupling_evaluate() exposes what a spec actually writes. The
inner Newton chains each spec's eta-space derivatives through the design and
scatters them immediately, so nothing a spec computes was visible from R. This
export drives any registered spec at one cell and returns the cell log-density,
the per-arm gradient, the per-arm negative-Hessian diagonal and every dense
cross block, with the same buffer-allocation policy the kernel applies (pairs
read from the spec's own dense_cross_pairs(), rank-1 descriptor supplied).
It is the surface a finite-difference check of a spec's analytic derivatives
runs on.
The exact-quadrature ground truth reaches the coupled case.
test-inner-skew.R held the separable scalar reference: integrate the exact
posterior on a grid and hold gamma_3 against its central moments. The same
construction is now carried to two dimensions over the coupled fixture's
intercept-only conditional posterior, with three things asserted rather than
assumed -- the two-dimensional quadrature reproduces the trusted scalar
reference on a product posterior, the R density agrees cell by cell with what
the compiled spec evaluates, and the grid is converged under widening and
refinement. The fixture's exact marginal skewness is 0.53 on the occupancy
intercept and -0.13 on the detection intercept, so a coupled cubic term
(gcol33/tulpa#301) checked against it has something to be wrong about. The
joint kernel's current behaviour on it is pinned alongside: every probed index
returns NaN with the reason "coupled_arm", never a silently-wrong 0.
New tests: tests/testthat/test-cell-coupling-occupancy-mixture.R (the
per-cell contract at tier 1 -- value against the closed form, gradient against
a difference of the value, the full cross-arm Hessian against a difference of
the gradient, the coupled/factorising branch split, the declared dense pairs,
the grad-only path; then at tier 2 an end-to-end joint fit landing on the exact
mode of the posterior it claims to solve, a spatial ICAR fit whose cross-arm
curvature is measured nonzero at its own fitted mode, and dense-versus-sparse
agreement) and four blocks in tests/testthat/test-inner-skew.R. Shared
scaffolding is in tests/testthat/helper-coupled-fixture.R.
The joint nested-Laplace grid no longer returns numbers that depend on what else the machine was doing. Two identical fits could disagree in their last bits, because two inputs to the coupled-cell scatter's partition were read from live machine state rather than from the problem:
The scatter splits its per-cell loop into C chunks and reduces them in a
fixed chunk order, which makes the reduce independent of which thread ran
each chunk. C itself, though, came from team / act, where act was a
count of the solves in flight at that instant (an atomic fetch_add). The
chunk count sets the chunk boundaries, the boundaries set the summation
order, and floating-point addition is not associative -- so C moving with
the machine's load moved the answer. It is now read from the cell index:
n_grid - k_grid bounds how many peers a cell can have and estimates the
same tail width from grid geometry alone. n_outer also replaces
omp_get_num_threads(), so an OMP dynamic team adjustment cannot move it
either.
The outer width n_outer was clamped against a live available_ram_bytes()
reading, so the same model fitted twice in one session could resolve
different widths (and therefore different partitions) depending on what the
box had allocated in between. Both memory readings are now taken once per
session. The model-dependent term is still computed per call, so a larger
model is still clamped harder; only the machine-state term is frozen.
No parallelism is given up for this. The chunks are dispatched as OpenMP tasks, so however many threads are genuinely idle still drain them -- only the partition is pinned, never the number of workers executing it. In the bulk of the grid the budget is 1 exactly as before, so those cells stay serial and allocate no partial buffers.
tulpa_nested_laplace_joint() reports n_outer, the outer width the
solve actually ran at after the memory clamp. When two fits of one model
report different widths, that is the explanation for a shift in their last
bits..NL_REGISTRY entry declares, per
dispatch path, the fields its converter indexes; the shared
.nl_check_block_fields() checks them at the four boundaries that feed the
kernels (.nl_dispatch(), .nl_block_axis_grid(),
.nl_block_spec_for_cpp(), .joint_block_spec_for_cpp(), plus the
single-block joint packer). A block naming a field wrongly -- a typo, a stale
name after a rename, a block copied from a different family -- used to reach
the C++ side as a zero-length vector, which the kernels index with no bounds
check; it now raises
prior block 'icar' is missing required field(s): spatial_idx, adj_row_ptr,
adj_col_idx, n_neighbors. A field present but empty counts as missing, since
that is the same out-of-bounds read. The per-branch presence checks that had
accumulated in the joint converter are replaced by the shared one, so the
declaration is the single source of truth; test-nl-required-fields.R walks
every registry entry on every declared path dropping one field at a time, and
lints both converters' sources so a field read unconditionally by a branch but
left undeclared fails the suite.fit_st_nested()'s auto-recenter no longer switches itself off when a grid
knob is set to the engine's own default value (gcol33/tulpa#294). The
spatiotemporal rescue guarded on the PRESENCE of any of tau_lower,
tau_upper, n_grid_spatial, n_grid_temporal, n_grid_rho, rho_lower,
rho_upper in control, so control = list(n_grid_spatial = 4L) -- 4L being
the default -- returned at the first guard and left a railed grid railed. That
is gcol33/tulpa#293 one level down: a wrapper package exposing its own
n_grid argument, defaulted to the engine's value, threads it through on
every fit and disabled the rescue for all of them. A knob is now a PIN only
when its value differs from .nl_st_default() and it carries no
[auto_grid()] mark, the same provenance question the three grid-vector
rescues ask.Pinning is also PER AXIS rather than all-or-nothing: tau_lower / tau_upper
hold the two precision axes (they build both), n_grid_spatial /
n_grid_temporal one each, and n_grid_rho / rho_lower / rho_upper the
ar1 autocorrelation axis. A pinned axis keeps exactly the nodes its knobs
built and is named in the new outer_grid_pinned_axes; the rest are recentred
as usual. Only when EVERY axis is pinned does the rescue decline outright.
auto_grid() now marks a scalar knob or a prior specification, not just a
grid vector. One front door for "this value is my default, not the user's
choice", across the three shapes that question arises in.
The auto-recenter's second-attempt PC prior is no longer suppressed by a
prior_sigma the caller merely supplied (gcol33/tulpa#297). The escalation
that exists for a runaway, near-separation mode engaged only when
prior_sigma was NULL, so a wrapper stamping a prior_sigma of its own
turned attempt 2 into a second geometry recenter while the fit still reported
outer_grid_recenter_attempts = 2 as if the full escalation had run. The
suppression is now decided by provenance -- an auto_grid()-marked spec, or
one equal by value to the engine's own PC(U = 3, alpha = 0.01), is a default
and does not hold the prior back -- and when a genuine pin does suppress it
the fit carries outer_grid_prior_declined = "prior_pinned".
The outer Pareto-k now says WHY it declined (gcol33/tulpa#295). Roughly
two dozen distinct decline paths all arrived as the single value
pareto_k = NA, and the print method admitted as much ("outer diagnostic not
run or proposal degenerate"). "You turned it off", "this family's support can
never be scored", "the outer Hessian came back non-finite" and "the weights
carry no mass" are not interchangeable, and a batch reading pareto_k across
many fits could not tell a permanent structural limitation from a live signal
about the fit. Every decline now carries a reason from a closed vocabulary in
fit$pareto_k_declined: "not_requested", "not_applicable",
"unguessable_axis" (naming the axis, e.g. car_proper's rho_car -- read the
quadrature ESS instead, permanently), "draws_too_few", "grid_too_small",
"no_varying_axis", "degenerate_proposal", and
"internal_inconsistency" (an engine bug, which diagnostic_summary() now
WARNs on). Wired through the joint single- and multi-block paths, the
registry path, the SPDE grid and CCD paths, and the shared IS cores;
surfaced by diagnostics(), print.laplace_diagnostics() and
diagnostic_summary().
The inner-Laplace gamma_3 diagnostic now says why it declined too
(gcol33/tulpa#296). gamma_3 never returns a silently-wrong 0
(gcol33/tulpa#272), but its NaN carried no reason, so a structurally
unscorable model -- a coupled multi-process likelihood such as tulpaObs's
occu_cover, which this formula can never score -- printed as
control$diagnose_skew = FALSE, attributing an impossibility to a knob the
user had most likely left at its default TRUE. The reason now travels from
the point of decline: build_spec_curvature3_fn() reports
"coupled_likelihood" / "curvature3_unavailable" through an out-parameter
rather than a second predicate that could drift from it, the per-arm oracles
travel as a JointCurvature3Oracles carrying "coupled_arm" and which arms
it applies to, and the R side adds "not_requested", "no_probe_indices",
"backend_unsupported" and "solve_failed". Reported on the fit as
inner_skew_declined / inner_skew_arms_declined -- the latter also on a
PARTIALLY scored joint fit -- and read back by the combined verdict, which
now distinguishes a layer that was not assessed from one that is unscorable
by construction (for those models the outer k-hat is the only reliability
number available, permanently).
Fixed: .nl_inner_skew_at_theta() guarded its probe with return(NULL)
written inside a tryCatch() expression (gcol33/tulpa#298). That returns from
the ENCLOSING function, so a fit hitting any of those guards had the whole
res replaced by NULL by a diagnostic that was only meant to decline. The
probe is now its own function.
Every engine default now lives in one file (R/settings.R). A default
outer hyperparameter axis used to be written where it was consumed, so the
same numbers appeared in several places at once: the field-SD axis
exp(seq(log(0.1), log(3), length.out = 5)) in five (the three single-block
joint areal backends, the multi-block copy-block axis builder, and the
bym2 / iid registry entries), the copy-coefficient axis
c(0, exp(seq(log(0.1), log(3), ...))) verbatim in two, the wide
intrinsic-precision axis in three, k_samples = 200L in five, and the
reported Pareto-k usable threshold 0.7 in seven. .NL_GRID now holds one
entry per DISTINCT default axis (keyed by what the axis measures --
field_sd, gmrf_tau, gp_lengthscale, copy_alpha, ... -- so families
that integrate the same quantity share the entry and move together), with
.NL_RECENTER, .NL_ST_GRID and .NL_DIAG alongside it for the
auto-recenter policy, the spatiotemporal driver's own grid, and the
diagnostic thresholds. Every call site reads an accessor
(.nl_grid_axis(), .nl_grid_par(), .nl_recenter(), .nl_st_default(),
.nl_diag()); no number is restated anywhere else. All 20 default axes and
every registry defaults() closure are byte-identical to what they produced
before, verified node by node.
.NL_FAMILY_AXES binds family + path -> axis, so a new family cannot be
half-registered. The registry's per-family defaults() closures and the
auto-recenter's axis-provenance check (#293) now read the SAME binding, and
a plain Cartesian default is one line
(.nl_fill_family_axes(p, "bym2")) instead of a hand-written
is.null() / expand.grid() block per family. This closes a gap the #293
fix left: provenance carried a hand-maintained list of two fields
(sigma_grid, tau_grid), so an engine default on any OTHER axis
(gp_var, phi_gp_grid, ar1_rho, mo_lengthscale, ...) coming back in
through a wrapper's prior was still read as a user pin. Every defaulted axis
is now covered. type narrows the comparison to the axis that one
path-and-family lays -- passed explicitly, never inferred from
block$type, since a joint areal block carries type = "icar" while its
sigma_grid default comes from the joint path and the icar REGISTRY entry
defaults a precision axis instead.
Two source-level tests keep the defaults from re-scattering: a geometric axis
over literal bounds outside R/settings.R fails
test-settings.R, as does a restated Pareto-k threshold or k_samples
default. test-settings.R also pins every axis to its exact nodes -- these
numbers are the engine's behaviour on any fit that does not name its own
grid, so changing one is deliberate enough to update a test.
.default_tau_grid() and .nl_default_sigma_axis() are gone; call
.nl_grid_axis("gmrf_tau") / .nl_grid_axis("field_sd"). Pre-release, so no
shim.
!is.null(prior$sigma_grid) -- "the caller named a grid, so it is an
override". A wrapper package that computes the engine's own default axis
itself (tulpaObs's occu_cover() does, because it also derives the copy arm's
amplitude axis from that vector and hands the same axis to several blocks)
writes a non-NULL sigma_grid on a fit where the USER named nothing, so every
such fit looked pinned and the recenter never ran. SIGMA_GRID = "auto" was
inert for exactly the fits it exists to rescue.Provenance is now explicit, in one predicate (.nl_axis_is_pinned(),
R/nested_laplace_auto_grid.R) that all four rescues share -- the joint
single-block, the joint multi-block copy block, the standalone registry
(icar tau_grid / bym2 sigma_grid) and fit_st_nested(). An axis
counts as a DEFAULT (recentre-able) when it is absent, when it is marked with
the new auto_grid(), or when its node set is exactly the engine's own
default axis for that field -- a grid identical to the default carries no
information a pin would add. Anything else is a pin and is never moved.
auto_grid(x) / is_auto_grid(x) (exported) let a wrapper package declare
an axis it defaulted rather than one the user chose. The marker is an
attribute, recorded and stripped once at the front door
(.nl_grid_provenance()), so no downstream consumer ever sees an attributed
numeric.control$auto_recenter = FALSE (new, on tulpa_nested_laplace(),
tulpa_nested_laplace_joint() and fit_st_nested()) is the opt-out that
integrates any grid exactly as given, the engine's default axis included.res$outer_grid_recenter_declined reports why a "fixed" placement stayed
fixed: "grid_not_collapsed", "axis_pinned", "no_usable_curvature",
"auto_recenter_disabled", "grid_knobs_overridden", "refit_failed". An
inert rescue was previously indistinguishable from one that was never
needed, which is how #293 went unnoticed through #289 -> #292..nl_axis_alias()): the same axis is
spelled sigma in a single-block grid, b<k>.sigma in a multi-block one,
and theta when a single-axis vector grid is coerced for the regime
diagnostic. The multi-block rescue matched only the prefixed spelling and
the registry rescue carried a hard-coded value_axis_name for the coerced
one; both now go through the alias set, plus the "a lone log-tagged axis IS
the family's scale axis" fallback..nl_default_sigma_axis()
(it was copy-pasted across the three single-block backends, the multi-block
copy-axis builder and the bym2 registry default).Contract change: handing the engine's own default axis in explicitly used to
hold the grid fixed; it now recentres like the defaulted axis it is
(test-nested-laplace-registry-auto-grid.R updated, and that fit's k-hat now
agrees with the defaulted-grid fit's on a collapsed grid). Use
control$auto_recenter = FALSE to hold a grid where it is.
The 0.0.130 auto-recenter now also engages under the default
diagnose_k = FALSE, and covers fit_st_nested()'s spatiotemporal grid
(#291, #292). Two gaps left open by #289/#290:
The joint driver's rescue only fired when control$diagnose_k = TRUE,
because it read pareto_k_mode_u/cov_u -- fields only the full outer
Pareto-k diagnostic populated. Production batch runs default
diagnose_k = FALSE, so a collapsed fit stayed railed regardless of the
auto sigma grid. .joint_attach_pareto_k_placement() now computes the
same (mode, covariance) via the same .joint_pareto_prepare() the full
diagnostic scores its proposal from, and runs whenever the grid has
collapsed onto an edge, independent of diagnose_k.
fit_st_nested()'s tau_spatial x tau_temporal [x rho] tensor grid was
the one nested-Laplace family #289/#290 left out, since it had no
mode-find machinery to reuse. .st_auto_grid_rescue()
(R/fit_st_nested_auto_grid.R) adds one: a box-constrained L-BFGS-B
mode-find (finite-difference gradient, no analytic one available from the
compiled kernel) over the unconstrained per-axis coordinate -- log for
the two precision axes, qlogis((rho+1)/2) for ar1's autocorrelation --
seeded at the collapsed grid's own highest-weight cell, then a refit on a
grid recentered at the mode. Same trigger as every other family
(pareto_k_regime == "collapsed_edge"), one attempt only, bounded 6 nats
past the default axis on each side, and declines whenever a
grid-construction knob (tau_lower/tau_upper/n_grid_*/rho_lower/
rho_upper) was set explicitly.sigma_grid = exp(seq(log(0.1), log(3),
length.out = 5))). A fit whose field-SD posterior mode sat above the top
node collapsed every outer weight onto that boundary node
(pareto_k_regime = "collapsed_edge"), silently -- on Michael Glaser's 78
real EVA occu_cover fits, 10 railed the 5.0 sigma ceiling.The fixed grid is now a starting axis, not a ceiling: a fit that collapses
onto a boundary re-centers via the mode-Hessian its own outer Pareto-k
diagnostic already computes (R/nested_laplace_auto_grid.R) and refits,
reusing that curvature rather than running a second optimizer. An explicit
sigma_grid / tau_grid always wins -- auto-recenter only engages when
left at its default. Wired through:
- the joint driver's single-block backends (bym2/icar/car_proper) and
multi-block copy blocks, with a second attempt composing a light default
PC(U=3, alpha=0.01) prior on sigma for a genuinely unidentified
(near-separation) mode that geometry alone cannot settle;
- the standalone .NL_REGISTRY path (icar's tau_grid, bym2's
sigma_grid), one recenter attempt, reusing the joint path's generic
axis-tagging and FD-Hessian machinery;
- fit_spde()'s explicit method = "grid" path, which now attaches the
same pareto_k_regime diagnostic (visibility only -- fit_spde()'s
default control$method is already "ccd", the mode-Hessian path, so
"grid" is a deliberate opt-in the fix respects rather than overrides).
Byte-stable when the mode already sits inside the old default axis (a
no-op branch, exercised by regression tests in
test-nested-laplace-joint-auto-grid.R and
test-nested-laplace-registry-auto-grid.R). car_proper (its rho axis is
unguessable, same limitation the existing outer-k-hat diagnostic already
has) and MCAR (log-Cholesky axis geometry, a materially different
recentering problem) are out of scope; fit_st_nested()'s
spatiotemporal grid got the diagnostic only, since it has no existing
mode-find machinery to reuse -- tracked as #291.
temporal_gp() now reaches a fitter (#287). The constructor was
exported, documented, and carried a tulpa() worked example, but tulpa()
rejected type = "gp" by name, validate_temporal_gp() had no caller, and
nothing in R/ or src/ populated TemporalGPData or set
TemporalType::GP. The C++ was not the gap -- tulpa_priors_temporal.h has
carried a complete templated temporal-GP prior, in both parameterizations,
the whole time. What was missing was the marshalling.tulpa(y ~ x, temporal = temporal_gp("t"), mode = "hmc") now fits: the spec
is validated at the front door, build_sampler_model_inputs() accepts
type = "gp" and fills TemporalGPData from the unique time instants, and
the field's two hyperparameters are sampled jointly with it. They are named
too -- log_sigma2_temporal_gp / logit_phi_temporal_gp rather than
param[3] / param[4]. The field is sampler-path only (there is no
nested-Laplace kernel laying a grid over a dense T x T Gaussian), and it
cannot yet share a fit with a spatial or latent() block; both now say so.
temporal_gp(cov =) selects a kernel (#288). cov, nu and period
were match.arg-validated, documented with their closed forms, carried on
the spec object -- and read by nothing. The live density hardcoded
exp(-dt/phi), so cov = "gaussian" and cov = "periodic" silently fit an
exponential field: a misspecified prior with no error or warning.New src/temporal_gp_kernel.h holds the covariance templated over the scalar
type, so the sampled (sigma2, phi) can be autodiff variables -- which is
why the old plain-double kernels could never have been wired here. The
exponential kernel (equivalently Matern nu = 0.5) is an Ornstein-Uhlenbeck
process, so it keeps the exact O(T) Markov recursion and its numbers are
unchanged; Matern 3/2 and 5/2, Gaussian and periodic have no
finite-dimensional state-space form and are evaluated from a dense T x T
Cholesky, in both the centered and non-centered parameterizations.
Matern is offered at nu in {0.5, 1.5, 2.5} only -- the smoothnesses with a
closed form -- and anything between them is now rejected at construction
rather than quietly run as exponential. test-temporal-gp-frontdoor.R
asserts the five kernel configurations DISAGREE (a test asserting they agree
would have passed before this), that Matern nu = 0.5 reproduces the
exponential fit to the bit, and that the periodic kernel tracks its period.
cpp_nested_laplace_* was already well
factored on the part that matters -- every entry builds its latent blocks and
hands them to a shared kernel -- but the plumbing wrapped around that call was
copied per model.The structural fingerprint is the one that punished a mistake quietly: it
keys the grid checkpoint, so a copied block that folds the wrong structure
produces a checkpoint that MATCHES across runs it should not, and a resumed
run then reuses cells computed under different inputs. New
tulpa::NlFieldIdentity (nested_laplace_checkpoint.h) names each structural
group once -- areal(), nngp(), hsgp(), temporal() -- and each entry
point chains the groups it carries:
cpp
const std::uint64_t struct_seed =
tulpa::NlFieldIdentity("st_icar")
.areal(n_spatial_units, adj_row_ptr, adj_col_idx)
.temporal(temporal_type, n_times, cyclic, temporal_idx)
.seed();
Fold order is part of the fingerprint, so the optional members (the BYM2
mixing scale, the standalone temporal field's group count) sit in the slot
they have always occupied and every seed is unchanged. New
test-nl-field-identity.R checks that bit for bit against the folds the
entry points used to write by hand, for all eleven field models, and pins
that each structural input still moves the seed.
The 1-based-to-0-based skew_idx conversion had nineteen copies across
src/; all now call the tulpa::unwrap_skew_idx that already existed for it
in laplace_spec_fit.h. The five spatiotemporal entries returned their
temporal axes through a repeated pair of lines carrying an ar1-only
conditional; that is nl_attach_temporal_grids now.
No behaviour change: the entry points shed 225 lines of plumbing, and the fingerprint values and returned lists are identical.
src/ (#284). Four headers
and a set of functions nothing called. Each was checked by grepping the whole
tree for the symbol, then by compiling all 95 translation units after the
deletion -- nothing referenced any of it, so the change cannot alter
behaviour.Deleted whole: hmc_latent_grad.h (a closed component superseded by
hmc_latent.h's apply_first_zero), hmc_tvc_autodiff.h and
log_post_car_proper_det.h (never #included anywhere), and
hmc_temporal_gp.h. That last one WAS included, by hmc_sampler.h, but
every symbol in it was unreachable: its namespace tulpa_temporal_gp is
named nowhere but its own opening and closing brace. The live temporal GP is
tulpa_priors_temporal.h, which is templated for autodiff and writes the
exponential-kernel state-space recursion and the non-centred transform
inline; the deleted header was a plain-double implementation that could not
serve the gradient modes the sampler uses. Its five temporal_cov_* kernels
were a second, untested copy of covariance math the canonical templated
tulpa_svc::compute_cov already holds, pinned by test-cov-kernel.R.
Deleted in place: gp_nngp_gradient_w_analytical (hmc_gp_gradients.h),
multiscale_gp_log_lik (hmc_gp_log_lik.h), and hmc_tvc.h's dead
hyperparameter-prior and gradient block (log_prior_tau_pc,
log_prior_rho_uniform, log_prior_rho_beta, the finite-difference
rw2_gradient, parse_tvc_structure). The live TVC priors are in
tulpa_priors_tvc.h (PC prior on log-tau via pc_prior.h, Uniform(-1, 1) on
rho) and the live gradients are the analytic ones in hmc_tvc_grad.h.
SelectedInverse::at was reported as dead and is not: implicit_diff.h:196
calls it as H_inv.at(...), which a search for the qualified name misses. It
stays.
linalg_fast.h shipped two triangular-solve pairs on
opposite conventions with names that said neither: chol_forward_solve /
chol_back_solve indexed row-major with an explicit leading dimension,
tri_solve_lower / tri_solve_upper_transpose indexed column-major with n
as the stride. The two are related by transposition, so a factor handed to
the wrong pair does not crash, does not produce NaN and trips no dimension
check -- it solves against the transpose and returns a plausible vector.
#283 was that mistake on a cuSOLVER factor, and it corrupted every NNGP fit
with 51 or more locations while staying finite and ordinary-looking.There is now one implementation of each solve, templated on
tulpa_linalg::TriLayout, and the layout is a required template argument
rather than something a call site inherits from argument order:
tri_solve_lower<TriLayout::RowMajor>(L, n, ld, b, y) and
tri_solve_lower_transpose<...>. chol_factor_lower and
nngp_moments_from_chol -- the producer and the consumer that must agree
with the solve -- carry the same parameter, so a call site states the whole
convention it is asserting. chol_log_det does not: the diagonal sits at
i * ld + i under both. The layout folds at compile time, so the emitted
arithmetic and its summation order are unchanged.
tri_solve_lower's old column-major body had no callers and is gone; the one
caller of tri_solve_upper_transpose (the dense mass matrix's momentum draw,
which reads an Eigen factor and so really is column-major) now says so. New
test-tri-solve-layout.R states the contract -- each routine reads the lower
triangle of the matrix its declared layout spells out of the buffer -- and
checks it on matched and mismatched buffers, including the cuSOLVER-shaped
one whose opposite triangle still holds the input.
batch_nngp_scatter hands the NNGP
neighbour-covariance factorizations to cuda_batched_cholesky once the batch
reaches 50 locations. cuSOLVER writes L[i][j] at offset j*k + i and leaves
the opposite triangle holding the input; chol_forward_solve /
chol_back_solve index i*k + j. The accepted "factor" was therefore the
input covariances with a Cholesky diagonal -- finite, ordinary-looking, and
wrong. Every NNGP fit with 51 or more spatial locations on a machine with
usable CUDA was affected; below 50 the CPU path ran and was always correct.
On a 150-point unit-square fixture (exponential covariance, sigma2 = 0.9,
phi_gp = 0.4) the conditional variances were off by up to 0.28 in absolute
terms and 47 of 150 nodes were pushed onto the 1e-10 variance floor, against
a true minimum of 2.5e-02. After the fix both sides of the dispatch threshold
agree with an independently computed reference to 3e-16 and nothing floors.info codes, so a non-positive-
definite neighbour set came back as a partially written factor that looked
valid. cuda_batched_cholesky now reads them, and batch_nngp_scatter
checks the returned factor for one batch element against the CPU
factorization before using the batch, falling back to the CPU for all of it
on mismatch -- one extra m^3 factorization against 50+ matrices. The
fallback now also restores the original covariances, which a partial GPU
write had been corrupting.test-nngp-prior-scatter.R straddles the 50-location dispatch threshold and
compares conditional variances against a from-scratch reference, so the CPU
and GPU paths are held to the same answer. A fixture that stays under 50
locations exercises only the path that was already right.apply_nngp_full_prior_dense and apply_nngp_full_prior_sparse
were documented as the same math in different containers, and a measured
log|H| gap of 2.9e-03 between them put that in doubt. Assembling
Lambda = (I - A)' D^-1 (I - A) independently from the (alpha, cv) the
batched scatter returns settles it: both reproduce it to ~1e-16 RELATIVE at
every neighbour-set size from 2 to 10. The gap was an ABSOLUTE difference on
matrix entries of magnitude 1e13. The dense twin is deleted -- it had no
callers once blocks_require_sparse() pinned NNGP to the sparse Newton path
-- and the survivor is held to the definition by the new
test-nngp-prior-scatter.R, which also pins the gradient to -Lambda w.nngp_moments_from_chol
floors the conditional variance at 1e-10, and 47 of 150 nodes hit that floor
on the reference fixture at nn = 8, putting 1e10 on Lambda's diagonal
and taking cond(Lambda) to numerically infinite. That conditioning, not a
defective Hessian, is what stalls a Newton solve at large nn. The floor
turned out to be a symptom rather than the cause -- see 0.0.125, where the
broken CUDA factor behind it is fixed and nothing floors on that fixture.cpp_laplace_fit_spde_precomputed -- the fixed-hyperparameter fit behind the
fractional/rational SPDE path -- on its own solver, and with it
spde_run_single_fit and laplace_newton_solve_sparse. It is now a one-cell
run of the same joint multi-block driver every other SPDE/GP entry takes:
make_spde_block_precomputed seeds the block's SpdeQBuilder from the CSC
.spde_rational_assemble() hands it, makes prep() the 0.5 log|Q|
normalizer only (the precision does not move with the cell), and leaves the
latent uncentred -- the auxiliary weights x are not the field u = Pr x,
and the proper SPDE prior already identifies the constant mode. Everything
else -- obs_indices, the H pattern, both prior scatters, log_prior -- is
shared with the FEM entries through the new spde_assemble_block. The path
now inherits gamma_3, grid-cell checkpointing and the per-cell score_max /
converged reporting instead of needing a wiring pass each time.
laplace_newton_solve_sparse and spde_run_single_fit are deleted.0.5 G (log tau_re - log 2 pi), so a fit
with an iid RE block reported a marginal that was not comparable across
sigma_re (a 3.4-nat error at G = 6, sigma_re = 0.7). The shared driver
supplies it. The mode, the fitted linear predictor and the field are unchanged
-- measured across nine rational-SPDE fixtures, eta agrees to 3e-07 and the
penalized objective to 1e-08, with two fixtures reproducing bit-for-bit; the
residual movement is confined to the auxiliary-weight directions the rational
precision leaves unidentified.The SPDE FEM assembly builds the operator order it was asked for (#280).
SpdeQBuilder::rebuild() branched if (alpha == 1) ... else <alpha 2>, so
every alpha >= 3 was assembled as alpha = 2 with no error -- a user
asking for a smoother Matern field silently got the nu = 1 operator.
init() now takes the operator order and builds the chain M_0 = C,
M_1 = G, M_j = G (C^-1 G)^(j-1); rebuild() is the binomial expansion of
Q = tau^2 K (C^-1 K)^(alpha-1) over it, so one loop covers every integer
alpha and the sparsity pattern widens with the order instead of every nu
reusing the alpha = 2 stencil. The analytic marginal-SE mirror
.spde_precision_Q() carries the same expansion. alpha = 2 is reproduced
term for term, so nu = 1 results are unchanged.
(range, sigma) -> (kappa, tau) carries nu in both coordinates (#279).
The C++ conversion had nu entering kappa but not tau, which is the
nu = 1 special case; the general d = 2 relation is
tau = 1 / (sqrt(4 pi nu) kappa^nu sigma) (Lindgren, Rue & Lindstrom 2011),
as the R side already used. At nu = 2 the old tau was ~14x too large, so
a requested sigma mapped to a different marginal SD, and the single fit and
the nested integrator disagreed with each other. The conversion is now one
function, spde_range_sigma_to_kappa_tau() in src/spde_qbuilder.h, shared
by the block factory and the implicit-differentiation entry.
spatial_spde(nu = 0) is refused at construction (#281). The Matern
parameterisation is degenerate there -- kappa is 0 and tau infinite -- so
a fit could only report an infeasible cell (log_marginal = -Inf, no Newton
iterations) with nothing saying why. .validate_spde_nu() now requires
nu > 0 and names both broken quantities.
Joint-hyper NUTS refuses an integer nu != 1. Its non-centered transform
differentiates the alpha = 2 assembly, so higher orders would have been
sampled as nu = 1; the fixed-hyper sampler and the nested-Laplace path both
assemble any integer alpha. New test-spde-nu-general.R checks the compiled
assembly against an independently built K (C^-1 K)^(alpha-1) at
alpha = 1..4, the conversion against the closed-form marginal variance, and
a nu = 2 field end to end.
cpp_laplace_fit_gp and cpp_laplace_fit_spde are now one-cell runs of
the shared machinery (#277). Both were fixed-hyperparameter spatial kernels
carrying their own Newton loop, so a feature added to the joint-multi driver
had to be wired into them separately -- which is what the gamma_3 pass
(#273) ran into. Each is now a thin wrapper: the same make_single_arm, the
same make_nngp_block / make_spde_block, the same driver at one grid cell,
reading the result back through nl_grid_cell_to_result_list(). The
equivalence is exact, asserted at tolerance = 0 in
test-laplace-spatial-gp-spde-equiv.R, and gamma_3 is now inherited rather
than wired.
The SPDE single fit no longer reports an off-mode mode. The mesh field is
sum-to-zero centred after the Newton loop. The bespoke path centred it
without moving the removed constant into the intercept, which shifts eta
away from the mode the loop found, and then re-evaluated log_marginal and
the Hessian there: a converged fit reported a fixed-effect score of ~0.47.
The driver folds the constant into the arm intercept, so eta is preserved.
Fixed effects and log_marginal move for every fixed-hyperparameter SPDE fit;
the mesh field is unchanged.
NNGP is pinned to the sparse Newton path. Its prior scatters only into
the sparse builder, and the dense route disagreed with it -- measurably at
nn = 5, and at nn = 8 a 300-iteration non-convergence returning
log_marginal = NaN against a 23-iteration convergence. blocks_require_sparse()
(latent_block.h) now reads that requirement off the blocks: a block whose
prior has only add_prior_sparse forces the sparse path, instead of each
caller remembering force_sparse. This closes the silent case where the dense
path called an absent add_prior and contributed nothing at all. NNGP is the
only block whose dispatch changes; MCAR, HSGP-MO and the latent factor are
already non-INDEXED_SINGLE and were forced sparse before.
Nested fits report per-cell solve health. log_det_Q, score_max and
converged join log_marginal / n_iter / modes on every grid-driver
result, so a grid fit can see which cells settled instead of only that the
run finished.
cpp_nested_laplace_nngp() accepts an offset, which the generic driver
already read off ParsedArm; make_spde_block() takes its axes as
(kappa, tau) directly via direct_kappa_tau, so a fit handed the operator
parameters does not round-trip them through the Matern conversion.
fit_spde() with nu = 0 now reports an infeasible cell (log_marginal =
-Inf, no iterations) rather than iterating on a degenerate precision. The
Matern parameterisation has no (kappa, tau) at nu = 0
(kappa = sqrt(8 nu) / range is 0 and tau is infinite); this is the verdict
cpp_nested_laplace_spde() has always returned there.
pareto_k no longer over-flags collapsed-grid fits (#276). On a
sharp hyperparameter posterior the outer grid collapses onto ~1 cell, the
existing grid-mixture rescue cannot engage (its few bumps cover worse than
the Gaussian), and the k-hat is left scored against a SYMMETRIC Gaussian
proposal on a right-skewed variance-component marginal. A symmetric proposal
against a skewed target has a heavy importance-ratio tail whatever the
integration's quality, so the k tracked the grid collapse and the marginal's
asymmetry rather than the fit. Surfaced by a collaborator's 78-species
occu_cover run in which 42/78 species binned as "problematic/unreliable" on
a bare k threshold while their point estimates were sound.Three changes, all on the joint nested-Laplace backend (single- and multi-block paths alike):
A skew-normal proposal rescue. After the Gaussian / grid-mixture
dispatch, a k-hat still above the good band is re-scored against a product
of univariate skew-normals in the chosen Gaussian's whitened coordinate,
matched to the target's PSIS-weighted mean, sd and skewness -- estimated
from the pass's own draws, so it costs no extra inner solves and is
automatically located and scaled where the target is. Adopted only if it
strictly lowers the k-hat. Because a skew-normal has GAUSSIAN tails on both
sides it can absorb asymmetry but never a heavy tail, so the rescue cannot
launder a real tail problem: test-outer-skew-rescue.R asserts both
directions, including a skewed HEAVY-tailed target on which the rescue is
built, scored and rejected.
Engagement is screened for significance, not just magnitude. A sample
skewness has standard error sqrt(6/n) (~0.17 at 200 draws), so a bare
magnitude floor fires on noise: measured on a GAUSSIAN outer target, an
unscreened floor adopted the skew proposal in 18% of RNG states, a two-SE
screen in 5%, and the shipped three-SE screen in 0%. The screen reads the
importance weights' effective sample size, not the raw draw count.
A grid-regime classifier. pareto_k_regime reports "spread" /
"collapsed_interior" / "collapsed_edge", with
pareto_k_grid_edge_axes / pareto_k_grid_edge_sides naming the axes a
collapsed mode sits against and on which side. Below two effective grid
cells no axis carries resolved spread, so the outer integration has
degenerated to a point evaluation and pareto_k is scoring a mode-Gaussian
stand-in for the hyperparameter marginal rather than an integration. An
interior collapse is benign; a boundary collapse is actionable (the grid may
be too narrow). Axes the grid pins to a single value are excluded -- pinned,
not at a boundary. Read off stored weights, so it is attached even with
control$diagnose_k = FALSE.
The context is surfaced, so a downstream bare-k threshold is not the
whole story: diagnostics() gains outer_regime, grid_edge_axes,
grid_edge_sides and outer_skew_max attributes plus outer_regime /
outer_skew_max summary columns, print.laplace_diagnostics() prints the
marginal's skewness and a one-line reading of a collapsed regime, and
diagnostic_summary() raises a WARN with the widen-this-axis
recommendation on an edge collapse.
The pareto_k band itself is left to speak for the number it reports: the
fix is the number, not a verdict override. sn_match() remains the single
source of truth for the cumulant inversion; the proposal path adds only a
vectorized sampler and log-density (sn_cdf() / sn_quantile() route through
Owen's T by quadrature per point, which is right for a few reported quantiles
and unusable for hundreds of proposal draws).
gamma_3 wired through the SPDE / GP bespoke Newton pair (#273 item
3). cpp_laplace_fit_gp, cpp_laplace_fit_spde and
cpp_laplace_fit_spde_precomputed are standalone, fixed-hyperparameter
single fits (laplace_mode_gp() / spde_run_single_fit()) that route
through their own Newton implementation rather than the joint-multi driver
#272/#273 item 1 already wired -- the nested "nngp" / "spde" registry
entries integrate hyperparameters via the shared joint-multi machinery
instead and were unaffected. Both the dense branch (laplace_newton_solve
/ run_spde_laplace) and the fully sparse CHOLMOD-only branch
(laplace_newton_solve_sparse, n_x >= SPARSE_THRESHOLD) now accept
compute_skew / skew_idx, matching the icar/bym2/car_proper/hsgp
kernels' existing surface. #273 item 2 (the coupled non-separable
cubic-term derivation) remains open.gamma_3 wired through the joint multi-block dispatch (#273). The
inner-Laplace skewness diagnostic #272 shipped for every single-arm kernel
and the joint driver's single-block backends, but explicitly left the
MULTI-block joint path (nested_laplace_joint_multi.R, used when a fit
carries a per-group RE / trend field / arm-specific field block) unwired.
.nlj_multi_inner_skew_at_theta() closes that gap: the multi-block
counterpart of .nlj_inner_skew_at_theta(), re-dispatching the SAME
call_kernel at the fitted MAP grid cell with compute_skew = TRUE (the
C++ kernel already accepted the parameter; only the R-side threading was
missing). Same defaults as the single-block path: every arm's
fixed-effects coefficients scored by default, NA (not a silently-wrong
0) for a non-separable coupled arm.
Combined outer/inner reliability verdict no longer conflates "not
assessed" with "good" (#274). .tulpa_combined_reliability() collapsed
an unassessed inner layer (gamma_3 not computable for a given backend or
likelihood) into the same verdict string as a genuinely good inner layer
whenever the outer layer was flagged, so a batch consumer reading the
reliability string off many fits couldn't tell "outer bad, inner
genuinely fine" from "outer bad, inner never checked". Every combination
naming an unassessed layer now says so explicitly ("... not assessed"),
symmetric in both layers (an unassessed OUTER layer -- e.g. a multi-block,
multi-axis grid that declines Pareto-k rather than apply a guessed support
transform -- gets the same honest treatment).
Fixed a crash in the joint multi-block dispatch on a malformed per-arm
index vector (#275, found while testing the #273 fix). A block's
per-arm spatial_idx / temporal_idx / obs_idx entry shorter than that
arm's actual observation count (in particular, an empty vector for an arm
the block is meant to "skip") was read out of bounds by the C++ kernel's
per-arm index closure, crashing the R session instead of raising an error.
There is no supported "this block excludes arm k" shorthand via a
short/empty index vector -- every arm needs a matching-length index vector
for every block; a block's contribution to an arm is excluded via that
arm's field_coef = 0 instead. .multi_block_per_arm_idx() now validates
every per-arm entry's length across all 7 call sites (icar/bym2/car_proper,
mcar, rw1/rw2/ar1, iid, miid, tgmrf, lf) and raises a clear error naming
the block, the arm, and the expected/actual counts.
pareto_k scores the OUTER hyperparameter-grid
integration around a fixed inner Laplace; it read as a whole-fit verdict
even though the inner Gaussian approximation to the latent-field
conditional posterior is a separate, unscored layer -- an occu_cover
batch flagged 42/78 species "unreliable" on outer k-hat alone when their
point estimates, governed by the healthy inner layer, were fine. gamma_3
(src/inner_laplace_skew.h, the leading-order Edgeworth skewness estimate
from Rue, Martino & Chopin 2009 Sec 3.2.3's cubic correction, generalized
from their augmented representation to tulpa's general eta = compute_eta(x)
and to the joint multi-arm case) closes that gap: opt-in
(control$diagnose_skew, default TRUE) and computed with one extra
deterministic Newton solve at the fitted MAP grid cell, scoring every arm's
fixed-effects coefficients by default (control$skew_idx extends it).
Declines to NA -- never a silently-wrong 0 ("perfectly Gaussian") -- for
a likelihood the formula cannot score (a coupled multi-process spec such as
zero-inflation or tulpaObs's occu_cover, or a family with no registered
third derivative); this also caught and fixed a real bug in the diagnostic
as first staged, where an entirely absent oracle silently summed to
0 / sigma_i^3 == 0 instead of NA. diagnostics() /
print.laplace_diagnostics() report a combined whole-fit verdict naming
which layer degrades, if either does. Wired through every single-arm
nested-Laplace kernel (icar/bym2/car_proper/temporal/nngp/hsgp/the ST
variants/SPDE) and the joint driver's single-block backends; validated in
tests/testthat/test-inner-skew.R against a direct numerically-integrated
exact posterior skewness (a rare-event binomial intercept), not just shape
checks. Known remaining scope (joint multi-block wiring, a genuinely
coupled-arm cubic-term derivation, the SPDE/GP bespoke large-n Newton
pair) tracked in #273.test-tulpa-entry-nested.R's
"more than one random-intercept term alongside a block errors" test
asserted a restriction gcol33/tulpa#265 (0.0.113-era, commit cd80b95)
deliberately removed: every (1 | g) term on the nested-Laplace + latent()
path now becomes its own iid block integrated on the outer grid, so N
random-intercept terms beside a block are no different in kind from one --
the same change test-smoother-re-integrated.R already covers beside a
smoother. #265's commit updated R/tulpa.R but never touched this test, so
it kept asserting the old stop() and started failing with a NULL
condition once the guard it expected was gone. Bisected past the #267
auto-mode change #271 suspected as the cause: auto_select_mode() checks
has_latent before has_re, so a model combining RE terms with a
latent() block already routes to nested_laplace, not re_cov_gibbs,
regardless of #267. The test now asserts the current, intended behavior
(routing, re_block_index, and bit-exact equivalence to the direct
multi-block tulpa_nested_laplace() call).HessianPatternGuard
(introduced after the tulpaObs v0.0.101 pin, so this had never been
checked) caught occu_cover() dropping 10592-124160 nonzero contributions
per fit whenever a coupled ICAR field met either a correlated random-slope
RE block private to the detection arm, or its own detection-arm beta under
the rank-1 s2z fold path (fields above TULPA_S2Z_DENSIFY_MAX, default
256 units).The per-cell cross-Hessian scatter multiplies EVERY active dof of one
coupled arm's row -- beta, RE, and any latent-block dof that row's idx /
obs_indices resolves to -- against every active dof of another (or the
same) coupled arm's row sharing the cell. build_joint_hessian_pattern's
cross-arm section only ever registered the beta/RE part of that product; a
block reached by just one side (a private random-slope block, or a field
the other arm's field_coef = 0 decouples) had no pattern entry for its
cross term against the other arm's beta/RE/latent dofs, even though the
scatter produces a real nonzero value there whenever the coupling spec's
cross-Hessian for that arm pair is nonzero. A new section walks each
coupled cell's per-arm active-dof union (mirroring the scatter's own
collect_coupled_row_latents resolver) and adds the missing cross
entries, scoped per cell so it costs no more than the scatter already
does. Two regression tests reproduce both shapes on tulpa's own bivariate
test-coupling spec (test-cell-coupling-cross-hess.R), independent of
tulpaObs.
mode = "auto" no longer conditions a random-effect term's SD at 1 (#267).
tulpa(y ~ x + (1 | g)) on the default mode reported sd = 1, source =
"conditioned", while the same term with a slope added ((1 + x | g)) had
its whole covariance inferred -- the richer model was handled better than the
plainer one. auto_select_mode() took no random-effect argument at all, so a
mixed model fell through to the same Tier-1 default a plain GLM reaches.auto now routes any random-effect term -- intercept-only or slope -- to the
exact Metropolis-within-Gibbs covariance debias (re_cov_gibbs), which
already treats a scalar (1 | g) as the degenerate c = 1 covariance block,
so no special-casing by term shape was needed. An explicit mode = "laplace"
/ "mala" / ... still conditions on sigma_re (defaulting to 1) when the
caller names it directly -- only the auto default changes. The message()
that reported the conditioning is now a warning(), so a script that
promotes warnings sees it (same sub-issue as #265).
icar, rw1,
rw2, ar1's tau, iid) flat in log(theta), by construction of the grid
and its softmax weighting; tulpa_re_cov_nested() / tulpa_re_cov_gibbs()'s
Sigma estimate carried a PC + LKJ prior by default. The two paths put
different priors on the same statistical object, undocumented in either
direction, so the RE SD from one backend was not the RE SD from the other.tulpa_eb() and tulpa_re_cov_nested() now share a hyperprior argument,
"flat" (default) or "pc_lkj": "flat" matches the nested-Laplace
convention everywhere else in the engine; "pc_lkj" opts into the
weakly-informative PC + LKJ prior re_cov_pc_lkj_prior() builds, still
available on request. tulpa_re_cov_gibbs()'s Sigma | b conjugate draw
cannot go fully flat -- an improper prior is not a valid target for that
step -- so it keeps its existing minimal-proper Inverse-Wishart default
(prior_df = n_coefs + 1), documented as the closest analogue. See
vignette("priors").
tulpa(y ~ s(x) + (1 | site)) reported
sd = 1, source = "conditioned" -- on a formula whose smoother hyperparameter
was integrated, the random effect was the one variance component the fit never
estimated, and 1 is the value nobody supplied. There was no argument
combination that estimated it: mode = "eb" was overridden by the smoother
redirect (#266) and the RE-covariance integrators are reached only when a term
carries slopes.Each (1 | g) now becomes an iid latent block, so its SD is one more axis of
the outer grid beside the smoother's tau. The block type, its integration and
its recovery already existed (the #86 coupled field + RE capability); only the
front door was not using it, and .tulpa_fitter_args()'s own comment already
named this as the intended treatment. Consequences:
sigma_re supplied explicitly still conditions, as the one-point sigma_grid
the iid registry entry documents -- conditioning is the degenerate case of
the same path, not a second one -- and VarCorr() still labels it
conditioned rather than claiming the data produced it.VarCorr() reports the integrated posterior's median, not its mean: a
variance component at few groups is right-skewed, so its mean sits above its
bulk by construction.ranef() reports every group. The RE blocks are appended LAST, which is what
makes their latent segment addressable as the trailing sum(n_groups) columns
without re-deriving any other block's width -- a second source of truth for
something the driver already knows. Its exact-tail-width guard cannot fire
once the RE shares the latent vector with a field block, so without this the
accessor would have returned the empty table #264 just removed.iid block has no Z design) and pointing at the backends that do fit a
slope covariance.Recovery is asserted as coverage, not as a point tolerance. The grid
integrates the SD under a prior flat in log(sigma), the convention every
nested scale axis uses; that does not shrink, so at G = 15 the posterior is wide
and its point summary runs above the truth (mean of medians 1.02 against 0.9
over 6 seeds) while the 95% interval covers the truth 6/6. Asserting
|est - truth| would encode the +0.12 as the target. Whether the engine should
carry one hyperprior convention across the nested and RE-covariance paths is
#268, deliberately not bundled here: a PC prior on the iid axis alone would
trade the cross-path inconsistency for one inside a single fit.
An explicit mode is no longer silently overridden by a structural redirect
(#266). tulpa(y ~ s(x) + (1 | site), mode = "eb") fitted nested_laplace
and reported selection_reason as though mode had been "auto", so nothing
on the fit recorded that the requested inference method had been swapped. The
redirect machinery now carries that:
select_inference_mode() marks a selection explicit (anything but
mode = "auto") and keeps the literal request, so a later redirect can name
what it overrode.
.sel_redirect() records the override in sel$overridden and appends a
clause to the reason. The clause is re-appended to whatever reason the
selection ends up carrying rather than only the one that recorded it: each
redirect replaces sel$reason, so in a chain (slopes then a smoother) the
later one used to drop the statement and leave the fit looking as though
nothing had been overridden. Only the first override is recorded, since the
request the user actually made is the one worth naming.tulpa() warns, and stamps the machine-readable fit$mode_overridden
(requested + the backend it would have used). A warning() rather than a
message(), so a script that promotes warnings, or a chunk that traps them,
sees it.The warning is opt-in per redirect site (notify =), because two different
things were being conflated. A smoother sending an eb / agq request to the
nested kernels takes away the random-effect SD those two would have estimated,
and warns. A random slope under mode = "laplace" has no scalar sigma_re to
condition on, a temporal field has no conditional-Laplace kernel, and an SPDE
field redirected to the spde backend is the same mode and tier reaching its
own integrator -- those are documented routes for the structure, not
capabilities taken away, so they are recorded on the fit without warning on
every fit.
src/hmc_spatiotemporal.h removed (#261). Unreachable: its only
include was hmc_sampler.h, no external call site referenced
tulpa_spatiotemporal::, everything in it lived inside that namespace, and it
was in src/ rather than inst/include/tulpa/ so no LinkingTo consumer
could reach it either. It had also diverged from the live path
(src/tulpa_priors_st.h) in two places that were fixed only on the live side:
it overstated the cyclic RW1/RW2 rank by one, and hardcoded rank_space = S - 1
against the #241 component fix. Its st_sum_to_zero_penalty duplicated the
live one. A reader grepping for the ST rank found the wrong copy first, which
is the shape that made gcol33/tulpaRatio#12 possible.ranef() reports the per-group posterior on both RE-covariance backends
(#264). It returned a 0-row data frame for a fit from either integrator --
on exactly the fits whose free covariance over correlated slopes is the point,
and indistinguishable from a model carrying no random effects at all. Both
backends do hold the per-group information, and both now report it:
tulpa_re_cov_gibbs() samples b in its Metropolis-within-Gibbs sweep and
threw the draws away at the end of each sweep. The compiled sweep now
records them (fit$re, one column per (block, group, coefficient),
row-aligned with the beta draws so a row is a joint state), and ranef()
summarizes them as the exact posterior: mean, SD and 2.5%/97.5% quantiles.
posterior_predict() picks the same draws up through .re_draws_mat(),
so its replicates carry the random-effect uncertainty instead of falling
back to a population-level linear predictor.
tulpa_re_cov_nested() has a Gaussian per-group posterior at every
integration node. Each node's conditional mean and marginal variance are
retained (fit$re_nodes / fit$re_var_nodes, the inner solve now being
asked for its covariance blocks), and ranef() reports the exact moments
and quantiles of the weighted mixture of them -- so the interval carries
both the within-node curvature and the Sigma uncertainty. The interval
inverts the mixture CDF rather than assuming normality around the mean: a
mixture over a skewed Sigma posterior is itself skewed. New
.nl_gauss_mixture_summary() is that summary (verified against Monte Carlo,
and against the mixture CDF at its own returned quantiles); it is the
continuous counterpart of the discrete .nl_wtd_quantile().The adaptive Gauss-Hermite inner marginal (control$re_cov = "aghq", or
n_quad > 1) integrates each group out by quadrature and so forms no
per-group posterior at all. It now says that, with the two modes that do
report one, rather than returning the empty frame: ranef() errors on the
stated fit$ranef_unavailable reason. ?ranef documents what each backend
reports and why.
tglmm() and tgam() are removed; tulpa() fits both model classes.
The two doors dispatched through tulpa() and returned a byte-identical fit,
so they carried no engine of their own: what they added was a signature
missing spatial / temporal, a refusal of the structures outside their
model class, and a subclass that unlocked a richer print(). The first two
are reach removal on a call that would otherwise fit the model correctly, and
the third is now driven by the fit instead (below). R/doors.R, the
.TULPA_DOORS registry, and the tulpa_glmm / tulpa_gam subclasses are
gone; the GLMM and GAM formulas are unchanged, so tulpa(y ~ x + (1 + x | g))
and tulpa(y ~ s(x)) fit exactly what the doors did. The README now shows
both.
A fit reports the structure it carries, not the function that produced
it (#262). The random-effect covariance and the smoother table were printed
only for a fit that came through a door, though the metadata behind both is
attached by tulpa() itself (fit$smooth_terms at R/tulpa.R, VarCorr()
on any tulpa_fit). print.tulpa_fit() now ends in .print_re_section() +
.print_smooth_section(), each silent when the fit has no such structure, and
plot(fit, type = "smooth") draws the fitted curves for any fit carrying
s(...) terms. print.tulpa_nested_laplace() composes the generic fit body
after its own hyperparameter report, so a smoother fit reports its fixed
effects too -- that tier's print method had replaced the generic one rather
than extending it, so those were missing entirely.
VarCorr() reported sd = 1, conditioned for a fit that integrated Sigma
(#263). .varcorr_from_sigma() read only $Sigma, while
tulpa_re_cov_gibbs() / tulpa_re_cov_nested() report the posterior mean
under $Sigma_mean, so resolution fell through to the conditioning fallback
and invented a sigma_re the user never supplied -- on the very fits whose
free covariance is the point. Both fields are now read, with [[ rather than
$: a gibbs fit carries Sigma_draws alongside, making $Sigma an ambiguous
partial match there while it would silently resolve on a nested fit. A
conditioning fit is still labelled conditioned.
rw1_rank() and
rw2_rank() move from src/hmc_temporal.h into the exported
tulpa/sum_to_zero.h, alongside the augmented rank they feed. They already
encoded that a cycle-graph Laplacian still annihilates only the constant, so
a cyclic RW1 has rank T-1 and a cyclic RW2 T-1 as well; a consumer
computing cyclic ? T : T-1 inline overstates both by one and biases the
tau posterior. tulpa_temporal::rw1_rank / rw2_rank keep resolving, so
the engine's own call sites are unchanged.GraphPartition describes zero nodes and reports zero of them. A caller that
assigned the CSR adjacency alone therefore got an augmentation that pinned no
direction at all, while n_spatial_components kept its own default of 1 and
the rank normalizer went on crediting +0.5 log tau for the pin that was
never applied.Nothing looked wrong from R. icar_center_field still centres the field on
its way into eta, so the linear predictor stayed invariant and the field's
shape and the fixed effects came out right; only the level was loose. On
tulpaRatio's 8-unit chain the reported mean(phi_spatial) was +200.9, -342.8
and +423.1 at three consecutive seeds against a legacy level held within 0.02
of zero, and tau_spatial sat at 16.89 against 11.90 with a seed-to-seed
spread of 0.42 -- the variance component had a different posterior, not just a
different origin. A chain also spends its adaptation and treedepth on a flat
direction, which is what makes the field's convergence diagnostics
meaningless while it does.
ModelData::set_spatial_adjacency(n_units, row_ptr, col_idx, n_neighbors)
now sets the CSR arrays, the partition and the component count together; the
engine's own loader uses it at both of its sites, and compute_param_layout()
rejects a field whose partition does not describe its adjacency, alongside the
existing PC-range-anchor guard. After the fix the same three seeds report a
level of -0.003 and a tau_spatial of 12.05, inside the legacy spread.
ABI 39 -> 40. No struct layout or callable changed; the bump makes a consumer built against 39 report "rebuild required" at first NUTS use rather than tripping the new guard.
The inner Laplace solve now reaches stationarity at a large random-effect
scale (gcol33/tulpa#259, gcol33/tulpa#260). With #255 fixed, solves at a
moderate scale arrived; at a large one they still stopped short, returning a
joint score of 1e-04 with converged = TRUE while the stationary point sat
1e-05 away in relative terms. Nothing silently wrong shipped -- the settled-mode
gate in .laplace_exact_core() refuses the exact outer gradient there -- but
the fits did not arrive. They arrive now: the worst residual over the reported
configurations drops from 3.4e-04 to 4.7e-10, and each returned mode agrees with
the observed-curvature stationary point to 1e-12 relative or better.
The cause is neither a cycle nor a conditioning floor but plain local
divergence. The Newton weight is the working (expected / quasi-likelihood) one
wherever it differs from the observed curvature, so the error map near the mode
is I - Hw^-1 Ho, which contracts only while every eigenvalue of Hw^-1 Ho
is below 2. On the neg_binomial_1 fixture at phi = 6, sigma_re = 5 the
largest is 2.23: the iterate walks away from the mode geometrically, losing a
few parts in 1e9 of objective per step, and the stall test then reads the
growing step as a floor and stops. inverse_gaussian (Fisher weight
1 / (phi mu)) does the same, so this is not one family's quirk; families whose
Newton weight already is the observed curvature cannot reach it at all.
No acceptance rule reading the objective can fix that. The penalized
log-posterior is stationary at the mode, so a step gains about decrement / 2
while the objective's own accumulation noise is 8 eps |obj|; on these fixtures
the two cross at a joint score near 1e-6, which is exactly where every affected
solve stopped. Tightening the test makes it worse -- an Armijo sufficient-
decrease condition rejects genuine progress as noise and left the worst
residual at 1.4e-05, three of the reported rows unimproved and one clean
phi = 0.5 solve regressed from 4.9e-10 to 5.7e-07.
So the steering moves to the Newton decrement g' H^-1 g, which is already
computed every iteration, is the affine-invariant distance to the mode, and
keeps full relative precision exactly where objective differences are noise.
Below the near-mode gate the scale the line search opens with (newton_trust_scale)
halves whenever the decrement grew -- the previous step overshot -- and relaxes
back toward 1 by 1.5 whenever it fell. A solve whose decrement never grows below
the gate holds the scale at 1 and takes exactly the trial sequence it always
did, so every converging fit, the whole #255 fixture (81 iterations at
phi = 4, 323 at phi = 6) and the rational-SPDE case the stall path exists
for are unchanged.
Newton convergence reads the proposal, not the damped step. Both the
tolerance and the stall test now key on max|delta| rather than
max|step_scale delta|. Convergence is a property of the iterate -- H^-1 g is
small exactly when x is stationary -- while step_scale is the line search's
choice about how much of that proposal to trust, and conflating them fails in
the dangerous direction: a search that had to damp a large proposal to nothing
has not arrived, it has failed to move. With the trust factor able to open at
2^-20 this is reachable rather than hypothetical, a proposal of 1e-6 damped to
the floor otherwise clearing a 1e-12 tolerance. Solves that take the full step
are unaffected, since the two quantities coincide there.
Two probes make the mechanism testable from R rather than inferable from where
a fit landed: cpp_newton_trust_probe() replays the damping schedule over a
supplied decrement sequence, and cpp_newton_converged_probe() returns one
convergence verdict at a given accepted step scale.
dev_notes/probe_inner_stationarity.R gains curvature (the eigenvalues of
Hw^-1 Ho at the mode, i.e. how far the working weight understates the true
curvature) and bigscale (the residual table above).
newton_converged() carried a rescue path for an
ill-conditioned Hessian, where H^-1 amplifies the gradient's rounding
residual into a spurious step and max|delta| < tol can never fire. It
detected that as a Newton decrement that had stopped halving -- but the
decrement shrinks as the square of the convergence rate, so any solve
converging linearly at rate 0.707 or slower looked identical to one that had
hit its conditioning floor. neg_binomial_1 is that solve: its
quasi-likelihood Newton weight mu / (1 + phi) sits far below the observed
curvature at large phi, and the measured rate reaches 0.707 between
phi = 3 (0.57) and phi = 4 (0.70). Those fits stopped 30 iterations in at
a joint score of 7e-04 instead of 1e-11, reported converged = TRUE, and cost
the exact outer gradient five digits (relative error 1.0e-04 against a central
difference of tulpa's own log_marginal, against 1e-10 at phi <= 3).What separates the two cases is whether the step is still shrinking: a
converging solve sets a new shortest step every iteration however slowly,
while one at its conditioning floor bounces around it (on the rational-SPDE
fixture the step wanders over 3e-6 .. 4e-5 with per-iteration ratios from 0.25
to 7). The stall test now runs on that, so the affected solves are left to
finish -- 81 iterations at phi = 4, 323 at phi = 6 -- and the gradient
agrees to 1e-10 across every family and phi in
dev_notes/probe_phi_gradient_families.R. The rational SPDE case the path
exists for is unchanged (converged, n_iter 9, field correlation 0.96), and
so is every solve that was already tripping the step criterion.
Every Laplace solve reports the residual it achieved, not just whether its
stopping rule fired (gcol33/tulpa#255). LaplaceResult carries score_max,
the largest absolute component of the joint penalized score at the returned
mode, surfaced on the R side as fit$score_max. converged answers a
different question, and the difference matters because the log-marginal feels
a mode error quadratically while its theta-gradient feels it linearly. Read
off the final scatter every driver already performs, so it costs one pass over
the latent vector. The three LaplaceResult-returning exports that hand-rolled
their own result list (the SPDE Laplace pair and the implicit-diff gradient)
now go through laplace_result_to_list(), which is what makes a new diagnostic
reach all of them.
The exact outer gradient declines a mode that did not settle
(gcol33/tulpa#255). .laplace_exact_core() keeps only the log|H| path
because the joint score at x_hat is zero; a solve that stopped short leaves
behind exactly the term the derivation discards. The reported residual is
mapped through the same inverse the mode motion travels
(max|dx| <= ||H_true^-1||_inf max|score|), compared against the latent scale,
and refused above 1e-6 -- which bounds the outer gradient at ~1e-5 relative,
four orders under the 1.0e-04 above. Both sides of that threshold are measured:
settled solves swept over five families and n = 50 .. 18000 span 3.4e-16 to
2.1e-11, solves stopped short span 8.8e-05 to 1.2e-02. The refusal is signalled
as a tulpa_unsettled_mode condition carrying the residual, the implied mode
error and the iteration count, so a direct call says what to change while the
outer optimizers collapse it into one warning per fit rather than one per trial
theta.
A declined gradient no longer lets the outer optimizer report its own
starting value as the estimate (gcol33/tulpa#255). When the exact gradient is
unavailable at a trial theta, optim() receives a vector of zeros, which it
cannot distinguish from a stationary point -- so a refusal on the first step
ends the search there and phi is reported back unchanged from where it
started. The gradient-driven branch of tulpa_re_cov_nested() now discards
that run and restarts derivative-free, the same response it already had to an
outright optimizer failure, and says so in the warning.
Grid and chain checkpoint files carry a new payload field, so the format magic
is TLPACKP2. A file written by an earlier version errors with the existing
"point checkpoint$path at a fresh path" message instead of being misparsed.
The zero-inflation refusal names every family the gate admits
(gcol33/tulpa#250). The kernel guard stops on
compiled_zi_supported(family) but restated the supported set beside it as
three families, where the gate admits seven -- it omitted neg_binomial_1
and both zero-truncated bases, which are exactly the hurdle models. A caller
who reached the error for some other family was told a hurdle on a truncated
base was unavailable. The message is now built from the gate:
discrete_mass_families() is the candidate list has_discrete_mass() tests
membership in, and compiled_zi_supported_families() runs the gate over it.
A mistyped TULPA_S2Z_DENSIFY_MAX no longer forces the Woodbury path
(gcol33/tulpa#251). The override was read with atoi(), which maps every
unparseable string to 0 -- and 0 is a meaningful setting here, forcing
the rank-1 storage on every intrinsic field. =ture or =256a therefore
changed which algorithm ran for the rest of the session, with nothing in the
output saying so. It is parsed with strtol() against the end pointer now:
anything that is not a whole non-negative int is treated as unset, so the
documented default applies. Both storages remain exact, so no fitted value
moves. The other three getenv knobs in src/ are presence or single-char
tests whose unparseable case already lands on the default.
A warm start whose source supplies the wrong number of SDs is refused
(gcol33/tulpa#252). .build_warm_start() errors when a source fit's block
does not match the sampler's layout, but two branches took s_t[1] and
carried on -- seeding every coefficient of a term from the first
coefficient's SD, for both the mass scale and the log_sigma_re starting
value, and reporting that as a warm start. A single SD still broadcasts (the
scalar-term case); any length other than one or the term's coefficient count
now stops with the wording the coefficient placement already used.
get_mode_backends() derives its mode list from INFERENCE_TIERS
(gcol33/tulpa#254). Its error message restated the names its sibling
select_inference_mode() already builds from the registry. Correct today,
stale the moment a tier is added.
phi is the residual VARIANCE for the normal families
while the compiled kernels parameterize by the SD, and the conversion tested
the family name for exact equality. Every <family>_<link> spelling --
gaussian_log, gaussian_inverse, lognormal_log -- therefore reached the
kernel with the variance where the SD belongs, so a caller asking for a
residual variance of 0.8 was fitted at 0.64. The four R -> kernel boundaries
(tulpa_laplace(), the tulpa() front door, fit_spde()'s working weights,
and the SPDE predict path) now share one pair of converters resolving the base
family through .family_base(), the way every other consumer already did.It went unnoticed because nothing downstream distinguishes the two under the
canonical link: dW/deta is identically zero for gaussian_identity, so a
wrong phi cancels out of the mode motion and the curvature channel. It stops
cancelling for gaussian_log, whose observed-minus-working delta became
nonzero in 0.0.103. Changes fitted values for existing suffixed-normal fits.
tulpa_eb() and tulpa_re_cov_nested() take phi2 and
thread it through the pilot fit, both inner solves and the exact outer
gradient, which hard-coded NA_real_. family = "t" had been fitting at the
compiled default of four degrees of freedom whatever the caller meant --
reporting an ML-II phi conditional on a df nobody chose -- and
family = "tweedie" could not be fitted on those paths at all, failing as a
generic inner-solve error because its variance power had no way to arrive.
The power is now required by name, a phi2 supplied for a family that carries
none errors instead of being ignored, and tulpa(phi2 = ) forwards to
mode = "eb" and the nested Sigma integrator. Supplying phi2 = 4 for t
reproduces the previous default bit for bit, which is what pins the threading
as faithful rather than merely different. The backends carrying phi2 are
derived from the registry (.phi2_backends()) rather than restated in the
refusal message.estimate_phi covers every front-door family that has a dispersion
(gcol33/tulpa#247). It was offered for twelve and carried a derivative for
four; the other eight hit a hard refusal at fit time. lognormal, t,
neg_binomial_1, beta, inverse_gaussian, beta_binomial and tweedie
are now registered, so .dispersion_families() is exactly the set of
front-door families carrying a phi. Each new entry finite-differences
against the .FAMILY_OPS likelihood, score and weight it claims to
differentiate (~1e-9), and the assembled dm/dlog_phi matches a central
difference of the Laplace log-marginal to the same order.gaussian and lognormal now share one set of derivatives read at their own
residual -- the lognormal is the same normal density on the log scale plus a
dispersion-free Jacobian. Tweedie's log-density derivative reads the mean
event count off the SAME compound Poisson-gamma enumeration that produces the
density, so the two cannot be taken over different truncations of the series.
An exact gradient says the optimizer walks the right surface, not that the
maximizer of that surface is the generating dispersion, so each new family
additionally recovers its phi from simulated truth over several seeds
(test-eb-dispersion.R). Measured mean estimates against truth: lognormal
1.44 / 1.5, neg_binomial_1 2.11 / 2.0, beta 8.32 / 8.0,
inverse_gaussian 0.48 / 0.5, beta_binomial 8.08 / 8.0, t 1.21 / 1.2,
truncated_neg_binomial_2 2.96 / 3.0, with the random-effect SD recovered at
0.68-0.74 against 0.7 throughout. tweedie is the exception, and not for a
reason of its own: the EB path never threads phi2, so it cannot be fitted
there at all (gcol33/tulpa#257).
q_eta = (dW/deta) s. It is right when
either the working weight carries no eta (gaussian, lognormal, gamma, t) or
the mode-motion solve is on the TRUE curvature -- which needs the family's
observed curvature -l''(eta), and only six families had one. Adding the
mu-space ladder dgrad_mu_dmu / d2grad_mu_dmu2 alongside the existing
grad_mu closes that generically: -l'' = -(L'' u^2 + L' u1) covers every
family on the generic route in one branch, with explicit branches for
beta_binomial, tweedie and t. Verified against a numerical second
derivative of the log density for seventeen family/link pairs.This is what unparks beta, whose derivatives had been kept unregistered
since its assembled gradient landed ~1e-4 off the objective. It also makes
the exact mode Jacobian and the marginal fixed-effect precision available
across the registry, where before they were declined for gamma, beta,
inverse_gaussian, beta_binomial, tweedie and t.
gamma had been deferred on the grounds that
its Fisher working weight differs from its observed curvature; the operative
condition is narrower than that -- the weight is free of eta, which zeroes
every channel the two inverses could differ on -- and it holds. lognormal
and t join on the same terms.truncated_neg_binomial_2 needed the border itself fixed. It differentiates
u and the phi mode motion, both formed on the true-curvature inverse, while
reading Hinv -- correct only where the two weights coincide, which is what
the old registration gate quietly relied on. The border now carries
dH_true/dpsi, which needs the phi-derivative of the observed-minus-working
correction as well as the eta-derivative the random-effect block already had;
a family in that position supplies dobs_weight and .family_dphi2()
withholds the whole entry without it, rather than pairing two different
inverses. The second mode derivative's right-hand side reads the same
correction: the score's eta-derivative is the OBSERVED curvature, and the two
differ for exactly these families.
The truncated family's three weight derivatives are one chain rule over
f(a, a_e) and the shape's own derivatives, the same partials
curvature_deta2_for_family carries, so dweight, dweight2 and the mixed
dweight_deta come from a single derivation instead of three. The bordered
Hessian matches a central difference of the exact gradient to ~1e-9 for all
six registered families. neg_binomial_1 keeps the differencing stencil.
compiled_zi_supported() was defined as
has_observed_curvature(), which was the right answer only while the count
families were the only ones carrying one: the mixture's y = 0 branch reads
the log density at zero as log P(Y = 0), which for a continuous family is a
log-DENSITY -- finite and plausible for gaussian, infinite for gamma or beta.
It now also requires a discrete base. The R front door reads the compiled
predicate instead of keeping its own list of families, so the two cannot
drift; zero-inflated beta_binomial is admitted by that (it has both an atom
at zero and, now, an observed curvature).SparseHessianBuilder::add() dropped any contribution
whose (row, col) was absent from the registered sparsity pattern, and the
cached-slot fast paths did the same behind if (slot >= 0). Nothing counted
it. What survives a drop is a Hessian too small in one entry -- finite,
positive definite, correctly shaped -- which the Newton step, the Laplace
log-determinant, the marginal standard errors and the exact outer gradient all
inherit, so the only symptom is a recovery test drifting, the hardest signal
here to attribute. The invariant had already broken twice: unequal /
non-contiguous areal components (#241) and weighted-entry blocks (#242).src/hessian_pattern_guard.h adds the drop counter, the scatter_slot()
helper that single-sources every guarded slot write (14 sites across the
dense-basis, indexed-cache, joint-batch and sum-to-zero scatters), and
HessianPatternGuard, which snapshots the counter and raises. Detection is a
counter rather than a throw at the write because the scatter runs inside
OpenMP parallel regions, where an Rcpp::stop escaping the structured block
is std::terminate -- the constraint that also makes
LaplaceResult::start_infeasible a flag. The guard is a pure snapshot, so a
driver nested inside another measures its own window without disturbing it.
Placed in run_nested_laplace_grid (covering the single-block and both joint
paths), the batched joint driver, spde_run_single_fit and
cpp_laplace_fit_gp.
Only a NONZERO discarded contribution counts. The scatter index caches resolve whole cross products up front -- every (beta_j, RE_g) pair of an arm -- and legitimately hold -1 for pairs no observation ever touches; those slots are written with a structural zero, which changes nothing whether it lands or not. Counting at index-resolution time instead would fire on every real fit.
The shipping kernels scatter entirely inside their registered patterns: the
suite runs with the check armed and zero drops, so this lands as a regression
barrier rather than a bug fix. New test-hessian-pattern-guard.R covers the
counter on both write paths, the raise, the zero-versus-nonzero distinction,
and a real fit.
portable_math.h compiles for downstream packages on macOS again
(bugfix). The header returns std::pair from portable_digamma_lgamma()
but included only <cmath> and <limits>. libstdc++ reaches <utility>
transitively through those, so Linux and MinGW builds were fine and tulpa's
own sources were fine everywhere -- they include <utility> ahead of it. Apple
clang's libc++ does not, so any package that includes the header first failed
to compile: no template named 'pair' in namespace 'std', which took
tulpaObs's macOS R CMD check down at count_grouped_oracle.o. The header
now includes <utility> itself. Header-only change, no behaviour anywhere.Warm-starting the sampler from a cheaper fit of the same model.
New:
estimate_phi = TRUE now covers truncated_neg_binomial_2, and covers it
alongside a zero-inflation process when the base is zero-truncated (a hurdle).
The family was listed in .PHI_FAMILIES but had no entry in .FAMILY_DPHI,
so the dispersion was refused for it outright. Its three derivatives are the
untruncated ones plus the phi-derivative of the retained mass
P(Y > 0); the weight differentiated is the EXPECTED form Var(y | y > 0),
which is what the Newton solve builds H from for this family -- the opposite
choice from neg_binomial_2, where H carries the observed curvature. Each
finite-differences against the registry to ~1e-9 over phi in [0.5, 25], and
the assembled dm/dlog_phi matches a central difference of the Laplace
log-marginal to ~1e-9.Under a mixture the refusal now tests the BASE for zero-truncation rather than
testing for a mixture at all. A hurdle's zero branch is log(pi) and carries
no dispersion, so the base family's derivatives are the mixture's once masked
at y = 0; genuine zero inflation on an untruncated base keeps the refusal,
since its zero branch depends on phi through P(Y = 0) as well. Verified to
~1e-8 against the two-process log-marginal on a fixture with 35% zeros, and
phi_hat is consistent -- 2.921, 2.871, 2.998 as the count arm grows 750,
2269, 5057 (12 seeds each, standard error 0.144, 0.091, 0.065).
-Hess(log density), so its nine second partials are the FIVE distinct fourth
derivatives of one scalar, indexed by how many of the four derivatives are in
eta. New mixture_curvature_deriv2() returns those five; a hurdle is the
case where the three mixed ones vanish, so one assembly covers both and the
hurdle's terms multiply by zero rather than being special-cased.The coupled y = 0 branch differentiates D = pi + (1 - pi) P(Y = 0) a
fourth time, which is the only genuinely new input: it needs P(Y = 0)'s
fourth log-derivative and so the second eta-derivative of the observed
curvature, which has_zi_curvature_2nd_derivative() gates on. That gate is no
longer narrower than the gradient's.
The five fields are checked along every route to them: most are reachable by
differentiating two or three DIFFERENT third-order fields, in different
directions, and all routes agree to ~1e-11 -- which is what tests the
-Hess(log D) identification rather than the arithmetic. Against a central
difference of the exact gradient the assembled Hessian reaches 1.5e-9
(poisson, 53% zeros), 4.1e-9 (neg_binomial_2, 57%), 5.2e-9 (binomial, 60%),
and 3.3e-11 under a hurdle -- the last unchanged to every printed digit by the
generalization.
The closed outer Hessian also covers the families whose Newton weight is not
their observed curvature, which unblocks truncated_neg_binomial_2 on the
one-process path and hurdle_nbinom2 on the two-process one. The assembly
forms u on the observed-curvature inverse but was differentiating it through
the working-weight inverse -- the same thing only where the two weights
coincide. Pairing them needs dH_true/dtheta, and so the new
obs_curvature_delta_deta_for_family(): for a zero-truncated NB2 the
difference W_obs - w differentiates to the NB2 curvature derivative plus
d3 log P(Y > 0) minus the truncated working one, all three of which the
registry (and truncation_shape()'s d3a) already supplied. It
finite-differences against W_obs - w to ~4e-10, and the assembled Hessian
reaches 2.3e-09 one-process and 3.4e-09 under the hurdle.
estimate_phi = TRUE now runs alongside GENUINE zero inflation as well, and
the closed Hessian's dispersion border runs under any mixture. A hurdle's zero
branch is log(pi) and carries no dispersion, which is what let the base
family's registry stand in for the mixture's; genuine zero inflation has zero
branch log(pi + (1 - pi) P(Y = 0, phi)), which carries phi through
P(Y = 0) and couples it to BOTH linear predictors -- so the mode motion
solves against both scores, and the explicit dW/dphi runs over the whole
2 x 2 block.
The sixteen fields that needs are sixteen CALLS on one engine, not sixteen
formulas: the mixture density is q(z) B(eta, phi) + C(z) in every branch, so
every mixed partial of log D follows from the multivariate
cumulant-from-moment recursion over the alphabet (eta, z, phi), exact at any
order and any mix of directions. The base registry covers the rows the mixture
leaves additively separable and the engine covers the coupled y = 0 rows;
the two are summed over disjoint rows, and the engine returns exact zeros
wherever the registry applies (every y != 0 row, a hurdle's y = 0 rows, a
family with no free dispersion).
Anchored on zi_loglik / zi_score_eta / zi_neg_hessian and the curvature
engines, all written independently of this one: 40 field checks agree to
1.9e-10, with the two shared fourth-order names each reached from BOTH members
of their pair. The assembled dm/dlog_phi matches the two-process
log-marginal to 5.6e-09 and the bordered Hessian a difference of the exact
gradient to 2.1e-10, on a fixture with 57% zeros. The one-process path is not
a second assembly: it is this one with the six z-direction fields at zero, and
it expands term for term to the expression it replaces.
neg_binomial_1 is covered throughout, on the one-process closed Hessian and
under zero inflation. Its observed curvature is
-s + r^2 (psi'(r) - psi'(y+r)), so d(W_obs - w)/deta carries a tetragamma
and the next rung a pentagamma; both are now in portable_math.h alongside the
existing digamma / trigamma pair, in the same recurrence-plus-asymptotic form.
The first rung finite-differences to 2.8e-10 and agrees with the same formula
over R's own psigamma to 6.5e-10; the one-process closed Hessian reaches
3.5e-11.The same missing derivative was also what kept the family off the mixture
curvature gate, so that gate now asks for the property it needs -- the
observed curvature's eta-derivative, being the registered working-weight one
plus the correction -- rather than naming families. beta_binomial, t and
tweedie stay out, having no observed form at all. Under zero inflation
neg_binomial_1's third- and fourth-order mixture fields reach 2.2e-11 and
6.3e-11, and its closed Hessian 2.5e-11.
Still on the stencil, for a stated reason rather than by omission:
estimate_phi for neg_binomial_1 in any model, zero-inflated or not, since
the family has no .FAMILY_DPHI entry at all -- a separate gap from the one
closed here.
Weighted-entry intrinsic fields -- separable multivariate CAR (mcar) and the
areal / temporal varying-coefficient blocks -- are now hard-constrained by the
same augment-and-centre identification the uniformly-seen fields moved to in
0.0.99, instead of the old soft sum-to-zero pin (gcol33/tulpa#242). MCAR carries
the augmented precision Sigma^-1 (x) Q_aug
(Q_aug = Q + sum_c 1_c 1_c'/J_c per field, coupled across fields by
Sigma^-1 and folded by the sparse solver), so its Sigma-dependent normalizer
takes the full n log|Sigma^-1| per field. A weighted field's constant aliases
with the coefficient on the covariate it rides on, not the intercept, so the
centerer folds the field's global level into that column -- svc_beta_offset
for a single field, per-field field_beta_offset for MCAR, or -1 for a
covariate with no fixed-effect counterpart (the level is then left in the field
for the augmentation to identify). A declared column is verified against the
design so a wrong alias cannot silently shift eta.
tglmm() and tgam() are named front doors onto tulpa(): contract-narrowing
views, not new engines. Each carries tulpa()'s signature minus the arguments
its model class cannot use (spatial, temporal), requires the structure that
defines the class in the formula (tglmm() a random-effect term, tgam() an
s(...) smoother), and refuses the structures outside it with a pointer to
tulpa() rather than fitting them. Dispatch, tier selection, backends and every
tulpa_fit accessor are unchanged; the fit is byte-identical to the same call
through tulpa().
The returned fit carries "tulpa_glmm" / "tulpa_gam" ahead of "tulpa_fit",
so print() adds the section that model class is read for -- the random-effect
covariance (with VarCorr()'s estimated / sampled / conditioned label) or the
smoother table -- and plot() on a GAM draws the fitted smooths. A door's
print() composes the backend's own report rather than displacing it, so a
nested-Laplace GAM still shows its integrated hyperparameters, grid size and
outer Pareto-k.
Doors are registry-driven (.TULPA_DOORS): a new one is an entry plus a stub,
and test-doors.R asserts each signature stays exactly tulpa()'s minus its
withheld arguments, so a new statistical argument on tulpa() cannot leave a
door silently stale.
tulpa_check_control(control, allowed, where) is exported. It is the
control-list name check every front-door fitter already ran internally,
promoted so consumer packages validate their own control = list() surface
against the same rules instead of reimplementing them. tulpaRatio's
tratio() is the first caller.
tulpa(warm_start = ) seeds the NUTS sampler from a Laplace or empirical-Bayes
fit: "eb" or "laplace" fits one first, or pass an existing fit from either
mode. Chains start at that mode instead of the origin. Chains after the first
are dispersed around it, so the between-chain spread rhat() compares against
is not collapsed by a shared starting point. Only the NUTS/HMC backends take
one -- the rest error rather than sample from the default start and report a
fit that answers a different question.
It is offered as a capability, not as a speedup: over 10 seeds, redrawing both
the data and the sampler seed on a crossed random-intercept Poisson model,
warm and cold were indistinguishable in effective sample size per second
(paired Wilcoxon p = 0.32 and p = 0.49, with the source fit's own cost charged
to the warm runs). Single runs of that comparison disagree with each other by
2-3x, because min(ESS) is an order statistic over parameters. Whether it
pays on harder geometries than this one is untested.
cpp_tulpa_glmm_layout() reports the parameter layout the sampler will use --
block spans, per-term random-effect shape, and the column names -- without
sampling. The warm start places values by index against it rather than
reconstructing the sampler's naming convention, so the layout stays the single
source of truth for where a parameter lives.
tulpa_sample_glmm(warm_start = ) and cpp_tulpa_sample_glmm()'s init /
inv_metric_diag are the engine-level entry points. init is one row per
chain. The inverse-mass diagonal is assembled but not supplied by default: the
kernel reads every entry as a posterior variance, and today the random-effect
entries would be prior variances and the variance components would have no
estimate at all. return_joint_hessian and tulpa_eb(marginal = TRUE) now
supply both, so composing it properly is the follow-up.
tulpa(estimate_phi = TRUE) estimates the family's dispersion instead of
conditioning on phi, which then supplies the starting value. log(phi)
joins the empirical-Bayes maximization as one further coordinate carrying the
exact derivative of the Laplace log-marginal, so it is the ML-II estimate: the
hyperprior covers the random-effect covariances and the dispersion enters
unpenalized. fit$phi is the estimate and fit$phi_estimated distinguishes
it from a conditioned value. Available under mode = "eb" and for the
families whose dispersion derivative is registered; any other mode errors
rather than fitting at the starting value under a name that says otherwise.
This is what makes a Gaussian GLMM comparable to lme4::lmer(REML = FALSE).
With the residual variance held at its default the outer maximization absorbs
the mismatch into the random-effect scales: on a nested (1 | g1/g2) fit the
inner term came out at 0.10 against lmer's 0.31. With the dispersion free the
same fit reports 0.32, a residual SD of 0.5266 against lmer's 0.5260, and
fixed effects within 0.006 of an lmer standard error.
lme4, glmmTMB, betareg, pscl and MASS::glm.nb
(glmmTMB and pscl are new in Suggests). Every count family, the
zero-inflation and hurdle mixtures, beta regression, and the random-intercept
/ nested / correlated-slope structures are checked against a second
implementation on identical data.Conditioned on the reference's own dispersion and given a diffuse fixed-effect prior, the Laplace mode is the MLE of the same likelihood, so the tests assert agreement to a thousandth of a reference standard error rather than to a fraction of one. The dispersion half of each likelihood is checked separately by where the profile log-marginal peaks. A parameter-recovery test against a simulated truth cannot separate a likelihood bug from sampling noise at finite N; these can.
ziformula is now carried by mode = "eb" and mode = "re_cov_nested", and
tulpa_eb() / tulpa_re_cov_nested() take X_zi and zi_prior_sd directly,
so the random-effect covariance is estimated or integrated under the mixture
rather than under a model missing it. Their inner solve is tulpa_laplace(),
which already carried the second process, so the mixture changes that solve and
not the covariance coordinates the outer objective searches over. The mode is
[beta | beta_zi | random effects] and both fixed blocks are reported:
coef(), vcov() and confint() return ncol(X) + ncol(X_zi) entries, the
zero-side ones named zi_*.The exact outer gradient follows the mixture. With two linear predictors the
per-observation curvature is a 2 x 2 block over the count predictor and the
zero predictor, and both move with the mode, so d log|H| / d theta picks up
all six of its partials rather than the single dw/deta the one-process path
uses. laplace_family_zi_curvature.h supplies them in closed form, contracted
against the three linear-predictor (co)variances.
cpp_family_has_zi_curvature_derivative() gates it, and the gate is narrower
than zero inflation itself: an untruncated y = 0 branch differentiates
P(Y = 0) a third time, which needs the eta-derivative of the OBSERVED
curvature, and only the families whose working weight already is that have it
registered. A hurdle base is zero-truncated, so its y = 0 branch is flat in
the count predictor and needs no third derivative.
Refused rather than fitted as a different model: n_quad > 1 (the adaptive
Gauss-Hermite inner marginal runs through a single-predictor compiled oracle),
estimate_phi = TRUE (the registered dispersion derivative is the base
family's, while the mixture's zero branch depends on phi through P(Y = 0) as
well, so the two describe different objectives), and mode = "re_cov_gibbs"
(its conditional carries no second process).
Fixed:
spatial_gp()) fitted with exact NUTS
under-recovered the field: the centered parameterization sampled the field
jointly with its (sigma2, phi) hyperparameters -- Neal's funnel -- and a
diagonal mass matrix stalled in the neck, so the posterior-mean amplitude
collapsed toward zero and sigma2 was under-estimated regardless of warmup
(gcol33/tulpa#243). The sampling path now uses the non-centered
parameterization by default (sample z ~ N(0, I), reconstruct
w = f(z, sigma2, phi)), wiring the field's likelihood gradient back to
(z, log_sigma2, log_phi) through the hand-derived NNGP forward/backward in
an arena custom_backward; the stored draws are transformed z -> w on the
way out. spatial_gp(parameterization = "centered") restores the old path.The same transform is now available on the NNGP spatially-varying-coefficient
(spatial_svc(parameterization = "noncentered"), per term) and multi-scale
(spatial_multiscale(sampler = "noncentered"), per scale) fields, which the
three blocks reach through one shared per-term applier rather than a copy
each. The HSGP variants of all three are already spectrally non-centered and
are unchanged.
SVC keeps the CENTERED default, unlike GP. The issue that prompted this work
expected SVC to share the funnel but recorded that as untested; measured on
the test-svc-nuts-frontdoor.R recovery fit (Poisson, n = 120, same data /
seed / budget for both arms), the centered path is already funnel-free --
0/700 divergent, field correlation 0.939, sd ratio 0.662, mean treedepth 6.3
-- and it fits in 163s where non-centered needs well over ten times that for
the same answer. The funnel that motivated the GP flip was measured on a
weakly identified field, which is where non-centered earns its cost; a
well-identified response does not pay it. Reach for "noncentered" when the
field is weakly identified or the centered fit reports divergences.
Correspondingly there is no new SVC amplitude test: test-svc-nuts-frontdoor.R
already asserts the same sd-ratio band plus a divergence guard on
well-identified data, which is the correct design for that quantity -- on
weakly identified data a posterior mean shrunk toward zero is the right
answer, not evidence of a funnel, so a sd-ratio contrast there measures
shrinkage rather than geometry.
The multiscale transform is verified by finite differences against the
analytic backward on both scales, but its end-to-end amplitude test is
skipped against gcol33/tulpa#244: that block's range prior is still a Uniform
behind a hard -INFINITY wall (the defect #144 fixed for GP and SVC, which
multiscale escaped only by being unreachable), and it leaves 82-88% of
post-warmup draws divergent whichever parameterization is used.
spatial_multiscale() reaches exact NUTS through the tulpa() front door
for the first time: the prior and its parameter layout existed, but no
spatial-type branch ever built the sampler inputs for it, so the path was
unreachable. Both scales' fields are reported as gp_local[i] /
gp_regional[i] draws alongside their own (sigma2, phi). It has no
nested-Laplace kernel, so mode = "structured" still routes it to the
conditional Laplace path (which rejects it); auto routes it to exact NUTS,
the integrator that fits it.
The samplers started every coordinate at the origin, which puts a spatial
range at phi = 1. That is inside the support of every PC-range block, but
the multi-scale block declares hard range bounds, so bounds excluding 1 left
the chain starting at -Inf -- it never moved and returned an all-zero field
at a 100% divergence rate. Each bounded range now starts at the geometric
mean of its own bounds; a caller-supplied init is left untouched, and
unbounded blocks are unaffected.
A Laplace fit made with no beta_prior reported standard errors that omitted
the prior its mode was found under. The compiled kernels apply a built-in
N(0, 100^2) ridge whether or not one is supplied, but the reported
curvature added a penalty only when the argument was present, so the default
fit's vcov() described a different posterior than its point estimate.
Fixed-effect standard errors on a nested-Laplace fit carrying an RW1 / RW2
temporal field -- or any s(...) smoother, which is an RW2 over the binned
covariate -- were wrong for the intercept, which was reported at the
fixed-effect prior SD (100) regardless of the data. On a 400-observation RW1
fit the intercept SE was 99.998 against an exact-MCMC value of 0.047.
An intrinsic field's constant null direction is jointly unidentified with the intercept. The spatial (ICAR / BYM2) and sampler paths identify it by augmenting the field precision during the solve, so the level sits under the field's own tau; the temporal Laplace kernels never did. They identified the level only by centring the mode afterwards and folding the removed mean into the intercept -- which fixes the point estimate and leaves that direction flat in the Hessian the marginal covariance is read from. Point estimates, field shape, log-marginals and every other coefficient were unaffected, which is why nothing else moved: on the same fit the slope SE was already correct to 2%.
The RW1 / RW2 kernels now carry the same augmentation, over the field's global constant (the between-group level contrasts of a panel field stay improper by design, matching the sampler). The pin is registered for the solver to fold in rather than stored, so an RW chain's Hessian stays tridiagonal / pentadiagonal; both storages are exact and agree to five significant figures. The augmentation itself is now one helper shared with the ICAR kernels, whose numbers are unchanged.
tulpa_laplace()) carrying an intrinsic areal field were understated for the
intercept: 2.3x at 10 areal units, 2.7x at 40, 3.0x at 120 for ICAR, and about
1.3x for BYM2 (whose structured half carries the field). Reported intervals
were correspondingly too narrow.summary() / vcov() / confint() marginalize the field out of the joint
Hessian by reconstructing the field precision in R. That reconstruction
augmented the ICAR structure with a unit 11', where the kernel it is meant to
mirror augments with 1_c 1_c'/J_c per component -- a factor J too much
weight on the constant direction, which is exactly the direction the intercept
is aliased with, and no per-component handling on a replicated field. The
reconstruction now matches the kernel and follows the component split
for_each_icar_component walks. Slope standard errors are unchanged to five
decimals, the pin being orthogonal to covariate contrasts.
The reconstruction is now pinned against the kernel's own exported
log_prior_icar rather than against a second R statement of the convention:
the previous reference encoded the same unit 11', so it agreed with the
helper by construction and could not have caught this.
n / n_components nodes per component. This is correct for a
single connected map and for a spatial(by = ) replicate over the
block-diagonal I_L (x) Q (equal-size, contiguous copies), but wrong for a
genuine disconnected map whose components are unequal in size or not
contiguous in the node ordering (a mainland plus islands, the islands sorted
among the mainland cells -- the standard US-counties layout). The wrong node
sets shifted both the fitted field and its variance component, on the Laplace
and the exact-NUTS paths alike, and the marginal intercept SE with them.Components are now taken from the adjacency graph itself: the connected-
component partition (inst/include/tulpa/graph_components.h, one DFS computed
once per fit and carried on ModelData) drives every consumer -- the Laplace
gradient / Hessian / pattern / log-prior, the autodiff NUTS prior, the MCAR
block, and the R marginal-SE reconstruction -- so each component's constant is
pinned over its actual nodes. The rank-1 sum-to-zero fold the sparse solver
uses (Woodbury step and block-Schur log-determinant) now carries a node-index
list, so a component that is too large to densify folds exactly whether its
nodes are contiguous or not. A connected graph and an equal-size replicate are
the trivial partition and stay byte-identical. ModelData grew
spatial_partition and the ABI version is 37; consumer packages that
LinkingTo: tulpa must be rebuilt.
control = list(n_warmup = ) was silently dropped on the sampler backends.
tulpa()'s control surface is the union over the backends it dispatches, so
n_warmup (which the NUTS-SPDE driver reads) passed the front-door check, and
the subset that narrows control for the chosen fitter then discarded it
because tulpa_sample_glmm() spells the same knob warmup. Warmup silently
stayed at n_iter / 2 while appearing to honour the request. Control knobs
are now canonicalized through one alias table, so a fitter that reads the
other spelling still receives it and an explicit value always wins.An exact analytic gradient of the joint-field Laplace log-marginal with respect to the random-effect covariances, and the outer optimizer and marginal correction rebuilt on top of it.
New:
tulpa_laplace(return_joint_hessian = TRUE) returns H_joint, the full
joint posterior precision of [beta | random effects] at the mode, as a
symmetric sparse matrix. Previously only its fixed-effect Schur complement
(H_beta) was reachable, so nothing outside the kernel could differentiate
the quantity the Laplace approximation takes the determinant of.dw/deta for every compiled family (laplace_family_curvature.h): the
eta-derivative of the curvature the Newton system uses. For poisson,
binomial, neg_binomial_2 and the truncated families that is the true third
derivative of the log density; for the families whose Hessian carries an
expected or working weight it is deliberately the derivative of THAT weight,
since it is the weight the objective is built from.
cpp_family_has_curvature_derivative() gates it, mirroring
has_observed_curvature().Changed:
k == 1) or Nelder-Mead (k >= 2). Measured 1.6x fewer inner Laplace
solves for a scalar block and 2.8x for a correlated one; the gain grows with
the number of hyperparameter coordinates, since the simplex is what scales
badly. Falls back to the derivative-free path for a family with no exact
curvature derivative, for the AGHQ inner marginal (n_quad > 1, a different
objective), and if a gradient-driven run fails outright.tulpa_eb(marginal = TRUE) builds its correction from the analytic gradient:
the mode Jacobian J is now closed-form rather than finite-differenced, and
H_theta comes from differencing an exact gradient (2k solves) rather than
second-differencing the objective (1 + 2k^2 solves). Same correction, 2 to
2.8x fewer solves.Notes:
log|H|, which is wrong by 9 to 58 percent on the checked cases. It vanishes
only for a Gaussian response, where the curvature does not move with the
linear predictor. Derivation and numerical confirmation in
dev_notes/laplace_exact_gradient.md.Identification of intrinsic fields:
Q_aug = Q + sum_c 1_c 1_c' / J_c, with the field
centred on its way into the linear predictor. The constant direction then
carries the field's own precision and never reaches the likelihood -- a free
draw that integrates out -- instead of being held by a stiff penalty the
sampler still has to traverse (gcol33/tulpa#241). This is what INLA's
constr=TRUE and Stan's sum_to_zero_vector do. Applied to ICAR and BYM2 on
both the exact-NUTS and Laplace paths, and to the multiscale temporal trend
and seasonal arms. On the reference ICAR fit the correlation between the field
sum and the intercept goes from -0.996 to 0.035, the intercept SD from 0.31 to
0.026, and the field sum sits at the sqrt(J/tau) the augmented prior
predicts.Q_aug fills one direction per pinned component, so
ICAR -- whose null space is exactly its component constants -- becomes full
rank and the normalizer takes J log tau. Carrying the augmentation while
keeping the deficient rank would bias the variance component low. The rank
helper takes rank(Q) and the number of directions filled rather than the
field length, because that identity is not universal: a non-cyclic RW2 also
has a linear null direction a sum-to-zero augmentation does not touch, so it
stays deficient by one.temporal_multiscale() is reachable from tulpa(mode = "hmc"). The block's
layout, prior and gradient all existed but nothing in the package set the flag
they key on, so it could only be driven from a consumer package. Its
parameters are named (trend[i], seasonal[i], short_term[i], and the
matching scales) instead of positional. Recovery through the front door:
cor(trend, truth) = 0.97 on a 36-point RW1 trend.Fixed:
spatial(by=)
produces) was pinned once over the whole vector and normalized at rank
J - 1, while the sampler and the Polya-Gamma kernels derived the true count
from the same adjacency. The count is now derived there too.Sigma^-1 (x) 11'/J couples
fields, which independent rank-1 terms cannot represent). LatentBlock::center
reports where each removed constant belongs rather than assuming the arm
intercept, since a block reached through a per-observation weight aliases with
that covariate's coefficient instead.tulpa_laplace(return_hessian = TRUE) assembled the marginal fixed-effect
precision by hand as X'WX - X'WZ (Z'WZ + D^-1)^-1 Z'WX from
glmm_weights(). That form reaches ONE linear predictor, so it sliced the
random effects out of the mode at ncol(X) -- inside the beta_zi block when
a zero-inflation process is present -- and it carried the R-side Fisher weight
rather than the curvature the kernel found the mode under, measured 1.8% apart
on neg_binomial_2. It is now the Schur complement of the joint curvature over
the random-effect block, corrected to the observed weight for the two families
whose Newton weight is not it. The joint Hessian already carries the
fixed-effect prior and the random-effect penalty the mode was found under, so
nothing is added back.dW/dtheta channel formed its mode-motion solve on
the working-weight inverse. That channel is v_r' (dx_hat/dtheta), and the
mode follows the true stationarity condition, so it is governed by the observed
curvature -- the same inverse the mode Jacobian already used. The two coincide
wherever the working weight is the observed one, which is why a poisson or
gaussian check could not see the difference; on truncated_neg_binomial_2 the
inverses differ by ~9% and the gradient was off ~0.8%.neg_binomial_1 and
truncated_neg_binomial_2 instead of returning an inexact value. Its
du/dtheta differentiates u through the working-weight inverse while u
itself is now formed on the observed-curvature one, and forced on it misses the
exact Hessian by 2.6e-2 and 2.7e-4 respectively. Those two families take the
gradient stencil instead -- a central difference of the exact gradient, 2k
gradient evaluations rather than the 1 + 2k^2 objective re-solves of the full
fallback -- which is checked against a second difference of the objective.marginal is a formal argument of tulpa_eb() rather than one of its control
knobs, so the front door had no route to the hyperparameter-uncertainty
correction. control$marginal on tulpa() / tglmm() now forwards to it, and
passing it in control to tulpa_eb() directly errors rather than being
accepted and ignored.The soft sum-to-zero constant that identifies intrinsic latent fields, moved onto the reference idiom and single-sourced (#241).
Fixed:
sd(sum phi) = kappa * n with kappa = 0.001 (Morris et al. 2019; the
constant brms and the Stan ICAR case study use), via the new exported
tulpa/soft_sum_to_zero.h. The sampler's ICAR and BYM2 branches previously
applied a flat precision of 0.01, i.e. sd(sum phi) = 10, which leaves the
field's constant direction free and aliased with the intercept. On a 36-unit
ICAR binomial fit against a known intercept of 0.5, over 5 seeds:
cor(sum(phi), intercept) moves from -0.996 to -0.037, sd(sum(phi)) from
11.1 to 0.036 (= kappa * n, the target), and the intercept posterior SD from
0.310 to 0.027. Correlation between the posterior-mean field and truth is
0.989 either way: the penalty pins the level without shrinking the pattern.J - n_components. It now pins one direction per
connected component, matching the rank term and the Laplace path, so a
disconnected graph (spatial(by =) replicated CAR) is fully identified.S terms, a
time margin T). The SVC penalty is written on the sum rather than the mean;
the two forms are the same penalty, related by a factor of n_obs.Internal:
for_each_icar_component() moved from laplace_spatial_priors.cpp into the
shared icar_kernel.h, so both engines walk the null space identically.test-mcar-prior.R's reference now reads the same
helper the C++ does rather than hard-coding the value.A settable prior on the zero-inflation coefficients, and the correctness fix that finding it turned up.
New:
tulpa(zi_prior = list(sd =)) sets the prior SD on the zero-inflation
coefficients, beta_zi ~ N(0, sd^2); the default is 2.5. One scalar applies
to the whole block and the mean is fixed at 0, matching what the compiled
kernels carry -- a mean entry is refused rather than silently dropped, and
the list form leaves room for a per-coefficient prior later. sd = Inf
removes the penalty. tulpa_laplace() takes the same value as the scalar
zi_prior_sd.
The prior is worth setting rather than accepting: where a ZI design level
contributes no zeros the likelihood is monotone in that level's coefficient
and alone would send it to -Inf, so the prior is what makes the mode exist.
zi_prior is how that scale is now chosen.
Fixed:
The Laplace path applied the zero-inflation prior only when the caller also
supplied beta_prior. Those govern different blocks -- beta_prior is the
count block -- so the ZI coefficients silently carried a prior SD of 2.5 with
a fixed-effect prior present and the weak built-in 100 without one, while
every sampler path applied 2.5 unconditionally. The same ziformula
therefore fit a different model depending on mode, and on an unrelated
argument. The prior is now applied whenever the ZI block exists. A second
edge in the same branch: supplying only beta_prior$mean left the precision
vector empty, so the guard that resized it skipped the ZI tail.
ziformula on tulpa() and X_zi on tulpa_laplace() appeared in the usage
blocks with no \item, an undocumented-argument check warning carried since
the zero-inflation port. Both are now documented.
The hurdle-factorization test compared a joint fit's zi block against a standalone binomial without matching their priors, and absorbed the resulting gap in a 1e-4 tolerance -- the same prior discrepancy described above, hidden rather than caught. Matched, the factorization is exact to solver precision, so the two zi assertions are now pinned at 1e-9, and a companion test repeats the identity at a prior far from the default to show it is a property of the likelihood and not of the default scale.
One diagnostic front door, selected by draws provenance.
New:
diagnostics() is the entry point for posterior diagnostics on any fit. It
reads how the draws were produced and returns the reliability question that
applies: chain mixing (improved Rhat, bulk / tail / mean / sd / quantile ESS,
MCSE) for MCMC draws; the PSIS approximation-reliability table (pareto_k,
grid quadrature ESS) for i.i.d. draws from a deterministic backend; NULL
with a message for a point summary that carries no sample.
The routing is a registry keyed by provenance kind, so a new engine class is
one entry plus its table builder rather than another branch. This replaces the
hand-rolled if (!is_chain) dispatch that previously sat inside
mcmc_diagnostics().
Deprecated:
mcmc_diagnostics() and laplace_diagnostics() are deprecated in favour of
diagnostics(). Both still work and return exactly what they always did;
mcmc_diagnostics() in particular still routes an i.i.d. fit to the
reliability table, which is the behaviour that made its name wrong. The
laplace_diagnostics class and its print() method are unchanged, so code
that inspects the returned object keeps working.Fixed:
laplace_diagnostics() example fitted with mode = "laplace", which
returns a mode plus covariance and carries no draws, so the example printed a
"no posterior draws" message instead of the table it documents. Both it and
the new diagnostics() example now use mode = "smc", a deterministic
backend that does emit draws.Empirical Bayes over random-effect covariances, and the lme4 / posterior accessor surface.
New:
mode = "eb" and tulpa_eb(): estimate one or more random-effect covariances
by maximizing the Laplace marginal likelihood over them, then report the fixed
effects conditional on the maximizer. This is the plug-in counterpart of
tulpa_re_cov_nested(), and deliberately not a second implementation of it:
both call the extracted .re_cov_theta_fit(), so they share the objective, the
inner solve and the optimizer, and their theta_hat values are identical on
the same data (asserted with expect_identical, not a tolerance). EB stops at
the mode; the nested integrator carries on and marginalizes around it.
Registered as a Tier-2 backend and opt-in by name -- conditioning on
Sigma_hat drops the hyperparameter uncertainty, so auto never selects it.fixef() and the as_draws() / as_draws_array() / as_draws_matrix() /
as_draws_df() / as_draws_rvars() family on tulpa_fit. .onLoad() also
registers these methods (and the existing ranef()) on lme4::fixef,
nlme::fixef, lme4::ranef, nlme::ranef and posterior::as_draws* when
those packages are installed, so lme4::fixef(fit) dispatches without any of
them entering Imports -- and without tulpa masking their generics on attach.as_draws() on one errors by
default and names the alternative. as_draws(fit, n_draws = ) opts in to
sampling N(coef, vcov); that is a modelling decision (every downstream
posterior summary would treat the approximation as a posterior sample), so it
is never taken silently.Correctness:
$ is now exact on a tulpa_fit (new $.tulpa_fit). A fit is a list, so
$'s default partial matching let an ABSENT field resolve to any longer field
it prefixes -- and since the accessors decide which posterior shape a fit
carries by testing whether $draws / $mode / $modes / $cov is NULL, a
partial match there reads the wrong object outright. Live collisions:
$draws -> draws_kind (the string "iid", on every Laplace/EB fit),
$mode -> model_matrix (the design matrix, on every sampler fit),
$sigma -> sigma_re (AGQ fits), $theta -> theta_hat (EB fits).Three symptoms were live: posterior_sample() returned "iid" and
tulpa_draws_array() built a 1x1x1 array from it for every Laplace-shaped fit;
laplace_diagnostics(), which exists for exactly those fits, could never reach
its "no posterior draws" branch; and print() on an AGQ fit reported the
random-effect standard deviation under the sigma: label, where it reads as
the dispersion. The remaining collisions were latent -- masked by a branch
ordered ahead of them or by a companion && !is.null(...) guard -- so no
coefficient, standard error or interval changes: coef(), vcov(),
confint() and summary() return exactly what they did before on every
backend. The draws accessors additionally route through one .fit_draws()
helper.
This reaches model packages that set class = c("<model>_fit", "tulpa_fit"):
they inherit exact $ too, so any of their code that was relying on a partial
match now gets NULL. That reliance was always a bug, but it will surface here.
Cost is ~2 us per $ read (an S3 dispatch); coef() and summary() are ~0.5
ms, so it is not measurable at the accessor level.
- offset() was silently dropped by the RE-covariance backends. tulpa()
never threaded the offset into tulpa_re_cov_nested() (which had no offset
argument at all), so a rate model reached under mode = "re_cov_nested" -- or
via the automatic random-slope redirect off mode = "laplace" -- fitted counts
instead. On a simulated rate model with exposure spanning 50x and a true
intercept of -1.5, it returned +1.66. tulpa_re_cov_nested() and tulpa_eb()
now take offset and thread it through the inner tulpa_laplace() solve,
which always supported it. Where an offset genuinely cannot be carried it now
errors instead of dropping: n_quad > 1 (the compiled per-group AGHQ oracle
has no offset term) and the re_cov_gibbs backend.
- The outer optimization over the random-effect covariance(s) now warns when it
does not converge, in both tulpa_eb() and tulpa_re_cov_nested(). It
previously returned wherever the optimizer stopped without a word -- for EB
that is the estimate, and for the nested path it is the centre the integration
grid is placed around. control$outer_maxit (default 500) sets the budget, and
tulpa_eb() reports the code as $outer_convergence.
- One-dimensional outer optimization uses Brent rather than Nelder-Mead, which R
warns is unreliable there. k == 1 is the common case (a scalar (1 | g)
block), so this affects tulpa_re_cov_nested() as well as EB. Brent reports
success at a bracket endpoint, so a variance component pinned at the bracket
now warns rather than being reported as a fitted value -- the low end is the
classic empirical-Bayes collapse to sigma = 0.
Zero inflation as a composition over the count families, and compiled kernels for the last three families that had none.
New:
ziformula builds a second linear predictor and forms the mixture at the
likelihood level, so zero inflation is a composition over the base families
rather than a set of families of its own. The math lives once, in
R/family_zi.R and src/builtin_family_zi.h, mirrored term for term.
The two backend classes reach it through different hooks because they differ
structurally. The sampler paths take the logit_zi callback argument the
engine already plumbed; the Laplace spec shim hardcodes that argument to zero
and instead rides the zi predictor as process 1, so the mixture's cross term
reuses the existing row-major n_processes x n_processes curvature block
rather than introducing a second curvature contract. Coverage is derived from
BACKEND_REGISTRY, so a new sampler backend inherits ZI with its fitter, and
backends that would silently drop the mixture refuse instead.
Hurdle models fall out of this as ZI over a zero-truncated base: with no atom
at zero the mixture degenerates exactly, so they need no families of their
own. The degenerate case is reached by taking the p0 = 0 limit rather than
by evaluating a density at a point the family does not define -- the general
branch is correct in double arithmetic but yields NaN under AD. Both
directions are pinned, including the converse that an untruncated base keeps
a non-zero cross term.
Compiled kernels for neg_binomial_1, truncated_poisson and
truncated_neg_binomial_2 on both the Laplace and the AD / sampler paths.
These three were registered in R but absent from C++; .R_ONLY_FAMILIES is
now empty, so every registered family is fittable. Density, score, working
weight and observed curvature agree with the R registry to machine precision,
and each family recovers its generating parameters through the Laplace path.
The two zero-truncated families share their retained-mass term
log P(Y > 0) and its first two eta-derivatives, differing only in a and
its derivatives, so truncation_term() computes the pieces once and the
density, the weight and the observed curvature all read from it.
Two curvatures are now distinguished rather than conflated.
grad_hess_for_family() returns the Newton working weight, chosen for
positive-definiteness; obs_grad_hess_for_family() returns the observed
curvature at the realized y, mirroring .family_obs_weight() in R. The
zero-inflation mixture needs the latter, because its y = 0 branch
differentiates the density rather than taking an expectation. The distinction
is not cosmetic: neg_binomial_1's observed curvature turns negative once
y sits well above the mean, which would break Newton, while its working
weight mu / (1 + phi) never does. The mixture only ever evaluates the
observed form at y = 0, where it is positive.
Compiled ZI coverage follows from that: a family becomes ZI-fittable exactly
when its observed curvature is registered, which is now poisson,
binomial, neg_binomial_2, neg_binomial_1, truncated_poisson and
truncated_neg_binomial_2. The remaining count families keep the R-level
composition (zi_loglik() and friends) for density work and are refused at
the front door for fitting.
arena::Var and ad::Var gain expm1 (fwd::Dual already had it), and
autodiff_utils.h gains expm1_fn plus a log1m_exp_fn that splits at
log 2. Both branches differentiate, so the truncated AD densities are exact
rather than falling back.
cpp_family_obs_terms() and cpp_family_ad_terms() give the two new
surfaces test probes. The latter evaluates the AD density at fwd::Dual, so
its value checks against the independent double implementation and its
derivative against the analytic score.
Correctness:
The silent Poisson fall-through is gone. variance_fn, grad_mu and
log_lik_mu each ended in the Poisson branch, so a family known to R but not
to C++ fitted, and fitted the wrong likelihood -- which is what
.R_ONLY_FAMILIES existed to guard against. They now raise, and that is what
makes emptying the list safe.
.family_base() stripped link suffixes in registration order and so resolved
beta_binomial to beta. Every validator built on it was applying beta's
rules to beta-binomials, including .validate_family_counts, which silently
skipped integer-count validation for that family. It now matches exactly
first, then by longest prefix.
src/autodiff_fwd.h was a byte-equivalent copy of
inst/include/tulpa/autodiff_fwd.h serving a single include, and is deleted
rather than left to diverge once expm1 landed.
Audit fixes (0.0.91 review, issues #228-#239).
Correctness:
compute_eta_species) now adds the
per-observation offset, which it silently dropped -- multi-arm joint fits
with an exposure offset on a coupled arm no longer disagree with the
single-species path (#228).cyclic flag
reached the block but the rw2 precision / pattern / log-prior calls were
hardcoded acyclic, so a cyclic seasonal RW2 in a joint prior fit as acyclic.
This mirrors the single-arm fix in #218 (#229).rw1_rank / rw2_rank; the centered Type-IV path had drifted to a full-rank
cyclic normalizer inconsistent with the non-centered path (#230).(range, sigma) <-> (kappa, tau) conversion is now
nu-general (sigma = 1 / (sqrt(4*pi*nu) * kappa^nu * tau)) and single-sourced
via .spde_kappa_tau / .spde_range_sigma; the six copies hardcoded the
nu = 1 normalizer, mis-calibrating fractional-nu SPDE fits. The default
nu = 1 path is byte-identical (#231).select_main_params() now strips comma-separated multi-index latent names
(factor[i,j]), so latent-factor fields no longer flood the diagnostic
display (#232).test_dispersion, test_zero_inflation,
check_model) error via a shared .resolve_obs() when the observed response
cannot be found, instead of fabricating a "0 observed zeros" result (#233).tulpa_hyper_grid(var_of_means_consistency = TRUE) now recomputes the
per-cell log-prior after the consistency pass grows the grid, fixing a
length mismatch that returned NA reweighting (#234).Validation & docs:
spatial_gp() / spatial_svc() / spatial_multiscale() reject unsupported
covariances (gaussian, spherical) and Matern nu outside {1.5, 2.5} at
construction rather than deep in the fit (#238).tulpa_em_laplace(damping=) and agq_fit(sigma_eps=)
documentation to match the implementation (#239).Clean-up:
spatial_gp.R (#236); single-sourced the natural-scale hyperparameter
transforms shared by spatial_range() / temporal_corr() (#237).Audit fixes (0.0.90 review, issues #218-#227).
Cyclic RW2 dropped on the multi-block nested-Laplace path (#218). A
temporal_rw2(cyclic = TRUE) block was honored on the single-block and
exact-NUTS paths but silently ignored once a second latent block routed the
fit through the multi-block driver: the wrap-around second-difference penalty
and the T-1 (vs T-2) rank normalizer were hardcoded to the acyclic form.
The C++ multi driver and the joint-multi spec builder now thread cyclic for
RW2 as well as RW1.
NNGP marginal-SE precision builder over-allocated (#219). The triplet
accumulator pre-size squared the grand total of neighbour counts
(sum(...)^2) instead of the per-row sum((...)^2), allocating
O((n_spatial * nn)^2) integers -- tens of GB at a few thousand locations, so
summary() / vcov() / confint() on an NNGP fit could OOM. The result was
always numerically correct; only the allocation was quadratic.
tulpa() control surface omitted the joint keyset (#220). Inline
spatial() / temporal() field fits route through the joint nested-Laplace
driver, but tulpa()'s control whitelist did not union its keys and the field
fitter forwarded the raw control unmasked. Legitimate joint knobs
(adaptive_grid, prune, ...) were rejected at the front door, and some
keys valid elsewhere hard-errored inside the joint driver. The union now
includes nested_laplace_joint, and the field fitter subsets control to the
joint keys like every other backend route.
Outer Pareto-k target carried a spurious Jacobian on the single-block grid
path (#221). The default positive-scale grid is geometric (uniform in
u = log theta) and the integrator weights it with plain
softmax(log_marginal) and no volume element, so exp(log_marginal) is
already the u-space posterior density. Both the single-block
(.nested_grid_pareto_k) and joint / multi-axis (.joint_pareto_inv) paths
added a + sum(u) change-of-variables term on the log axes, tilting the
certified target away from the posterior the fit reports (a false reliable /
unreliable verdict; draws and moments were unaffected). Both now drop it on the
log axes, matching the integrator and the SPDE Pareto-k path. A
target-agnostic ground truth (PSIS of the grid-node log(w_k) - log q(u_k))
confirms the correction: with the Jacobian the outer k-hat overstated the truth
by ~0.25-0.55, enough to flip a verdict near 0.7. The correlation axis
(logit01) keeps its logit Jacobian, correct for the grid uniform in the
natural rho.
auto mode errored on SVC / TVC (#222). With the default
mode = "auto", a spatially- or temporally-varying-coefficient model fell to
a Laplace / size heuristic and then hit the varying-coefficient guard, which
only the exact ModelData NUTS backend clears. auto_select_mode() now routes
SVC / TVC to the exact backend so the default mode fits end to end.
Gradient-check fallback leaked across fits (#223). A failed warmup
gradient check flipped the process-global gradient mode to numerical and never
reset it, so every later fit in the session silently ran slower
central-difference gradients. The fallback is now scoped to a single fit
(restored on return), while a mode set explicitly via set_gradient_mode() is
preserved.
find_reasonable_epsilon ignored the active integrator (#227). The
warmup step-size seed always used a first-order leapfrog step regardless of the
selected scheme (yoshida4/6/8, minerror2, adaptive, mts). It now walks the
same SIMP op sequence the trajectory integrator uses, so the seed epsilon
matches the scheme's per-step energy error.
Diagnostic and marginal-SE test coverage (#224, #225). Added known-answer
tests for the generic moran_i / durbin_watson diagnostics (hand-computed,
plus spdep / lmtest cross-checks where installed), and a correctness test
pinning the continuous-spatial (NNGP) marginal fixed-effect SE against an exact
dense-GP penalized-IRLS Schur reference.
Comment cleanup (#226). Removed residual meta / status comments from the sampler and autodiff sources and repaired truncated comment fragments in the joint Pareto-k module.
Audit fixes (0.0.89 review, issues #207-#217).
Nested-Laplace crash on a length-1 multi-block prior (#207). The outer
Pareto-k diagnostic was invoked with type = NULL on the multi-block path;
the single-block decline guard did not fire for a length-1 block list, so the
.NL_REGISTRY[[NULL]] lookup errored. A single latent block wrapped in a list
(the tgmrf() / custom-latent front door) or a single block with a
model-supplied likelihood (the tulpaObs consumer path) now resolves the type
from the block itself. Default-config crash on both paths.
Self-loop adjacency corrupted the CAR/ICAR precision (#208). A user
adjacency with a non-zero diagonal was fit on a self-referential Q after only
a warning. .validate_adjacency_arg() now zeroes the diagonal (the graph is an
off-diagonal adjacency everywhere downstream), and the CSR builder excludes a
node from its own neighbour list as defence in depth.
temporal_ar1(rho_prior = ) is now wired end to end (#209). Supply a
prior_beta(alpha, beta) to place a Beta prior on u = (rho + 1)/2. The
compiled sampler kernel adds a*log(u) + b*log(1-u) (new ModelData
ar1_rho_prior_a / ar1_rho_prior_b, ABI 35 -> 36) and the nested-Laplace
outer grid is reweighted by the Beta density. Default Beta(1, 1) reproduces
the previous Uniform(-1, 1). Previously the argument was accepted and
documented but silently ignored.
Nested-Laplace fit accessors (#210). ranef() now grid-marginalizes the
random-effect tail of a nested fit (was empty); spatial_range() /
temporal_corr() summarize the spatial / temporal axes of a mixed or
spatiotemporal nested fit instead of erroring on the all(...) type check; and
diagnostic_summary() unwraps a $joint_fit wrapper like the shared
reliability readers, so a model-package subclass surfaces its Pareto-k.
Input-robustness fixes (#211). SVC / TVC terms = ~ f / character
interfaces expand a factor covariate to its contrast columns (shared
.resolve_varying_coef_columns()) instead of failing a bare-name match; the
inline temporal() field enforces the RW2 >= 3 time-point guard;
moran_i() drops self explicitly under coincident coordinates; and
check_diagnostics() returns NA on a too-short chain rather than a spurious
"checks passed".
C++ defensive fixes (#212). TapeScope move-assignment re-points the
active thread-local tape; a zero-byte file at a checkpoint path is treated as
fresh rather than a bad-magic error.
GP / NNGP Laplace pinned to serial (#217). Running the GP / NNGP Laplace
kernel multi-threaded triggered a flaky heap corruption under the mingw OpenMP
toolchain (a hard crash the second time it ran in a session). The
observation-scatter is the only OpenMP region there and its speedup is
negligible (small n_spatial, serial Vecchia prior scatter dominates), so the
kernel now runs on one thread; the crash is eliminated.
Docs and internal single-sourcing (#213). Removed the tulpa_psis()
tail_points "(with a warning)" doc that never fired; corrected the default
phi PC-prior description to match its behaviour (phi is the NB2 size, so
the default keeps phi finite / allows overdispersion, with phi -> Inf the
Poisson limit); .tulpa_param_layout() builds RE names through the single
.re_names_from_layout() source; grammar fix in priors_default().
Front-door hyperparameter recovery + CI coverage (#214). New
test-hyperparameter-coverage.R gates the hyperparameter posteriors of the
nested-Laplace paths against simulated truth with >= 20-seed CI-coverage
checks: temporal AR1 (rho, precision), proper CAR (sigma, rho), GP / NNGP
(sigma, range), HSGP (sigma), BYM2 (total spatial SD), and the free-Sigma
random-slope correlation. Previously only field shapes and fixed slopes were
recovered; the variance / range / correlation hyperparameters had no coverage
test.
Test quality (#215, #216). The SVC / TVC exact-NUTS front door gained a
divergence guard and scale recovery (the configuration the #144 divergence bug
survived in). The debias is now shown as a differential: on small binary
groups the exact Gibbs draw of Sigma lifts the under-dispersed nested sigma
toward truth. tgmrf_cpp() recovery is checked across seeds (cpp == R to 1e-6
per seed, inheriting the R closure's verified recovery). test-gpu-nngp was
retitled to the path it exercises; test-hsgp-recovery renamed to
test-hsgp-density-identity (it is an analytic identity, not a fit); the
leapfrog-drift baseline now fails loudly if missing; the inference-tier test
checks every backend's tier; and the native Rhat/ESS/MCSE reference tolerance
was tightened from 1e-4 to 1e-12 (the estimators match posterior:: to
~8e-16).
Audit fixes (0.0.88 review, issues #193-#206).
H_beta in the dense-Hessian branch
that ignores the spatial field, so summary() / vcov() / confint()
reported fixed-effect SEs at the wrong linear predictor and without
marginalizing the field. Added .marginal_H_beta_icar / _bym2 (field
precision L + 11'; the BYM2 two-block convolution), validated against an
independent penalized-IRLS reference (test-marginal-se-areal.R).control knobs no longer dropped (#194). The spde /
re_cov_nested / re_cov_gibbs / gibbs branches hand-built the inner control
list, silently discarding valid knobs (diagnose_k, k_samples,
checkpoint, n_threads, ...); they now forward the validated subset.(0 + x | g) was silently fit as a random intercept on the group-index-only
paths; .is_scalar_re_intercept() now gates AGQ / Gibbs / SPDE / nested-RE
consistently. Plain Laplace still fits it (carries the slope column as Z).n_save floored while the store loop
fires ceil(post / n_thin) times (out-of-bounds write); multi-term RE with
distinct sigma now hard-errors instead of freezing the extra terms.format = "summary" (s, t)
index was t-fastest while draws are s-fastest, mispairing labels when S != T.tulpa_hyper_grid
refreshes the per-cell log-prior on the refined grid (was stale-length -> NA
into power-scaling); the single-block outer Pareto-k is computed on the
default grid (not only when the user named it); a nested-prior EM block
attaches a grid-marginalized H_beta so the MI / Gibbs correction reports a
real pooled SE instead of NaN.spatial_range() /
temporal_corr() returned raw grid axes (tau, phi_gp, sigma2) on a
nested-Laplace fit but interpretable range / sigma / rho on a sampler fit. The
nested path now maps each axis to the same interpretable quantity, computed
per grid cell then weighted-summarized (sigma = 1/sqrt(tau) or sqrt(sigma2);
range = 3 * lengthscale).plot_pairs() selects fixed effects
from fit$fixed_names (was hard-coded to ratio's beta_num / beta_denom);
plot_diagnostics() guards an all-NA Rhat; NNGP neighbour builders no longer
index a non-existent row at N == 1; ranef() no longer emits the field /
hyperparameter tail as random effects; plot_acf() / geweke_test() handle a
3-D [iter, chain, param] fit; (x - 1 | g) drops the intercept;
spatial_multiscale() sampler narrowed to the documented modes.dev_notes/ pointers removed from
user-facing stop() messages and comments, refactor-history narrative
reworded to describe current behavior, a dead no-op and a duplicated MCSE body
removed, and the multi-block CAR_proper log-det cache made cell-keyed
(NlCellCache) to match the single-block path.cpp_test_funnel_nuts, src/tulpa_test_funnel.cpp) fits Neal's funnel
through the exact production NUTS path with the retry forced on or off, and
test-softabs-retry-invariance.R checks that toggling the retry on removes
divergences without shifting the v marginal (seed-averaged paired
mean/sd/tail differences within tolerance over 10 seeds). Verdict:
invariance holds empirically -- 24-seed paired |t| < 0.4 on every summary,
divergences 735 -> 8, no posterior shift.Bug fixes and cleanups from a second whole-repo audit (issues #176-192).
rank = T in the cyclic branch; a cycle-graph intrinsic GMRF has a single
null direction, so the rank is T-1 for both RW1 and RW2. The 0.0.86 fix had
landed only on the double-precision twin. The rank is now single-sourced
through tulpa_temporal::rw1_rank / rw2_rank, consumed by every site.cyclic flag and the front door only propagated
it for RW1, so temporal_rw2(cyclic = TRUE) silently fit a non-cyclic RW2.
The dense / sparse / pattern kernels now add the two ring-closing second
differences and the flag is threaded through the RW2 front door.car_proper nested grid (#178). Supplying only one grid axis no longer
discards the other, and the correlation axis is accepted under both rho_grid
and the joint-API rho_car_grid spelling.re_idx (#180)
so a resume onto a checkpoint written for a different grouping errors instead
of loading stale cells.get_compute_layout_fn() verifies the ABI version (#181) like the sibling
registered-callable getters, so an ABI-mismatched consumer gets a clear error
rather than silent memory corruption.divergent / n_divergent.ranef() on sampler-tier fits (#183) returns exactly the random-effect
coefficients (the re[...] draw columns), no longer re-including the latent
field and a spurious log_sigma_re row.compute_bym2_scale() now returns the
Riebler et al. (2016) generalized-variance factor 1 / sqrt(geomean(diag(Q^+)))
instead of the geometric mean of the ICAR eigenvalues, so the spatial fraction
rho stays interpretable across graphs.spatial_range() / temporal_corr() (#186) now summarise the outer
hyperparameter grid on a nested-Laplace fit rather than erroring on the
primary spatial / temporal fitting path.sigma_re ignored-argument warning (#187) also fires when the covariance
backend is selected by name (mode = "re_cov_*").tulpa_simulate(theta = <engine fit>) (#188) consumes a single-process
matrix-draws fit instead of erroring.A / A_t alias
the arena reverse-mode path rather than claiming separate kernels (#185)..default_tau_grid
argument cleanup and the spatiotemporal long-format s/t ordering (#192).The nested-Laplace tensor-grid integrator was investigated (#179) and left
unchanged: CAR_proper (tau, rho) parameter recovery confirms the marginal is
already in the internal log-scale parameterization, so equal-weight grid
integration is correctly calibrated and adding a user-scale Jacobian biases the
scale posterior.
Bug fixes from a whole-repo audit.
tulpa(y ~ (1 + x | g), mode = "mala" / "pathfinder" / "imh_laplace")
previously fell through the covariance redirect (gated on the Laplace backend)
and fit the term with a single scalar sigma_re per block -- dropping the
intercept/slope correlation, forcing the SDs equal, and conditioning at
sigma_re = 1. These modes now route to the exact Metropolis-within-Gibbs
Sigma debias, matching the documented coverage.rank = T; a cycle-graph GMRF has a single null direction
(the constant), so the correct generalized-determinant rank is T - 1 for
both RW1 and RW2. The old value biased the integrated / sampled precision
tau upward (oversmoothing), scaled by S - 1 in Type-IV interactions.compare_models() weights. The Akaike / pseudo-BMA weight on the elpd
scale used exp(0.5 * delta); corrected to exp(delta) (weights were
systematically too uniform).check_model() asked for residuals(type = "deviance"), which is not a
supported residual type and errored on the base fit; panel 2 now uses Pearson
residuals.geweke_test() and plot_acf() now
respect the draws-provenance gate (they returned a vacuous "converged" result
on i.i.d. nested-Laplace / VI draws); plot_pairs(), plot_divergences(),
and plot_energy() no longer error when $backend is NULL.spatial_range() / temporal_corr() named the quantile columns q025 /
q975 regardless of probs; the columns are now derived from probs
(e.g. q2.5, q97.5).tulpa() now rejects NA/NaN/Inf in the
response or model matrix with the offending row, instead of letting a NaN
propagate silently into the kernels (the model is built with na.pass).svc(approx = "nngp") now rejects duplicated coordinates (a distance-0
neighbour gave a singular per-node covariance) rather than failing silently
downstream.Warnings and cleanups.
tulpa() now warns when sigma_re is passed
for a random-slope model (the covariance is integrated, not conditioned on a
scalar SD), and spatial_multiscale() surfaces its default range-prior bounds
when they are left unset.validate_spatiotemporal,
prepare_spatiotemporal_for_hmc, build_st_index, and the two precision
builders -- superseded by fit_st_nested()), the internal
has_implicit_intercept(), and a no-effect temporal-Gibbs selection gate were
deleted; the proper-CAR rho-bounds eigenvalue roles were corrected (they were
swapped, though the (0, 1) clamp masked it).predict(type = "response") documents that a
binomial fit returns the per-trial probability (vs fitted()'s trial-scaled
count); the k-quality band index now shares the reliability bands the
band-confidence flag used (at the realised finite-draw count) rather than
recomputing them at the draw budget; the fractional-SPDE rational-order default
is single-sourced to 2; and two split-message errors and a couple of stale
comments were tidied.spatial_range() and
temporal_corr() now share one hyperparameter-summary scaffold; the three RNG
snapshot/restore helpers share one snapshot + restore-closure factory; the
outer Pareto-k importance core delegates to the batched core (identical draws);
and the temporal-GP PC prior routes through the shared pc_prior.h form
instead of a hand-rolled Jacobian. The GP and SVC NNGP analytic gradients now
share one Vecchia conditional-gradient assembler (nngp_cond.h; each keeps its
own solver and distance source), and the fast leapfrog drift carries the same
sparse-GMRF mass-block range override as inv_mass_times_p() so the integrator
and U-turn check cannot use different metrics.gcol33/tulpa#NNN in all
forms, parenthetical (#NNN) lists, and bare issue numbers) and stale version
tokens ("Phase 1.3") were stripped from code comments and roxygen, keeping the
domain rationale. References inside error messages, test_that() labels, and
design-principle / improvement enumerations (principle #5, improvement #1)
were preserved.Per-block quadrature order and an optional variance-component prior on the AGHQ path.
tulpa_re_aghq(n_quad = ...). n_quad now accepts an integer vector of
length length(re_terms) giving a per-block node count, alongside the existing
single integer broadcast to every covariance block. The tensor grid then uses
n_quad[b] nodes along every dimension of block b
(prod_b n_quad[b]^(dim_b) total nodes); a scalar reproduces the uniform grid
exactly (byte-identical). Per-block orders let a heterogeneous stack spend fewer
nodes on cheap scalar nuisance blocks (a dispersion or zero-inflation random
effect) than on the correlated coefficient blocks. R-only, no ABI change.
tulpa_re_aghq(sigma_prior = ...). A Penalized-Complexity prior on the
marginal standard deviations of one or more random-effect covariance blocks,
added to the ML-II objective (and hence the marginal Hessian). NULL (default)
is pure ML on the covariances, byte-identical to before. A c(U, alpha) pair
(P(sigma_i > U) = alpha, the re_cov_pc_lkj_prior() convention) applies to
every block, or list(blocks = <indices>, prior_sigma = c(U, alpha)) to named
blocks only. Reuses the exact PC log-prior + Jacobian of re_cov_pc_lkj_prior()
(single source of truth), so the + log sigma Jacobian repels sigma -> 0 and
the - lambda sigma term caps inflation. A weakly-identified variance component
(e.g. a scalar dispersion / zero-inflation random effect at few groups) can
drift to the boundary and flatten the marginal Hessian; a weak PC prior adds
curvature there, keeping the joint optimum non-singular without materially
shifting an identified fit. R-only, no ABI change.
Checkpoint fix (#161).
max_iter, tol, inner_refresh). Those
knobs are part of the checkpoint fingerprint, so with control$checkpoint set
and diagnose_k = TRUE the diagnostic pass computed a fingerprint that did not
match the file the main outer grid had written, and the fit stopped with a
"fingerprint mismatch" error after the main grid had already completed. The
diagnostic solve now runs checkpoint-free (via the same quiet-options path the
CCD / adaptive probes already use), so only the main outer grid owns the
checkpoint. A resumed fit stays byte-identical to an uninterrupted one; fits
with diagnose_k = FALSE are unaffected.Front-door API convention cleanup (#156, fully closed) and the
missing-front-door features (#158, fully closed). ABI break
(TULPA_ABI_VERSION 34 -> 35) for the proper-CAR exact-NUTS eigenvalue
log-determinant; downstream packages must rebuild.
beta_prior = list(mean, sd) is now the one
interface across tulpa_ep, tulpa_multinomial, tulpa_ordinal,
tulpa_gibbs, tulpa_gaussian, tulpa_re_cov_gibbs, tulpa_nuts_beta, and
tulpa_nuts_spde, replacing the four names (beta_prior_sd, sigma_beta,
prior_beta_sd, beta_prior_mean/_sd) it had before.control. New statistical re_prior =
list() argument on tulpa() carries the random-effect / variance-component
hyperpriors (prior_sigma, eta, prior_df, prior_scale,
prior_sigma_scale, sigma_re_scale); sigma_beta folds into beta_prior.
tulpa(control = list(prior_sigma = )) now errors and points at re_prior.tulpa_gibbs, tulpa_gaussian, tulpa_nuts_beta, and
tulpa_nuts_spde move their perf/sampler knobs into control = list();
tulpa_gibbs threads thin and seeds via the session RNG.tulpa_laplace_beta(mode = "nuts") and
fit_spde(mode = "nuts") delegate to the NUTS engines, folding the
tulpa_nuts_beta / tulpa_nuts_spde verb variants behind a mode= switch.tulpa_nested_laplace, tulpa_gibbs, and the
two re_cov fitters now check nrow(X) == length(y), n_trials length, and
re_idx / group range up front.tulpa_em_laplace() rejects the
never-consumed spatial= / re_list=; tulpa_re_cov_gibbs() errors when a
prior_scale matches no RE block; plot.tulpa_st_summary() implements
type = "spatial_map" and errors on an unknown type.tulpa(mode = "ep") fits a
fixed-effect GLM by Expectation Propagation (Tier 2 structured).(1 | g) term now rides
alongside a Matern SPDE field through fit_spde() and tulpa()
(integer-nu, conditioned on sigma_re).predict() kriging for continuous fields (#158): predict() now
interpolates the posterior-mean field to new coordinates for HSGP
(spatial_gp(approx = "hsgp")) and GP/NNGP (spatial_gp()) fits, not just
SPDE. Both are marginalised over the hyperparameter grid via new native
kernels (cpp_hsgp_field_predict, cpp_gp_field_predict); held-out recovery
cor > 0.9 against a known surface.tulpa() GP/NNGP nested-Laplace fit aborted with "'a' is
computationally singular" on ordinary data (Matrix's condition guard on the
ill-conditioned joint precision); the fixed-effect marginal-Hessian solve now
retries with a negligible jitter, so gp/nngp fits complete.tulpa(spatial = <spde>,
mode = "exact") routes to NUTS over the Matern field + hyperparameters.fit_st_nested() wires
the cpp_nested_laplace_st_{icar,bym2,car_proper} kernels (additive areal
field + rw1/rw2/ar1 temporal field, integrated jointly over the spatial /
temporal precisions and the ar1 autocorrelation), previously reachable only
from consumer packages. Recovers both fields (cor > 0.85) on simulated data.latent_factor() roxygen corrected: it is a ratio / multi-arm construct for
consumer packages, not the single-response tulpa() front door (#158).tulpa(spatial = <field>, mode = "exact") now samples the latent field, its
variance, and (where present) its range / correlation jointly with Tier-1
NUTS for GP / NNGP (spatial_gp()), HSGP (spatial_gp(approx = "hsgp")), and
proper CAR (spatial_car_proper()), joining the SPDE exact path. The generic
ModelData sampler keys each block on spatial_type, so the field enters the
parameter vector (previously only the fixed effects were sampled). The
proper-CAR log-determinant log|D - rho W| is evaluated in the
autodiff-friendly closed form sum_i log(1 - rho mu_i) from the precomputed
adjacency eigenvalues, not a per-gradient Cholesky. Each field's NUTS
posterior-mean recovers the simulated truth and matches its nested-Laplace
counterpart (cor > 0.88).tulpa(spatial = spatial_svc(...), mode = "exact") and
tulpa(temporal = temporal_tvc(...), mode = "exact") fit NNGP spatially-
varying and RW1/RW2/AR1 temporally-varying coefficients, threaded into the
generic sampler via new svc_spec / tvc_spec inputs. Exact-only: a nested /
laplace / auto mode errors rather than silently dropping the varying field.
Recovers the varying-coefficient surface / trajectory on simulated data
(cor > 0.94).The sampled spatial range gets a real prior. ABI break
(TULPA_ABI_VERSION 33 -> 34); downstream packages must rebuild, and any
package placing a gp() / svc() NNGP block must now supply the range
anchors (see the last bullet).
FIX (statistical): the GP and SVC NNGP paths placed a Uniform prior on the
spatial range phi behind a hard return -INFINITY outside
(phi_prior_lower, phi_prior_upper). Two defects compounded
(gcol33/tulpa#144). The rejection sits inside an autodiff log-posterior, so
a step landing outside the box produced no usable gradient and NUTS reported
it as a divergence; and the + log_phi Jacobian made the flat density in the
sampled log_phi a Uniform on phi itself, whose default (0.01, 10)
carries mean ~5. Under a weakly informative binary likelihood the posterior
collapsed onto that mean. Measured downstream on occu() + svc() with a
truth of phi = 0.25 on unit-square coordinates: 72-83% of post-warmup draws
divergent and phi at ~4 across every seed (gcol33/tulpaObs#118). Both paths
now sample log_phi unconstrained under a PC prior on the range, which is
proper on (0, inf) and needs no bounding box -- the same shape the SPDE
field has always used for log_kappa.
FIX (statistical): the SVC NNGP block's half-Cauchy prior on the marginal SD
was improper on the coordinate it is sampled on, so nothing bounded the SVC
marginal SD from above. The density is written on sigma
(-log(1 + sigma^2 / scale^2)) but carried to the sampled log_sigma2 with
that coordinate's variance-row Jacobian (a full log_sigma2) rather than
its own (-log(2) + 0.5 * log_sigma2). The surplus 0.5 * log_sigma2 flattens
the tail exactly: log p tends to a constant as sigma2 grows, so the mass
below a bound grows linearly in that bound instead of converging, and only the
likelihood pulled sigma back. The GP block was never affected -- its base
density (log_prior_sigma2_pc_t) is written on sigma2, so the full
log_sigma2 is the right Jacobian there. Both scale priors now come from
named helpers in pc_prior.h (log_prior_sigma_half_cauchy,
log_prior_log_sigma2_half_cauchy) rather than being spelled out at the
sampling site, which is the failure mode that header exists to prevent.
test-pc-prior.R asserts the density integrates to the half-Cauchy
normalizer scale * pi/2 and that doubling the integration bound leaves the
accumulated mass unchanged.
FIX: every NNGP fit whose locations carry replicates returned
H_beta = NULL, so all fixed-effect standard errors were lost -- degraded to
a warning rather than an error, and covered by a test that guarded its own
assertions behind if (!is.null(fit$H_beta)), which passes precisely when
the builder fails. The obs -> unit design read the map from spatial_idx,
the field an areal spec carries; a GP spec calls it obs_to_loc. Absent,
the map defaulted to seq_len(n_obs), whose column index runs past
n_spatial as soon as one location holds more than one observation. The
helper now takes the map and the unit count from its caller, and rejects a
map that is missing, short, or out of range instead of defaulting.
FIX (statistical): spatialRange() reported the reciprocal of the range. It
computed the effective range as 3 / phi, commented as a decay parameter,
but every kernel this engine ships is exp(-d / phi) (cov_exponential,
cov_matern32, cov_gaussian), so phi is the range and the effective
range is -phi * log(0.05) ~= 3 * phi. A fit at phi = 0.25 was reported at
12 rather than 0.75.
FIX (statistical): the unwired log_prior_phi_pc set its rate to
-log(alpha) / U while its own derivation, exp(-rate / U) = alpha, gives
-U * log(alpha), and it disagreed with both written PC range priors. It had
no call sites, so it never reached a fit; it is removed rather than wired.
test-pc-prior.R pins the contract it violated by integrating the density:
P(range < U) = alpha.
REFACTOR: the d = 2 PC range density now lives once, in pc_prior.h
alongside the PC scale densities, and the SPDE hyper-prior consumes it
instead of restating the closed form. test-pc-prior.R checks it against
pc_prior_log_density(), the independently written R twin the SPDE nested
path integrates against. The stale block comment in tulpa_priors_spde.h
still documenting the superseded d = 1 form is corrected.
REFACTOR: the logit-bounded parameterization (map + exact Jacobian) is
extracted from the temporal GP path into bounded_from_logit() /
log_jacobian_bounded() in autodiff_utils.h. Behaviour is unchanged; the
temporal range keeps its Uniform prior, since moving it off Uniform needs the
d = 1 PC density that pc_prior.h deliberately does not provide.
BREAKING: ModelData carries gp_phi_prior_U / gp_phi_prior_alpha and
svc_phi_prior_U / svc_phi_prior_alpha (encoding P(range < U) = alpha)
in place of the _lower / _upper bounds. They ship unset (-1.0) and
compute_param_layout() errors once, before sampling, on any NNGP block that
fails to set them -- matching the SPDE field's existing prior_range
contract. The engine does not default them: a silent default is a prior, and
an unanchored range under a weakly informative likelihood is precisely how
phi came to sit at the old Uniform's mean. The SVC HSGP path is unaffected;
it puts a LogNormal(0, 1) on an unbounded log-lengthscale and reads neither.
The internal tulpa_version() reads DESCRIPTION via
utils::packageVersion() instead of returning a version restated in C++. The
literal dated to the first commit and had never been updated, so it disagreed
with DESCRIPTION across the whole 0.0.x line; the test asserted the same
literal, which is why nothing caught it. The C++ entry point is removed rather
than kept in sync, and the test now reads DESCRIPTION directly.
TULPA_ABI_VERSION remains a compiled constant by design: it describes the
DLL a model package linked against, not the metadata beside it.
Third deep-audit pass: statistical, memory-safety, and backend-consistency fixes surfaced by a fan-out code audit.
1e-8 nugget to every diagonal, the autodiff copy added it only to an
already-degenerate pivot, so on well-conditioned data it added none at all.
The analytic GP gradients are finite-differenced from the double copy, so the
value and the gradient described different models on ordinary input, not just
degenerate input. (hmc_gp_autodiff.h carried a "known heisenbug with
autodiff" note.) Both twins now share one kernel and one pair of constants.double-only core the autodiff twin could not use, so the
twin kept its own literals; and the function it consolidated has no callers,
while the live SVC path runs the autodiff one. Both now read
tulpa_svc::kSvcJitter / kSvcVarFloor, pinned at the live path's values, so
behaviour on the reachable path is unchanged.nngp_cond.h, shared by the SVC and GP kernels on both the
double and autodiff paths -- the previous shared core was double-only,
which is precisely why the autodiff copies were hand-written and drifted. The
per-kernel conditioning constants stay explicit arguments (the SVC kernel
deliberately runs looser than the GP one), but a kernel can no longer differ
from its own twin. New test-nngp-twin.R asserts each autodiff kernel,
instantiated at T = double, returns its double twin's value -- including on
near-duplicate coordinates, where the conditioning actually bites
(gcol33/tulpa#142 A3).dcov_dphi in
hmc_gp_gradients.h dropped the factor of 2 in k * 2*d^2/phi^3. The SVC
copy of the same function had it. A spatial_gp(cov = "gaussian") HMC/NUTS
fit therefore explored the range on a mis-scaled gradient.spatial_gp(cov = "spherical") computed its covariance from the spherical
kernel and its gradient from the exponential one -- the value and its
derivative described different kernels. The spherical derivative is now
implemented (it needs sigma2, since unlike the others it is not proportional
to k(d), which is why it had been left to fall through).dcov_dphi is single-sourced: the GP copy delegates to the canonical
tulpa_svc::dcov_dphi_svc. New test-cov-kernel.R checks every covariance
type's range-derivative against a numerical derivative of the value function
it differentiates, so a copy cannot drift from its own kernel again
(gcol33/tulpa#142 A5).rw1_grad_w read one past the end of w at a single time point, and
rw2_grad_w wrote one past the end of its second-difference buffer for fewer
than two. The sigma2-parameterized twins guarded both cases; the precision
twins, written separately, did not. The RW2 and AR1 gradients are now thin
wrappers over the canonical precision kernels (as RW1 already was), and the
AR1 rho gradient floors only the dividing 1 - rho^2 so it stays finite at
the stationarity boundary without biasing the stationary precision. New
test-temporal-grad-equiv.R pins the wrappers against the canonical kernels
and both against numerical derivatives of their priors -- neither header was
reached from a compiled translation unit, so nothing would previously have
caught a wrong wrapper (gcol33/tulpa#142 A7).control$hessian was parsed and match.arg-validated on the
multi-block joint nested-Laplace path and then never passed to the kernel, so
"psd" and "fisher" silently ran as LM / observed curvature. The
single-block path forwarded it correctly, so the setting worked or not
depending only on how many blocks the prior had. The seven .cpp_joint_multi
call sites now go through one fit-scoped factory
(.joint_multi_call_factory) that supplies the invariant arguments once, so
a per-fit setting cannot be honoured at one solve and dropped at another; each
site drops from ~19 arguments to a handful (gcol33/tulpa#142 A6).temporal_ar1() and spatial() silently accepted shared = FALSE
while every sibling constructor warned about unshared confounding structure.
The warning was copied into 12 constructors, two of which had also drifted to
a shorter one-sentence form; all now call one .warn_nonshared() helper.spatial_car(), spatial_bym2() and spatial() validated a raw
adjacency matrix only for squareness and exact symmetry -- strictly weaker
than the validator adjacency() / check_adjacency() already use. A graph
with self-loops, non-binary weights or isolated nodes was reported by
check_adjacency() and accepted silently by the constructors, which then
built an improper field. They now share .validate_adjacency_arg(), so the
same graph reports the same way whichever door it enters. This also stops the
constructors densifying a sparse graph to test symmetry (O(n^2) memory) and
accepts float-rounded symmetric matrices that check_adjacency() accepts.
New test-field-constructor-shared.R pins the cross-constructor consistency
(gcol33/tulpa#142 A8).lchoose(n, y)
normalizer, so it was not a true log-density: a binomial logLik / WAIC /
cross-backend comparison was off by sum(lchoose(n_i, y_i)) whenever
n > 1, while the autodiff and GLMM-oracle paths (and dbinom()) kept it.
The term is eta-independent, so no mode, gradient or normalized grid weight
moves; only the reported absolute value changes. Bernoulli data (n = 1) is
unaffected.test-family-cross-path.R evaluates every per-family (loglik, grad,
curvature) kernel maintained in parallel -- the Laplace/Newton dispatch, the
compiled GLMM oracle, and the explicit triplets -- at a shared (y, eta, phi)
and pins them against each other and against R's own densities. It guards the
0.0.73 bug directly: phi is the residual SD in the Laplace kernels and the
residual VARIANCE in the GLMM oracle, bridged only by convention in R, so the
test asserts glmm_elt(phi = s^2) == log_lik_for_family(phi = s) and checks
that passing the wrong scale is actually detectable. Adds the three probes
that made this constructible (cpp_family_terms, cpp_glmm_elt_terms,
cpp_test_laplace_gaussian -- the gaussian triplet carrying the phi
convention previously had no callable surface) (gcol33/tulpa#142 A9).+2*log_tau, tilting the prior by tau^2 toward large precision (improper as
tau -> Inf) and biasing time-varying-coefficient SDs low -- coefficients were
over-smoothed toward constant. The identical error had been found and fixed in
the spatiotemporal block but was never propagated to the TVC copy.+log(sigma), leaving the density
flat (improper) at the origin and biasing the HSGP amplitude low. The same
applied to the HSGP-ST block.pc_prior.h is templated over the
autodiff types and provides the density on every scale a sampler parameterizes
(sigma, log_sigma, sigma2, log_sigma2, tau, log_tau), each derived
once from the base exponential plus its Jacobian. Every site delegates. The
header was previously double-only, which is why each autodiff path re-derived
the algebra by hand -- and why two of them got it wrong. New
test-pc-prior-scales.R pins each scale against the base density plus a
numerical Jacobian and checks each integrates to one (gcol33/tulpa#142 A4).spec_inner_solve now compensates process 0's intercept like the joint
driver; RW1/RW2 intercept recovery tests added.tulpa_priors_re.h / lkj_chol_helpers.h. The test now
cross-checks the Stan Cholesky lpdf plus an independent
det(R)^(eta-1) * |dR/draw| pushforward.z -> f Jacobian (the NC target is -0.5 z'z); the spatiotemporal PC-prior
Jacobian had the wrong sign for the log-precision parameterization; the SPDE
range PC prior flipped its tail in the nested path and used the d=1 shape for a
2-D field in the NUTS path. All corrected to the d=2 Fuglstad et al. (2019)
form; the stale d=1 test reference is updated.rho in (0,1);
it now maps to (-1,1) to match the documented Uniform(-1,1) prior and
build_ar1_precision.S - k) like the Laplace
path, via a shared count_graph_components helper (ABI 32 -> 33).i to
field node i, mis-mapping and dropping observations when coordinates repeat
(n_spatial < N); it now uses the per-observation obs_to_loc map.theta
component to sum-to-zero (it has a proper N(0,1) prior and no centerer);
tulpa_re_aghq() reported log_marginal including the LKJ penalty; and a
nested-Laplace mixture node with a failed Cholesky became a zero-variance
point mass that deflated fixed-effect CIs. All corrected.n_threads_outer exceeded the
environment thread count (e.g. OMP_NUM_THREADS=1 with n_threads_outer > 1),
corrupting the shared CHOLMOD factor; the outer width is now clamped. Also
fixed: an out-of-bounds write in the NNGP non-centered forward, a missing
exception barrier around a tweedie Rcpp::stop in a parallel region,
per-thread reduction buffers allocated inside a parallel region, and an
untrusted checkpoint record length driving a multi-GiB allocation.store_Q left the
stored per-cell Q misaligned with the refined grid and crashed
tulpa_posterior_draws; local-CCD is now skipped when store_Q is set. The
k_quality grid-refinement verdict no longer over-claims on multi-block fits.sgld, sghmc, and (unadjusted) mclmc are reclassified
from tier "exact" to "optimized" -- they carry discretization / minibatch
bias, so mcmc_diagnostics() no longer certifies them as exact and auto-mode
never selects them silently. smc stays exact.print.tulpa_spatial was defined twice (the SPDE copy
shadowed the areal ICAR/CAR/BYM2 formatter); merged into one method. A dead
duplicate .with_preserved_seed was removed.tulpaRatio example blocks moved from \donttest to \dontrun;
tools declared in Imports; the ggplot2 vignette chunks gated on
requireNamespace().test-pg-spatial-recovery.R): the ICAR tau shape now counts connected
components and the field mean is absorbed into the intercept; the single- and
multi-scale GP kernels use the NNGP-correct conjugate sigma2 and MH phi
(shared update_nngp_scale, DRY) with the field level anchored (they diverged
before -- sigma2 railed, phi did a data-free random walk); the temporal AR1
conditional now uses both neighbours and rho is sampled (it was frozen at its
init yet reported as a posterior); and the BYM2 component updates remove only
the other component's contribution. The two-process negbin kernel now draws the
exact real Polya-Gamma shape (rpg_real).CRAN-preparation release.
N(m, (L'L)^{-1}) instead of N(m, (LL')^{-1}) -- a forward
substitution where the transpose back-substitution was required -- so every
PG Gibbs backend (binomial ICAR/BYM2/RSR/GP/multiscale/temporal, negbin,
the two-process negbin) sampled beta with the wrong covariance whenever
p >= 2 and the design was correlated. Means were unaffected. Draw
covariance is now pinned against the asymptotic glm vcov in a regression
test.H_beta (and
the RE / spatial-field Schur complements) evaluated the GLM weights at
eta WITHOUT the observation offset, so vcov() / SEs / confint()
described a different model on any non-gaussian offset fit (poisson probe:
link-scale SEs off ~2.3x). The offset now enters the Hessian eta on every
path.omega ~ PG(y + r,
eta) rounded its real shape to an integer, biasing the chain worst at
zero counts with small r. New rpg_real() draws the exact real-shape
Polya-Gamma (truncated sum-of-gammas with tail-mean correction). Same
pass: the spatial negbin kernel's ICAR tau shape counts adjacency
components, its sum-to-zero centering absorbs the field mean into the
intercept (posterior-invariant), and all three negbin kernels share the
exact half-Cauchy auxiliary scheme for sigma_re.prep-rebuilt cache; under the joint driver's parallel
outer grid (control$n_threads_outer > 1) concurrent cells silently
solved at each other's hyperparameters. Per-cell state is now published
under the cell id (NlCellCache), safe under the coupled-cell scatter's
work-stealing tasks; joint and single-arm car_proper use the same cache.
Regression test: joint fit identical under n_threads_outer = 2 and 1.std::terminate); the remaining six registered C callables carry the shim
exception guard; every OpenMP region is sized by an explicit
num_threads(...) clause through the OMP_THREAD_LIMIT-aware clamp and
the process-global omp_set_num_threads() mutations are gone; allocation
sizes compute in size_t.control names against a canonical
whitelist (a misspelled knob was a silent no-op) and tulpa() errors on
stray ... arguments (familly = "poisson" used to fit a gaussian). The
joint fitter's diagnose_draws knob is renamed to k_samples, the one
name every nested-Laplace fitter uses (hard rename, no alias).coef / summary / confint / vcov /
predict (the nested return now carries an anchored mode plus the
field-marginal H_beta); direct tulpa_nested_laplace() /
tulpa_nested_laplace_joint() fits slice coef() to the actual fixed
effects instead of relabelling the full latent vector; tulpa_gibbs()
returns a classed tulpa_fit; fitted() includes the offset and trial
scaling so y - fitted(fit) equals residuals(fit, "response");
tidy / glance re-export the generics generics (no masking next to
broom).compare_models() / model_average() on Laplace-tier fits are
deterministic (the Gaussian-synthesis draws behind the reconstructed
log-likelihood are seed-pinned and RNG-neutral).thread_local C++ objects inside
OpenMP parallel regions corrupt the process heap under the mingw toolchain.
One call to the NNGP CG/PCG solver (spatial_gp(solver = "cg")) was enough
to crash the R session minutes later (Windows fail-fast 0xC0000374/09), and
the joint coupled-cell scatter / parallel-chain NC-GP storage carried the
same pattern -- the long-standing "OpenMP joint-parallel test heap-corrupts
under parallel testthat" instability. All four sites now use either
function-local buffers or constant-initialized POD-pointer TLS (the safe
pattern the autodiff arena already used). The full test suite completes in
a single process again.temporal(~ ... || time) varying-coefficient fields reported
empty per-time field means (fit$temporal_fields[[...]]$mean had length 0):
the shared bar-field core read the joint layout's field_starts, which
registers only areal block types. It now reads the per-block block_start,
which is aligned for every block type.tulpa_poisson_gamma() / tulpa_binomial()
and pre-control arguments) are rewritten as runnable single-response
engine examples (plot_rhat, spatial_bym2, spatial_hsgp, spatial_rsr,
temporal_rw2, temporal_ar1) or marked \dontrun with their tulpaRatio
provenance; all 318 Rd example sets now run clean including \donttest.
Slow-but-runnable examples moved from \dontrun to \donttest (\dontrun
kept only on experimental-path examples); new runnable examples for the core verbs
(tulpa_criteria(), compare_models(), model_average(),
mcmc_diagnostics(), tidy(), glance(), ranef(), moran_i(),
posterior_sample(), tulpa_em_laplace()); R sources and Rd are
ASCII-clean; plumbing exports (ccd_grid(), hyper_axis_spec(), sn_*(),
findbars(), ...) are @keywords internal.residuals.tulpa_fit ("pearson" / "response", computed
from the family registry) and nobs.tulpa_fit; print.tulpa_fit rewritten
to be shape-robust across the sampler / Laplace / nested tiers (it printed
Gaussian-only fields before).$draws$log_lik, the
pointwise log-likelihood is computed from the linear-predictor posterior
draws and the family registry, so compare_models(), model_average() and
the criteria layer work on tulpa() fits from any tier.pit_residuals(), test_dispersion(),
test_outliers(), test_zero_inflation(), ...) read the engine fit's
stored response ($y); the default residual type for moran_i() /
durbin_watson() is "pearson", matching the new residuals() method..Random.seed when the fitter exits (9 fitters plus bayes_R2()),
so a seeded fit no longer clobbers the session RNG stream.control$re_cov = "aghq": random-slope fits can route the nested Sigma
integrator through the AGHQ inner marginal (control$n_quad, default 9
there).tulpa_re_aghq(): maxit renamed max_iter (hard rename, no shim). The
EM drivers report progress via message() (suppressible); tulpa_gibbs()
is quiet by default.extern "C" C-callable shim (the model-package ABI) now catches C++
exceptions at the boundary and converts them to R errors instead of
unwinding across the C ABI.OMP_THREAD_LIMIT via the shared
tulpa_omp_team_size() helper (CRAN 2-core test compliance).Remotes dropped (tulpaMesh is on CRAN), Description expanded
with method references (DOIs), cph role added; new inst/WORDLIST,
inst/COPYRIGHTS, and a cleanup script; inst/CITATION reads
meta$Version. Vendored src/simp/ headers carry license lines.skip_on_cran()-gated; vignette
sampler chunks shrunk to CRAN-friendly iteration counts.integration = "grid_adaptive" now declines to the dense tensor BEFORE any
inner solve when the outer grid is small (fewer than control$adaptive_grid_min_cells
cells, default 48), instead of paying a coarse-seed pass first. On a small
tensor the coarse seed is already most of the grid, so there is no tail worth
skipping; the guard means the adaptive integrator can never be slower than the
dense tensor on a small grid. Measured on the real MOTIVATE 25 km shared-trend
fit (a 22-cell outer grid), grid_adaptive and grid are now the same fit at
the same wall-clock. The adaptive integrator earns its speedup only on large
outer grids whose hyperparameter posterior concentrates.integration = "grid_adaptive" for the multi-block
joint nested-Laplace driver, the low-dimensional companion to the CCD. It seeds
a coarse subsample of the hyperparameter tensor lattice (latent block axes and
phi axes together), floods outward from the posterior mode on the fine lattice,
and evaluates only the cells within a log-density cutoff of the peak. The kept
cells are a strict, uniform-weight subset of the dense tensor, so the posterior
matches the dense grid to that cutoff (each omitted cell carries dense-grid
weight < exp(-cutoff)) while evaluating far fewer inner solves when the
hyperparameter posterior concentrates -- the fine-grid regime where the mass
sits in a handful of cells. On a diffuse posterior it declines back to the dense
tensor after only the coarse seed, so it never costs accuracy. Tuned by
control$adaptive_grid_cutoff / adaptive_grid_stride / adaptive_grid_max_frac;
the kept-cell / dense / solve counts are returned as $adaptive_grid_info. The
main kernel call re-evaluates the selected cells with the full contract
(store_Q, phi tensor, tile warm start), so the fixed-effect and per-cell
posteriors are unchanged from the dense path. Fills the gap the CCD leaves at
1 <= d <= 3 latent axes, where the CCD mode-find is not worth it but a dense
tensor still pays a full inner solve for every near-zero-weight cell.mem_budget.h
(outer_thread_mem_budget(), outer_thread_cap()), single-sourced and
unit-tested.as_cholmod view and supernodal cholmod_common the inner solve uses)
and reads the factor's true L->xsize, via the new
SparseCholeskySolver::analyzed_factor_bytes(); the analysis is symbolic (no
numeric values, runs once in serial setup) and falls back to the old estimate
only if it produces no factor. The O(n_x) Newton scratch is now counted too.control$checkpoint to make the run resumable) rather than crashing.available_ram_bytes() (Windows ullAvailPhys,
Linux /proc/meminfo MemAvailable, macOS Mach free + inactive pages)
alongside the existing total_ram_bytes(), exposed to R for diagnostics as
cpp_available_ram_bytes() / cpp_total_ram_bytes().The coupled-cell scatter in the sparse joint nested-Laplace driver (the
occu_cover() hot loop, about 94% of runtime on a large fit) runs in parallel
when the outer hyperparameter grid is under-saturated: the tail of any grid,
small grids, or a many-core server where more outer threads are free than there
are active grid cells. Those idle team threads steal per-cell scatter chunks,
each accumulating into a private partial Hessian that is reduced in a fixed
chunk order, so no grid cell's scatter runs single-threaded while freed cores
sit idle. When the grid saturates the thread pool (the bulk of a large fit) the
scatter stays serial and byte-identical to before; only the under-saturated
cells chunk, and they stay within the thread-invariance tolerance (means agree
to about 1e-14 and are reproducible run to run via the fixed-order reduce). Set
TULPA_GRID_WORKSTEAL=0 to force the serial scatter for exact reproducibility.
The documented starting point for the PC prior on the copy coefficient
alpha is now c(U = 8.0, alpha = 0.01). A sweep over prior strengths on the
alpha-recovery fixture showed the earlier U = 2.0 recommendation over-shrinks
the copy coefficient below its truth and, through the alpha * sigma copy
axis, lifts the coupled donor amplitude sigma above its truth; U = 8.0
regularizes the tail without that bias. The sigma-pos-prior recovery test
asserts the retuned, measured behaviour.
SPDE field prediction standard errors (predict(se.fit = TRUE) with the field
included) are computed by a streaming C++ kernel (cpp_spde_field_se) that
factorizes the joint (beta, field) precision once and solves one query cell at
a time. The dense working set is O(p + n_mesh), independent of the number of
query cells, so the per-cell SE over a large prediction grid holds a bounded
amount of memory. Numerical results are identical.
The sparse joint nested-Laplace driver shares one per-worker resource pool
(Hessian builder, scatter cache, Newton scratch, arm specs, dense-basis
buffers) across the cheap-screen and full-solve passes, which run in disjoint
phases. This halves the pre-grid setup working set at a given
control$n_threads_outer; fits are numerically identical.
adjacency() gains a settable neighbourhood. order = k extends the grid /
raster stencil to the k-th ring: queen keeps every cell within Chebyshev
distance k ((2k+1)^2 - 1 neighbours: 8, 24, 48, ...), rook every cell
within Manhattan distance k (2k(k+1): 4, 12, 24, ...). The advanced
offsets argument takes a custom stencil (a two-column integer matrix or a
list of c(dx, dy) lattice offsets) for any anisotropic / off-axis
neighbourhood; because an ICAR / CAR field is undirected, an asymmetric
stencil is symmetrized to an undirected graph with a message. order = 1
(default) and offsets = NULL reproduce the previous queen / rook graphs
byte-for-byte.posterior_predict() draws replicated responses from
the posterior predictive (per-draw linear predictor -- fixed and random
effects jointly from the draws, or the Gaussian approximation on the
Laplace tier -- through new per-family sampling functions);
simulate.tulpa_fit() is the base-R alias, and pp_check() falls back to
generated replicates when the fit stores no y_rep.tulpa(weights =) scales each row's log-likelihood on
the non-spatial Laplace path and the log-posterior samplers; other backends
refuse loudly. A weight of 2 reproduces the duplicated-row fit exactly.y ~ s(x, k =, structure = "rw2"/"rw1") puts an RW
GMRF over the binned covariate and integrates its smoothness
hyperparameter through the nested-Laplace temporal kernels (single block
alone; the joint multi-block stack alongside an areal spatial field,
temporal field, or a second smoother). smooth_effects() extracts the
fitted smooth at the nodes.phi2 threads through the family registry, the
compiled kernels, and every fitter surface. The Student-t degrees of
freedom are now configurable (family = "t", phi2 = df; default 4).tulpa()'s phi is the residual VARIANCE for
gaussian/lognormal on every backend, as documented. Previously the
compiled kernels (Laplace, ModelData samplers) read it as the residual SD,
so mode = "laplace" and mode = "mala" fit different models at phi != 1;
the conversion now happens once at each R-to-kernel boundary. The direct
doors (fit_spde(), tulpa_nested_laplace_joint(), tulpa_sample_glmm())
keep their documented SD parameterization.lognormal (variance convention, front-door fittable on the
Laplace and sampler tiers) and tweedie (compound Poisson-gamma, phi2 =
power in (1, 2); Dunn-Smyth series density in R and the Laplace kernel,
pinned against tweedie::dtweedie).tulpa(family =
"multinomial" / "ordinal" / "ordinal_probit") routes to the Laplace
drivers; tulpa_ordinal() gains link = "probit" (pinned against
MASS::polr).bayes_R2() (per-draw R^2 with model-based residual
variance from new per-family variance functions); tulpa_reloo()
(PSIS-LOO with exact refits at observations above the Pareto k-hat
threshold, sharing the kfold refit machinery); tulpa_kfold() now threads
stored n_trials / weights per training partition (previously a stored
n_trials expression evaluated full-length against the subset data).log_marginal, exact for gaussian), so EP fits enter model comparison via
logLik().hyperparameter column: the
nested-Laplace mixture paths record per-draw hyperparameter log-priors at
draw-synthesis time and tulpa_powerscale_sensitivity() reweights them.predict(se.fit = TRUE) with the field included
propagates the joint (fixed-effect, field) posterior precision at the
fitted (range, sigma), including the cross term (integer-nu, no-RE fits).temporal_ar(time_idx, p =) generalizes temporal_ar2()
via the Levinson-Durbin PACF parameterization (always stationary) and the
exact Yule-Walker banded precision.LikelihoodSpec; the
engine keeps only the generic interval/truncated gaussian kernels.tulpa_ep(): Expectation Propagation for GLMs with a Gaussian coefficient
prior -- one Gaussian site per observation on the linear predictor, tilted
moments by Gauss-Hermite quadrature. Exact for a Gaussian likelihood;
matches marginal moments rather than mode curvature, so it is typically
more accurate than Laplace on skewed GLM likelihoods.tulpa_multinomial(): baseline-category (unordered K-class) multinomial
logistic regression as a Laplace fit driven by the validated native kernel
(cpp_multinomial_logit_terms) -- a joint Newton solve over the K-1
coefficient blocks, no engine change.tulpa_ordinal(): cumulative-logit / proportional-odds ordinal regression
(L-BFGS mode + Laplace); cutpoint ordering is guaranteed by the
log-increment reparameterization.tulpa_kfold(): refit-based K-fold cross-validation, the exact counterpart
to PSIS-LOO in tulpa_criteria() for fits whose outer Pareto k-hat flags
importance sampling as unreliable. Fixed-effect / GLMM fits only --
subsetting observations would break a spatial or temporal field's structure,
so those are rejected loudly.tulpa_powerscale_sensitivity(): power-scaling prior / likelihood
sensitivity (Kallioinen et al. 2024) by importance-reweighting the existing
draws through the native PSIS smoother -- no refits. The cumulative
Jensen-Shannon distance and its gradient reproduce the priorsense reference
implementation.temporal_ar2(): exact stationary AR(2) temporal GMRF as a user-defined
latent block (latent(temporal_ar2(time_idx))) -- pentadiagonal precision
from the Yule-Walker autocovariances, with the PACF parameterization mapping
the open square bijectively onto the stationarity triangle.gamma, inverse_gaussian, beta_binomial, and
Student-t (fixed df = 4, matching the C++ kStudentTDf) in both the
R-closure family ops and the C++ AD path, so they run under Laplace and NUTS
alike (test-family-ad-nuts.R).fit_spde() gains family = "gaussian" (continuous-field
geostatistics; phi is the observation-noise SD), and predict() kriges a
fitted Matern field to arbitrary newdata coordinates by re-projecting the
posterior-mean mesh-node field through the spec's mesh (include_field =
FALSE gives the fixed-effect / population prediction).fit_spde(), tulpa_re_cov_nested(), and tulpa_re_cov_gibbs() move
their perf / numerical knobs (method, n_grid, integration,
n_per_axis, span, n_draws, seed, max_iter, tol, n_threads,
diagnose_k, k_samples) into the single control = list(), matching
the front-door convention. Statistical arguments stay in the signature.modelAverage -> model_average,
postHocLM -> post_hoc_lm, spatialRange -> spatial_range,
temporalCorr -> temporal_corr. getSVCSamples() is removed -- use
svc(fit, summary = TRUE). The internal formula helpers
find_latent_terms, no_latent_terms, and parse_bar_term are no longer
exported.R/nested_laplace_moments.R, and the Polya-Gamma Gibbs dispatch plus the
tulpa_gibbs() front door to R/fit_gibbs.R.cpp_laplace_fit_spatial() and cpp_laplace_fit_bym2() gain a force_sparse
argument (0 = size threshold, 1 = force sparse, -1 = force dense) threaded to
the shared Newton solver. It lets the same problem run through both
factorization paths for a byte-level dense-vs-sparse equivalence gate on the
single-response path, the analogue of the joint path's force_sparse control.
Default behaviour (0) is unchanged.test-sparse-cholesky.R), a tulpa_pit()
calibration check (Uniform-under-correct-model, with a powered misspecification
counter-case, test-pit-calibration.R), and a stored-draws reference for the
default leapfrog integrator so a change to the stepper fails loudly rather than
silently reproducing its own new draws (test-integrator.R,
tools/gen_leapfrog_ref.R).tulpa_integrator("adaptive2") / "adaptive3" cap the warmup-end curvature
estimate at DENSE_MAX_PARAMS (200) parameters. Above that bound the
step-adapted coefficient resolves at the well-adapted operating point
(nu_max = epsilon, omega_max = 1) rather than building the dense p x p
finite-difference Hessian, matching the cap the dense mass matrix already
applies. A high-dimensional latent field (thousands of BYM2 / ICAR cells)
selecting an adaptive integrator no longer allocates an O(p^2) Hessian or runs
p + 1 gradient sweeps at warmup end. Default leapfrog, the fixed schemes, and
any model at or below the bound are unchanged.tulpa_integrator("adaptive2") and "adaptive3" wire the SIMP step-adapted
minimum-error integrators into NUTS. Each chain resolves its multistage
coefficient at the end of warmup for its own operating point: the coefficient
that minimizes the worst-case energy error over the band of dimensionless
steps (0, nu_max] the chain actually takes. nu_max = omega_max * eps
follows from the adapted mass matrix and the local posterior curvature
(omega_max is the square root of the largest eigenvalue of M^{-1} times the
curvature, found by power iteration on a warmup-end finite-difference Hessian),
so the integrator is tuned to the target rather than to a fixed compromise --
the nested-approximation-informs-the-sampler synthesis. The coefficient is
per-chain (carried on the NUTS workspace, not a shared global); warmup runs a
fixed placeholder of the same stage family so the dual-averaged step size
transfers. Default leapfrog and the fixed schemes are byte-for-byte unchanged.
tulpa_integrator("mts", mts_substeps = ) adds a multiple-time-stepping
(RESPA / Verlet-I) integrator to NUTS. The trajectory leaf splits the force
into a stiff but cheap prior part (the Gaussian latent structure, taken with
mts_substeps inner leapfrog substeps) and a smooth but expensive likelihood
part (one full gradient per leaf, as leapfrog pays), so a larger outer step
handles the stiff prior without evaluating the likelihood at the inner rate.
The split reuses the existing skip_obs_loop decomposition: a new prior-only
gradient path (arena-AD or central differences, folding in any model-package
prior) supplies the fast force, and grad_full - grad_prior gives the slow
(likelihood) force -- exact when the full and prior gradients are additive,
which holds unless a model-package gradient hook alters the full gradient
non-additively. The leaf stays symplectic and time reversible, so the U-turn /
tree machinery is unchanged. Helps most when the latent field is stiff relative
to a comparatively flat likelihood.
Refreshed the vendored SIMP snapshot, which adds the step-adapted
minimum-error integrators (an exact harmonic energy-error analysis picks the
multistage coefficient for a target's step band, from omega_max * eps) and
the parametric two_stage / three_stage constructors. These ship in
src/simp/; the fixed tulpa_integrator() schemes are unchanged. The
integrator decls now include only simp/scheme.h, so the Eigen-heavy headers
do not enter every translation unit.
src/simp/ (snapshot via
vendor_simp.sh) instead of pulled in through LinkingTo: SIMP. tulpa builds
self-contained -- no dependency on a non-CRAN package and no
Additional_repositories -- while SIMP (gcol33/SIMP) stays the upstream
development home for the integrator core. Re-run vendor_simp.sh to refresh
the snapshot after updating SIMP. No behaviour change: tulpa_integrator()
and the schemes are identical.tulpa_integrator("minerror2") selects the two-stage minimum-error
integrator from SIMP 0.2.0. Its coefficient cancels the leading energy error
on a Gaussian target, so near the mass-adapted optimum it conserves energy
well and adapts to larger step sizes without the stability cliff of the
high-order Yoshida schemes. It recovers with zero divergences on the
linear-Gaussian recovery test and is the recommended advanced integrator for
near-Gaussian posteriors. Requires SIMP (>= 0.2.0).LinkingTo: SIMP). Both leapfrog steppers (the in-place
NUTS step and the fixed-trajectory HMC step) walk a SIMP scheme's op
sequence with tulpa's own fused mass-matrix kernels, so the integrator
identity has a single source of truth while the hot path keeps its
specialised drift kernels. The default, leapfrog, is byte-identical to the
previous step (verified against the bit-for-bit chain checkpoint and the
existing NUTS recovery / reproducibility tests).tulpa_integrator() selects the integrator process-wide: "leapfrog"
(default) or the higher-order "yoshida4" / "yoshida6" / "yoshida8",
generated from leapfrog by SIMP's triple-jump composition. "yoshida4"
samples reliably; the higher orders are experimental for NUTS (a sharp
step-size stability threshold interacts poorly with dual-averaging
adaptation). See ?tulpa_integrator and test-integrator.R.tulpa_pit() runs in C++ (cpp_tulpa_pit), and the leave-one-out PIT
weighting is exposed as cpp_psis_loo_pit -- per-observation PSIS leave-one-out
weights (reusing the deterministic PSIS core) applied to predictive-CDF limits,
with the PSIS columns parallelised and the single uniform jitter drawn in index
order. Both draw from R's RNG stream in the same order as their former R
bodies, so results are byte-identical under a fixed seed (test-pit-cpp.R).tulpa_psis() runs its deterministic core -- the Zhang-Stephens generalized-
Pareto tail fit and the Pareto smoothing of the upper-tail log weights -- in a
C++ kernel (cpp_tulpa_psis). The R helpers .tulpa_gpd_fit() /
.tulpa_qgpd() are kept as the reference oracle (test-psis-cpp.R). The tail
size (with its expert-control cap and warning) stays in R, and the bootstrap
k-uncertainty still resamples with R's RNG, so results are unchanged and
reproducible; each per-observation LOO fit and each bootstrap refit is now the
C++ path. Byte-close to the former R body (~1e-12).tulpa_nested_laplace_joint() gains prior_phi, a regularizing hyperprior on
the per-arm dispersion axes declared through phi_grid (a Beta precision, a
negbin size, a Gaussian residual SD). Mirrors prior_sigma / prior_alpha:
NULL (flat over the phi grid, default), list("pc.prec", c(U, alpha)), or
list("half_normal", scale). A single spec re-weights every phi_<arm> axis
by its density at the kernel-call boundary, so refinement and Pareto-k passes
see the regularized posterior; with no phi_grid it is a no-op. Threads
through the single- and multi-block paths (gcol33/tulpa#139).New control$local_ccd refines a multi-block tensor outer grid with local
central-composite-design node clouds (R/nested_laplace_joint_ccd_local.R):
a few high-weight, mutually non-adjacent interior cells are each replaced by a
small curvature-aware CCD design, so a coarse base grid resolves the
sharply-peaked hyperparameter directions without the k^d tensor blow-up. The
local curvature is a diagonal finite difference of the outer log-marginal over
each cell's own grid neighbours -- no mode-find, only the off-centre nodes are
new inner solves, warm-started from the cell's mode. Refined cells carry
partition-of-unity design weights, so the total integration weight is conserved
exactly (no double-count); the design scale is shrunk per cell so the cloud
fits the cell's Voronoi box (the local-Gaussian mass beyond it belongs to the
neighbouring cells, which carry their own mass). Engages only on the tensor
path at >= 4 transformable latent axes with no active phi_grid -- the
regime where a uniformly fine tensor is k^d-expensive and grid densification
is the wrong tool; below it, the tensor grid is already dense and boundary /
interior grid refinement covers a too-narrow grid. The applied refinement is
summarised on the result as $local_ccd_info.
control$k_refine gains a "ccd" rung alongside "grid". Under
k_quality = "ok" / "good", a bad outer Pareto-k-hat now escalates by
refining high-weight cells with local CCD node clouds (forcing a tensor base so
the curvature stencil has axis neighbours), the right response when the grid is
too coarse to resolve a sharp direction rather than too narrow at the boundary.
Each round refines more cells; the verdict never silently downgrades and reports
when local CCD finds no peaked interior cell to act on. Recovery, weight
conservation, and the escalation path are pinned by
test-nested-laplace-joint-ccd-local.R.
Joint nested-Laplace CCD outer integration is now robust to a sharply-peaked,
ill-conditioned hyperparameter posterior -- the shape a joint occu_cover fit
with an observation-arm random effect produces, where a narrow field-SD axis
sits at a grid edge alongside a wide, weakly-identified axis. The CCD mode-find
(.joint_ccd_grid) keeps the grid-median seed with a fixed finite-difference
step as the default and, when that declines, falls forward to a rescue that
warm-starts at the best latent grid cell (.joint_ccd_grid_seed evaluates the
latent Cartesian grid in one batched call and takes the joint argmax) and uses
a per-axis step calibrated to the local curvature (.joint_ccd_calibrate_step
spans ~1 posterior sd per axis: small on a sharp axis, wide on a weakly-curved
one, since one fixed step cannot resolve both). A ridge-safe coordinate-ascent
seed (.joint_ccd_pilot_seed) is the fallback when the latent grid is too large
to evaluate. Well-conditioned and sigma-alpha-ridge fits are unchanged (they
engage on the default path); previously a sharply-peaked posterior declined to
the full tensor grid. A recovery test pins the rescue.
The CCD mode-find's backtracking line search now evaluates all max_halve
step lengths in one batched call, so the candidates run across the outer-grid
threads rather than one full-field inner solve at a time. On a large field (an
expensive inner Laplace) the line search no longer serialises the mode-find;
the accepted step is the same the sequential backtrack would take.
adjacency() front door builds the symmetric graph that spatial() and
spatial_car() consume, so areal models no longer need a hand-coded
adjacency matrix. It is one generic that dispatches on the layout: a
data.frame of cell centroids (queen / rook contiguity over the inferred
lattice, no extra dependency), an sf polygon layer (shared-boundary
contiguity via DE-9IM; queen = point or edge, rook = edge), and a raster
(a terra SpatRaster or a stars object) whose non-NA cells become the
nodes. It returns a printable tulpa_adjacency object carrying
the sparse 0/1 matrix ($adjacency), the per-node cell identifier ($ids),
the inferred cell size, and the node count -- the model still receives an
explicit graph = g$adjacency, so the graph stays inspectable before
fitting; nothing is guessed from coordinates silently.node_index(graph, ids) maps original cell identifiers to 1-based node
indices by key, for remapping observation data (many rows per cell, different
row order) onto the graph without assuming row alignment.check_adjacency() validates a hand-built matrix in one pass -- square,
symmetric, zero diagonal, 0/1 valued, isolated nodes, and matching unique
ids -- and adjacency() runs the same checks on the graphs it constructs.spatial(), spatial_car(), and spatial_bym2() now accept a
tulpa_adjacency object directly (unwrapping its $adjacency), in addition
to a bare matrix.CellCouplingSpec can now declare, via the
new dense_cross_pairs(n_coupled, rank1_self_supported) virtual, which
(kk, ll) arm-pair cross-Hessian slabs it actually writes densely. The
single-response cell loop allocates a dense rc_kk * rc_ll slab only for those
pairs; every other pair keeps a nullptr buffer (the scatter already guards
null). This bounds a cell with J observations on a self-coupled arm to O(J)
rather than O(J^2): a self block emitted through the rank-1 self-cross
descriptor, or a cross a factorising likelihood never writes, no longer
reserves a J x J slab. The default returns every pair, so a spec that does
not override it is unchanged; the change is numerically inert for the
occu_cover spec (the dropped slabs were allocated, zeroed, and never written
even before). Removes a std::bad_alloc on grids with a very high-visit cell
(e.g. an all-undetected cell with tens of thousands of plots).k_quality escalation is now driven by adaptive integration-grid refinement
rather than diagnostic-draw doubling (gcol33/tulpa#131). When the outer
Pareto-k is bad the integration grid does not faithfully represent the
hyperparameter posterior, so each escalation round now REFINES THE GRID
(adaptive_grid boundary extension / interior densification, one more pass per
round) and re-diagnoses, driven by the bad k, until the band is reached or the
round budget is spent. Doubling diagnose_draws only re-scores the same grid,
so it is no longer the escalation lever; diagnose_draws stays the separate
knob that sharpens the k ESTIMATE (still auto-raised for "ok" / "good" so
the bootstrap CI resolves).control$k_refine value "mixture" is renamed to "grid" (it refines the
integration grid), and the default for k_quality = "ok" / "good" changes
from "none" to "grid": asking for a reliable band now chases it by
refining. k_refine = "none" opts out (the band is reported but not chased)..K_DIAG_MM_MAX is raised from 3 (a cost throttle added in #127) to 8,
a runaway-loop backstop rather than a budget. The loop self-limits well below the
cap: it still stops as soon as the k-hat reaches the usable band, and now also
stops once a refined pass no longer improves on the proposal it was estimated
from, so the extra passes are spent only on a stubborn k that is still above the
usable band and still falling. A fit that already fits, or one whose moment
matching has plateaued, pays the same as before.control$k_quality now climbs the reliability ladder (gcol33/tulpa#131). When
an "ok" / "good" target is not confidently reached on the first fit, the
engine escalates: it doubles diagnose_draws each round and, with the new
control$k_refine = "mixture", refines the integration grid (adaptive_grid)
on the final round when more draws alone do not resolve the band, re-fitting and
re-diagnosing up to control$k_max_rounds (default 2) times. k_quality_rounds
reports how many re-fits were used, and the verdict stays honest -- the request
is a target, not a promise. The k_quality verdict (k_quality_requested /
reached / best / reason / rounds) is now attached for BOTH the single-
and multi-block paths (previously single-block only). The single fit is factored
into an internal engine driven by the escalation front door, with no change to
the fit itself for the default k_quality = "report".control$k_quality reliability front door for the joint nested-Laplace
outer Pareto-k (gcol33/tulpa#129). A single statement of the reliability the fit
should report: "report" (default) computes the diagnostic and reports the
achieved band; "ok" / "good" name a target band (the k-hat confidently
usable, resp. good) and raise the default diagnose_draws (to 800L / 2000L,
unless diagnose_draws / k_samples is set) so the bootstrap CI can resolve
it; "none" disables the diagnostic. The fit carries an honest verdict --
k_quality_requested, k_quality_reached, k_quality_best, k_quality_reason
-- and never silently downgrades: if the requested band is not confidently met
it reports the band actually reached and why. The adaptive draw-escalation loop
and the integration-refinement rung (k_refine) are tracked in
gcol33/tulpa#131. The reliability vignette covers the new front door.min(1 - 1/log10(S), 0.7) for S importance draws
(Vehtari et al. 2024; matches loo's ps_khat_threshold): about 0.565 at the
small-S end (S = 200), reaching the fixed 0.7 cap only past S ~ 2154. The
good cut stays at 0.5. pareto_k_band_confident now tests the bootstrap CI
against c(0.5, min(1 - 1/log10(S), 0.7)) at the realised draw count, so a
k-hat near the upper band at a modest diagnose_draws is correctly read as
not-yet-usable rather than acceptable. control$k_conf_bands defaults to NULL
(the size-dependent bands); pass a strictly-increasing vector (e.g.
c(0.5, 0.7)) to fix the boundaries. New vignette section and the helpers
.ps_khat_threshold / .ps_conf_bands.control$diagnose_draws
importance draws, and the k-hat's sampling uncertainty is estimated by
bootstrapping its raw importance log-ratios (control$k_bootstrap replicates,
re-fitting the GPD tail; no new inner solves). New control / output:diagnose_draws (default 500L; legacy k_samples accepted as an alias) is
the precision knob -- a tighter k needs MORE actual tail ratios, so raise it,
NOT k_bootstrap. The bootstrap only quantifies how unstable the current
estimate is; it cannot create tail information.pareto_k_se_boot, pareto_k_ci_low / pareto_k_ci_high (2.5% / 97.5%
bootstrap quantiles), pareto_k_se_formula (the closed-form GPD-shape MLE
asymptotic SE (1 + k) / sqrt(M), a cross-check), and
pareto_k_band_confident (TRUE iff the bootstrap CI lies within one
reliability band).k_tail_points (default NULL = the automatic PSIS rule
ceil(min(0.2 N, 3 sqrt(N)))) is an EXPERT tail-threshold control, capped at
the 20%-of-draws ceiling with a warning; the used / requested counts are
reported in pareto_k_tail_points / pareto_k_tail_points_requested.k_conf_bands (default c(0.5, 0.7)) sets the reliability-band boundaries,
with intervals (-Inf, 0.5] (0.5, 0.7] (0.7, Inf).diagnose_cost_ratio (and diagnose_draws) attached at the top level: the
diagnostic's wall-clock cost relative to the fit it certifies.
The per-arm k (diagnose_k = "by_arm") carries the matching per-arm fields. A
borderline k is reported with its honest (wide) CI rather than chased to false
precision; for a tighter estimate, raise diagnose_draws. tulpa_psis() gains
a tail_points argument and returns tail_len.reliability-pareto-k: what the outer Pareto-k certifies (the
integration, not the posterior), the reliability bands and their
draw-count-dependent usable boundary min(1 - 1/log10(S), 0.7), the bootstrap
band-confidence flag, and the reliability ladder from reporting to debiasing.B, so it
cannot signal convergence. The batched diagnostic (control$k_batches > 1) now
reports, alongside the median pareto_k:pareto_k_mcse = sd(batch k-hats) / sqrt(B), the Monte Carlo standard
error of the estimate (shrinks as 1/sqrt(B));pareto_k_band_confident = TRUE iff pareto_k +/- 2 * pareto_k_mcse lies
within one reliability band (does not cross 0.5 or 0.7);pareto_k_n_batches and the secondary observed pareto_k_lo / pareto_k_hi
range (kept as QA fields).
The per-arm k (diagnose_k = "by_arm") carries the same fields
(pareto_k_by_arm_mcse, pareto_k_by_arm_band_confident), over the same number
of batches the joint loop settled on.control$k_adapt = TRUE: starting at k_batches
batches (defaulting to 4L rather than the off sentinel), the diagnostic adds
batches until pareto_k_band_confident becomes TRUE or the
control$k_batches_max cap (default 20L) is reached. A fit on the wrong side
of a band boundary keeps sampling until the good/ok/unreliable verdict resolves;
a fit whose true k sits ON a boundary (whose interval always straddles) stops at
the cap with the honest band_confident = FALSE, so the cap is both a cost
bound and the correct classification for an on-the-line fit. The seed pool is
drawn for the cap, so an adaptive run is a reproducible prefix of the full-cap
run. k_batches must be >= 2 when k_adapt = TRUE (an MCSE needs at least
two batches). The 1/sqrt(B) rule reduces the seed-to-seed variance, not the
GPD k-hat's small-sample bias (controlled by k_samples, kept >= 200).< 0.5 good, 0.5-0.7 ok, >= 0.7 unreliable). The new
control$k_batches (default 1L = OFF, byte-identical to the prior
single-value behaviour and cost) evaluates the CHOSEN proposal's k over that
many independent importance batches and reports pareto_k as the MEDIAN plus
the observed pareto_k_lo / pareto_k_hi range and pareto_k_n_batches; the
reliability band is classified off the median. The opt-in per-arm k
(gcol33/tulpa#120) is batched the same way (pareto_k_by_arm_lo /
pareto_k_by_arm_hi). The proposal SELECTION (single Gaussian vs grid-mixture,
gcol33/tulpa#121) is made once on the canonical pass so every batch scores the
SAME proposal (no per-batch source flip); per-batch seeds are drawn up front
from the restored RNG state, so the diagnostic stays reproducible and the fit's
draws are bit-for-bit unchanged. The spread is the Monte Carlo uncertainty of
the PSIS k-hat across independent importance samples -- NOT a posterior credible
interval and NOT a coverage-calibrated CI; the per-cell estimates and
coefficients do not move. Cost is k_batches times the scoring, gated behind
the already opt-in / slow diagnostic; a handful of batches (5-10) gives an
honest min/max range.New built-in truncated_gaussian family (gcol33/tulpa#122): an
upper-truncated Gaussian latent, the bounded-support sibling of the
lognormal arm. On the natural scale it is an upper-truncated lognormal --
a positive response known to lie below a ceiling, modelled as a Gaussian on
the log-response conditioned on log y <= u for a per-row bound u
(+Inf => no truncation). The likelihood is the Gaussian density divided by
the retained mass Phi((u - eta)/sigma), the continuous-density counterpart
of the interval_gaussian (ordinal) family. It is log-concave in eta so
the inner Newton needs no Fisher fallback, and reduces exactly to the
gaussian arm as u -> +Inf. Wired through the joint nested-Laplace path
via a per-arm trunc_upper ceiling (src/laplace_family_link.h,
src/laplace_builtin_family_spec.h, R/nested_laplace_joint_helpers.R).
FD gradient/Hessian, deep-truncation stability, the lognormal reduction, and
truncated-normal moment identities are checked in
test-truncated-gaussian.R.
New baseline-category multinomial-logit kernel (src/multinomial_logit.h,
gcol33/tulpaObs#106): a nominal (unordered) K-class likelihood with K-1
coupled linear predictors sharing the softmax denominator. The per-observation
negative Hessian is the full (K-1)x(K-1) multinomial information
(diag(p) - p p'), positive semidefinite for any eta, so the inner Newton
needs no Fisher fallback; the softmax is formed overflow-safe. This is the
engine primitive backing tulpaObs's occu_categorical() positive arm. FD
gradient/Hessian, the PSD data-free information identity, and overflow safety
are checked in test-multinomial-logit.R.
grid_moment path now also scores the proposal the engine
actually samples, a defensive mixture of local Gaussian bumps at the grid cells
mixed by the grid weights (proposal_source = "grid_mixture"); each bump's
per-axis SD is getOption("tulpa.kdiag.mix_bw", 0.5) times the largest grid gap
on that axis. Because that mixture is confined to the grid, it is adopted only
when the grid actually covers the posterior (the grid-moment Gaussian's
importance weight stays inside the mixture's coverage hull) and it lowers the
k-hat: a target tail beyond the grid keeps the single Gaussian's higher k so the
grid-width deficiency is still flagged, a near-collapsed grid keeps the
moment-matched Gaussian, and no fit that already read usable can regress. A true
delta collapse still uses the finite-difference mode-Hessian and a supplied CCD
mode-Hessian proposal still uses the single Gaussian, both unchanged. The fit,
its per-cell estimates and coefficients are untouched; only the reliability
diagnostic changes. On a real EVA-scale occu_cover fit whose occupancy field-SD
posterior is right-skewed the reported k-hat goes from a seed lottery (0.10 to
0.81, peaks flagged unreliable) to a stable usable band (max ~0.47 across seeds,
median ~0.16) with a roughly two-fold higher importance-sampling effective size.tulpa_criteria() gains a group argument that sets the LOO unit explicitly
(gcol33/tulpa#118). A column of log_lik is the leave-one-out fold; with
group = NULL (the default) the result is byte-identical to before
(leave-one-row-out, e.g. per plot / per visit). When supplied, the per-draw
pointwise log-likelihoods are summed within group to a [n_draws x n_groups]
matrix before PSIS, so each fold is a whole group (leave-one-group-out
cross-validation, LOGO-CV) -- e.g. leave out a whole cell rather than one of
its rows. The grouping streams over the (possibly EVA-scale) input once and
never materialises it; lppd, p_waic, elpd_loo, cpo and pareto_k are
all computed on the grouped matrix and the standard-error multipliers follow
the fold count. DIC is a plug-in deviance over all observations and is
unaffected. The result reports n_groups and prints the fold count, and the
pointwise data frame is keyed by group.
Joint nested-Laplace outer Pareto-k: opt-in per-arm reporting
(gcol33/tulpa#120). control$diagnose_k = "by_arm" computes, in addition to
the single joint k over the whole hyperparameter posterior, a k-hat restricted
to each arm's hyperparameter axes (the other arms held at their posterior
mean), so a tail-heavy joint k can be localised to one arm rather than reported
as one pooled number. Each axis is attributed to the arm(s) whose linear
predictor it enters -- a latent block's axes to the arms the block loads on, a
copy coefficient alpha to the recipient arm, a phi_<arm> dispersion axis to
that arm -- reusing the same proposal-build + moment-matching + PSIS path as
the joint k over the arm's axis subspace. Reported in pareto_k_by_arm (named
by arm) with pareto_k_by_arm_is_ess / pareto_k_by_arm_scope, surfaced in
diagnostic_summary(). OFF by default: the joint k stays the default behaviour
and cost, and is bit-for-bit unchanged by the opt-in (it is scored first, and
the diagnostic is RNG-restored). Defined for the multi-block layout with two or
more arms; the single-block shared-field layout declines rather than
mis-attribute its axes.
New built-in interval_gaussian family: an interval-censored Gaussian latent
(an ordered probit with KNOWN thresholds). The latent value is
Normal(eta, sigma^2) and the observation records only that it fell in the
half-open interval (lower, upper] on the linear-predictor scale, with
-Inf / +Inf the open outer classes. The log-density is the class
probability MASS, log(Phi((upper - eta)/sigma) - Phi((lower - eta)/sigma))
-- a genuine PMF over classes with no change-of-variable Jacobian, so the
score is comparable across arms. The mass is differenced in the accurate tail
to avoid catastrophic cancellation, and P(eta) is log-concave so the
analytic -d2 logP/d eta2 >= 0 needs no Fisher fallback. The family is read
through tulpa_nested_laplace_joint() (family = "interval_gaussian", with
per-arm lower / upper in place of the point response y); it backs
tulpaObs's cover(positive = "ordinal") Braun-Blanquet cover arm. The kernel
is FD-gradient tested via the internal cpp_interval_gaussian_terms()
(test-interval-gaussian.R).
Joint nested-Laplace outer Pareto-k: the importance-sampling proposal now
carries an optional moment-matching refinement (gcol33/tulpa#119, after
Paananen, Piironen, Burkner & Vehtari 2021). When the integration grid is
sharply concentrated the node-covariance proposal can mis-scale -- too wide
scatters draws to extreme hyperparameters where the inner Laplace log-marginal
inflates, too narrow leaves the target tail uncovered -- so the k-hat reads
unreliable even on a fine fit. The proposal is now re-estimated from the
PSIS-smoothed importance-weighted moments of its own draws and re-scored, up
to a few passes, keeping the lowest-k-hat proposal and stopping early once the
k-hat reaches the usable band (<= 0.7); the smoothed weights bound any single
draw's influence so a sharp posterior is matched in a couple of passes
(proposal_source = "moment_matched"). The fit's RNG is restored so the
posterior draws are bit-for-bit unchanged. The collapsed-grid FD mode-Hessian
rescue (gcol33/tulpa#116, #117) is now reserved for a TRUE delta collapse (no
grid-weighted spread on ANY axis); a partial collapse where some axis still
carries weighted spread keeps the grid-moment proposal and lets moment matching
refine it, rather than over-widening the proposal with the local mode curvature
of a non-Gaussian outer marginal.
Joint nested-Laplace outer Pareto-k: the collapsed-grid mode-Hessian rescue
(gcol33/tulpa#116) now engages when a hyperparameter axis is pinned
(gcol33/tulpa#117). When the integration grid concentrates (ess_grid <= d)
the diagnostic reconstructs a Laplace-at-mode Gaussian proposal from a
finite-difference Hessian of the outer target. Previously it differenced over
all d axes, so a pinned axis (zero weighted variance: a copy() alpha
fixed at 0, or a one-point dispersion grid) made the FD Hessian singular, the
conditioning guard rejected it, and the k fell back to the grid_moment
proposal #116 was meant to supersede -- which can be spuriously high and label
an otherwise-fine fit "unreliable". .joint_pareto_mode_cov() now restricts
the stencil to the varying axes (the same var_tol set the proposal build
uses, factored into the shared .joint_pareto_vary_axes()) and embeds the
inverse curvature block-diagonally, pinning the zero-variance axes at u_hat.
Excluding a pinned axis from the curvature is exact, not an approximation.
Affected any joint occu_cover fit with an uncoupled (no-copy()) cover arm
plus a concentrated hyperparameter posterior.
Per-cell warm start for the outer Pareto-k re-solves (gcol33/tulpa#118
follow-up). Each importance draw's inner solve now starts from the converged
latent mode of its NEAREST integration cell (.joint_nearest_grid_mode,
threaded through cpp_nested_laplace_joint_multi(x_init_per_cell=) and the
grid driver) instead of the single broadcast modal mode. Unlike the 0.0.42
near-neighbour re-order -- which only helps the serial chain -- this also warms
the PARALLEL pilot-mode path, so a threaded diagnostic (n.threads.outer > 1)
gets it too: on a real EVA occu_cover fit (402 cells, beta) the per-cell warm
cuts the parallel diagnostic a further ~1.5x on top of threading. The k-hat is
byte-stable (each draw converges to the same mode regardless of start;
validated == the broadcast-mode path and loo::psis). Knob
tulpa.kdiag.percell (default TRUE; falls back to the re-order, then the
broadcast mode, when grid modes are unavailable).
.K_DIAG_REFRESH):
the diagnostic's inner re-solves run with inner_refresh = 4, so off-factor
steps re-apply the cached factor to a refreshed gradient and scatter
grad_only (skipping the curvature fill). The final mode-pass always
re-factorizes with the true Hessian, so the converged log-marginal -- and
thus the k-hat -- is unchanged; only the path to the mode uses a stale
curvature, which the diagnostic (no per-draw SEs) does not need..joint_is_chain_order / .joint_is_solve_reordered): the serial
outer-grid driver warm-starts each cell from the previous cell's converged
mode, so the random-order proposal draws were each starting from a
random-neighbour mode (8-16 inner-Newton steps/draw). The batch is re-ordered
into a standardised near-neighbour chain seeded at the modal cell
(.joint_modal_theta), so each draw corrects only the small drift from its
neighbour; the result is un-permuted before the PSIS layer, which is
unaffected. The per-cell parallel path warm-starts from the pilot mode, so
the order is then immaterial..K_DIAG_TOL = 1e-4): a
large share of the per-draw steps was intrinsic convergence to the fit's
own tol (~1e-6), which the diagnostic does not need -- the Laplace
log-marginal error from stopping at gradient norm t is O(t^2),
immaterial to the tail-shape k-hat. Never tighter than the fit's tol.
Combined, the three cut the diagnostic 3-4x on the beta arm with the k-hat
byte-stable (validated vs loo::psis / posterior::pareto_khat on real EVA
occu_cover importance ratios: identical to 4 decimals). Each is overridable
via tulpa.kdiag.refresh / tulpa.kdiag.tol / tulpa.kdiag.reorder for the
byte-for-byte exact diagnostic, and tulpa.kdiag.capture exposes the importance
log-ratios for an external cross-check.pareto_k diagnostic built its importance
proposal from the grid-weighted covariance of the integration nodes; when the
hyperparameter posterior is sharp the grid concentrates on ~1 cell, the
grid-weighted covariance is then driven by negligible-weight far cells, and the
too-narrow proposal yields a spurious high k-hat even though the fit is fine.
The proposal is now built from a mode Hessian, on both integration paths.u_hat, L_scale) is captured
and spliced into the Pareto-k driver over the axes it spans, block-diagonal
with the grid-weighted spread on the independently tensor-crossed phi axes.<= d) and no
CCD proposal exists, the Laplace-at-mode covariance is reconstructed from a
finite-difference Hessian of the outer target at the modal cell (reusing the
CCD stencil / conditioning helpers). Degenerate or ridged curvature falls
back gracefully to the grid estimate.pareto_k_proposal_source in
{"mode_hessian", "grid_moment", NA} (documented on
tulpa_nested_laplace_joint) flags which regime a fit is in.alpha) there is no PD Hessian along the ridge, so the FD Hessian
declines and the path reports a grid_moment k. A ridge gives a wide
proposal and hence a low k, so the spurious-high-k failure does not arise
there; a principled diagnostic for non-Gaussian hyperparameter posteriors
is left as future work.grid_moment k = 0.895
(unreliable) -> mode_hessian k = 0.543 (reliable).Outer-grid progress reporter (tulpa_progress::GridProgress, the
nested-Laplace grid and parallel NUTS sampler): two fixes for long detached /
redirected runs (gcol33/tulpa#115).
ETA >=) rather than a point
estimate (ETA ~), and the printed s/cells tracks the running average.
At width 1 (serial) this reduces to the plain extrapolation.1/N) because every parallel tick suppressed
the console (worker threads must not touch the R print API), leaving only
the heartbeat file advancing. The master thread (thread 0, the R main
thread) now emits the newline-terminated line from inside the parallel
region on its own throttle clock, so a redirected log shows the grid
advancing cell by cell. Worker threads still only update the counter and
heartbeat file; the heartbeat-file wire format
(<done> <total> <elapsed_s> <eta_s>) is unchanged, so existing readers
keep parsing.Joint nested-Laplace multi-block driver: two random-effect capabilities for random slopes on an observation arm (gcol33/tulpa#114), so a downstream consumer can fit correlated and uncorrelated slopes, not just intercepts.
svc_weight) on the iid block,
mirroring the areal / temporal SVC path: the field's contribution to arm
k row i is row-scaled by svc_weight[[k]][i]
(eta_i += svc_weight[[k]][i] * sigma * u[obs_idx_i]). An uncorrelated
slope (0 + x | g) / (x || g) is then one weighted iid block per
coefficient, each with its own sigma axis. Unset svc_weight is
byte-identical to the plain random-intercept iid block.type = "miid": the non-spatial sibling of
mcar with Q = I. Per group g a coefficient vector
b_g ~ N(0, Sigma) (block dim n_fields = 1 + n_slopes), with the free
cross-coefficient Sigma integrated over the same p(p+1)/2 log-Cholesky
outer-grid axes as mcar. The precision Sigma^-1 (x) I is full rank, so
(unlike mcar) there is no sum-to-zero pinning and the normalizer carries
the full n (n log|Sigma^-1|). This expresses a correlated random slope
(1 + x | g). Copy (alpha) semantics compose as for mcar. A miid
block with n_fields = 1 is the centered counterpart of the scalar iid
block (Laplace-invariant). Direct-algebra prior assembly
(Sigma^-1 (x) I, gradient, log-prior) and parameter recovery are tested
(test-miid-prior.R, test-miid-recovery.R). The shared MCAR/MIID obs
scatter and copy arm_scale are single-sourced in mcar_block_factory.h.control$k_threads: outer-thread width for the joint nested-Laplace fit's
Pareto-k diagnostic importance batch. The k_samples re-solves run after the
grid (every core free), each solved single-threaded once the batch saturates
the pool, so widening the pool is a bit-identical wall-clock speedup with an
unchanged k-hat. NULL (default) follows the fit's own thread grant, "auto"
uses the physical performance-core count (capped at 2 under R CMD check), and
an integer pins the width.
laplace_diagnostics(): a front-door diagnostic for deterministic
(i.i.d.-draw) nested-Laplace fits, the class mcmc_diagnostics() declines to
treat as MCMC. It returns a per-parameter table (posterior mean / sd, plus the
i.i.d.-draw bulk / tail effective sample size and split-Rhat of the draws,
labelled as Monte-Carlo diagnostics rather than chain mixing) and attaches the
reliability headline as attributes and a summary row: the PSIS Pareto-k-hat
of the outer hyperparameter integration scored against the exact inner-Laplace
marginal (Vehtari et al. 2024; Yao et al. 2018), and the grid quadrature
effective sample size ess_grid = 1 / sum(w_k^2). mcmc_diagnostics() now
dispatches an i.i.d.-draw fit to laplace_diagnostics().
Pareto-k diagnostic on the joint engine: the importance-sampling proposal is
now built on the grid axes that actually vary, so an outer grid that pins an
axis (for example alpha.grid = 0) no longer yields a singular chol(Su) that
silently skips the diagnostic; the k-hat is computed on the remaining axes.
Speed: the per-sample marginal re-solves of the Pareto-k diagnostic refit now
honour n_threads_outer (previously hardcoded to one thread), giving a
5.8-7.7x speedup on diagnose.k = TRUE fits with k-hat unchanged.
tulpaMesh dependency floor is raised to tulpaMesh (>= 0.1.3),
locking it to the current tulpaMesh release. The Remotes install reference was
already gcol33/tulpaMesh@v0.1.3.tests/testthat/helper-tiers.R: tier 1 structural (ungated,
runs on CRAN), tier 2 recovery (skip_on_cran()), tier 3 full samplers and
coverage (skip_if_not_slow()). About 100 test files were re-gated and a tier
table added in tests/testthat/README.md.TULPA_FAST=1 is a fast smoke profile folded into both tier
gates, collapsing the suite to the tier-1 structural tests only (heavy fits and
samplers reported as skips, never dropped) for sub-minute plumbing iteration.
The default and CRAN runs are unchanged.LinkingTo: gcol33/tulpaMesh to v0.1.3.cpp_joint_inner_vcov_blocks now defaults
field_marginal = TRUE (and n_threads = 1). The cheap selected-inversion
recipe -- the betas block and betas x field cross solved exactly, the field
marginal variances from one Takahashi pass -- is the default per-cell
extraction; the full p x p block is opt-in via field_marginal = FALSE.scatter_cell_coupling_branch_impl, src/nested_laplace_joint_multi.h) gained
an optional per-arm rank-1 self-cross descriptor on CellDerivs
(arm_cross_rank1_coef / arm_cross_rank1_vec, inst/include/tulpa/cell_coupling.h).
When a coupled arm's (k, k) off-diagonal cross-Hessian is the symmetric rank-1
a v v^T -- every cross-row second derivative factoring through one scalar, as
in tulpaObs's all-undetected occupancy mixture -- the kernel collapses it to a
single a u u^T in joint-dof space (u = sum_r v[r] chain(row_r), accumulated
by accumulate_self_rank1_u and scattered by scatter_self_rank1_{dense,sparse})
instead of the O(rc^2) dense arm_cross_hess[k][k] loop, dropping the scatter
from O(sum rc^2) to O(sum rc) (gcol33/tulpaObs#94). Single-response path
(n_batch_ == 1) only; cross-arm blocks and the dense arm_cross_hess path are
unchanged, and the spec folds the rank-1's own diagonal into
arm_neg_hess_diag so the assembled Hessian matches the dense path to machine
precision.cpp_joint_inner_vcov_blocks
(src/joint_inner_vcov.{h,cpp}), replacing the serial-R per-cell
solve(Qk, E) over ~n_betas + n_field right-hand sides (gcol33/tulpa#112,
#113; gcol33/tulpaObs#93). For the field-marginal summary it solves only the
n_dense fixed-effect columns of Qk^-1 (the betas block and the betas x
field cross, exact) and recovers the field marginal variances from one
Takahashi selected-inversion pass (selected_inversion_diagonal,
sparse_cholesky.h); the field x field off-diagonal -- read by neither the SD
summary nor the Q_k-direct predict path -- is not formed. Cells run
concurrently over the supplied thread budget. A field_marginal = FALSE mode
forms the full block (the betas-only callers). Numerically identical to the
former dense path on the read sub-blocks (test-joint-inner-vcov.R).linalg_fast.h core (chol_factor_lower / chol_forward_solve /
chol_back_solve / chol_log_det / nngp_conditional_moments), replacing 8
hand-rolled copies across the Laplace, PG-Gibbs, SVC, spatiotemporal,
temporal-GP, GPU-fallback, and proper-CAR paths (#109). The default 1e-10
pivot jitter is named kCholJitter; the SVC kernel's 1e-6 is now an explicit
argument rather than a drifted literal. Behavior-preserving.1e-10); the dead *_autodiff.h twins are removed. Numerics shift at most at
the 1e-10 / ULP level (AR1 stationary term, summation order)..normalise_joint_arm() and .normalise_joint_arm_multi()
delegate to a shared .normalise_joint_arm_core(), collapsing ~50 duplicated
lines of arm-spec validation (#111). The single-block path still requires an
arm-level spatial_idx; the multi-block path still fills a zero placeholder --
the only behavioral split, now an explicit policy argument.tulpa_laplace(..., weights=) now scales the log-likelihood by
the per-observation weight, matching the already-weighted score and Fisher
Hessian (#108). The Newton globalization backtracks on the log-likelihood, so a
weighted-optimal step was judged against an unweighted objective, halved toward
zero, and stalled -- any non-uniform weighting returned a non-converged mode
shrunk toward the prior. The fix is a no-op when weights are absent or uniform
(every unweighted path is byte-identical); weighted gaussian / binomial fits now
match lm() / glm(weights=) to ~1e-5. test-laplace-weights.R locks it in.joint_inner_thread_budget() splits the pool so outer_used * inner <= n_outer
(never oversubscribed) and is a no-op when the grid saturates the pool. Verified
result-invariant (theta means byte-identical) with a ~1.12x single-fit speedup on
a 4-cell surplus grid; test-nested-laplace-joint-threading.R locks in the
invariance.cpp_spde_fractional_logmarginal, Eigen). The well-conditioned B /
matrix-determinant-lemma method -- built through the operator factor Pl
(cond = sqrt cond(Q)), never an explicit ill-conditioned Q inverse -- is
preserved: a direct precision-space marginal drifts by O(10) nats at large range
where cond(Q) ~ 1e10+, mis-identifying range. .spde_nested_logmarginal_at and
the single-point .spde_laplace_fractional_at delegate to it; the R det-lemma /
closed-form / family-weight code is removed. Reproduces the former R marginal to
~1e-10. The gaussian fractional phi is now the residual SD (variance phi^2),
consistent with the integer path (was the variance).(range, sigma) marginal now carries the GMRF
prior normalizer 0.5 log|Q(theta)| (#98). It was dropped on the integer-alpha
SPDE path -- the Occam term that bends the marginal down at large sigma -- so
fit_spde() (and the tulpa(..., spatial = spatial_spde()) front door) railed
sigma to the prior boundary and collapsed range, while the Tier-1 NUTS-joint
path recovered. The two integrators now agree: on a true 0.35/0.80 field the CCD
weighted means recover range ~ 0.38, sigma ~ 0.76 (matching NUTS), validated
by a new multi-seed recovery + CI-coverage gate in test-spde-ccd.R. The
normalizer is single-sourced in C++ (src/spde_logdet.h, a CHOLMOD log|Q|);
the fractional path's R-side fold is removed accordingly. Exposed and fixed a
latent gap where the SPDE CCD refit returned a NULL beta (the mode was never
split into fixed effects / field), now done once in laplace_spde_at().tulpa_re_cov_nested / _gibbs,
tulpa_laplace, fit_spde, tulpa_nested_laplace[_joint], tulpa_tgmrf)
routes its return through a shared .finalize_fit() so a directly-called fit
carries the same tulpa_fit class, fixed-effect layout, and explicit
draws_kind provenance tag as a tulpa()-dispatched one. The chain-vs-iid
diagnostic gate no longer computes a vacuous Rhat on a directly-called iid fit
(#102).tulpa() now validates phi > 0 for every dispersion-carrying
family (not just beta) and rejects non-integer / negative y for count
families at the front door; the inverse / 1mu2 links clamp eta off their
singularities (#104).copy= back-compat shim from
tulpa_nested_laplace_joint() (declare the copy coefficient on the arm via
field_coef, #105); consolidated the comparison / averaging verbs to native
PSIS-backed compare_models() / modelAverage() (stacking + pseudo-BMA, no
loo dependency -- loo moved to Suggests), and the four tulpa_tgmrf_*
fitters into one tulpa_tgmrf(mode=); standardized RE-Gibbs / PG-Gibbs
iteration arguments on n_iter + warmup (#103).fit_spde
CCD path does NOT recover (range, sigma) (the Laplace-marginalized SPDE
likelihood is prior-dominated in both); only the Tier-1 NUTS-joint path does.docs: runnable \examples and method @references on the front doors,
inst/CITATION, internal issue tokens stripped from rendered help, and stale
version / example strings refreshed (#106).
feat(frontdoor): mode = "agq" now reaches the adaptive Gauss-Hermite
quadrature fitter through tulpa(). A single random-intercept (1 | g) model
with a binomial / poisson / gaussian family routes to agq_fit(), with
control$n_quad (default 7; 1 recovers Laplace) selecting the quadrature
order and phi mapped to the gaussian residual sd. The front-door fit equals a
direct agq_fit() call. Previously the backend was registered but the design
dispatch fell through to a "reachable but not yet wired" error. Random slopes,
multiple RE terms, other families, and a beta_prior (AGQ is a
marginal-likelihood fit) are rejected with guidance.
coef, summary, confint,
vcov) now read a Gaussian fit's full-parameter $cov when it carries no
draws, grid moments, or $H_beta (the AGQ shape), and .fixed_draws_mat()
treats a zero-row $draws matrix as "no draws" rather than empty draws. Fits
with real draws / grid moments / H_beta are unaffected.feat(spatial): the single-arm multi-block nested-Laplace driver now honours a
per-observation design weight on an areal (icar) block, exposed as an optional
svc_weight field in the block spec. When present, observation i's eta
contribution is svc_weight[i] * z[spatial_idx[i]] rather than
z[spatial_idx[i]] -- a spatially-varying coefficient (the areal
f(cell, weight, ...)), the single-arm analogue of the row_weight the joint
multi-arm driver already carries. The weight enters at one layer (the
block-local weight resolved alongside the node index); the gradient inherits it
and the block Hessian its square through the chain rule, in compute_eta_spec
/ scatter_spec and the fitted_eta / predictive-variance reconstruction. A
block without svc_weight is byte-identical to before. This lets a standalone
occupancy fit carry a cell-indexed varying-coefficient field (consumed by
tulpaObs occu()'s spatial bar).
fix(joint): the coupled cell-coupling per-cell scatter now handles
INDEXED_MULTI prior blocks (a separable-MCAR block's several latent dofs per
row), not only INDEXED_SINGLE. Previously scatter_one_arm_row_{dense,sparse}
and build_arm_row_chain resolved a row's active latent dofs through
block.idx alone, so a multi-field block (e.g. a free-Sigma MCAR field) coupled
onto a coupled = TRUE arm received no gradient/Hessian from the cell-coupling
likelihood and stayed pinned at its prior mean. The active-latent resolution is
now a single collect_coupled_row_latents() helper shared by all three, so the
free-Sigma MCAR field couples through the joint occupancy mixture (consumed by
tulpaObs occu_cover()'s correlated | spatial bar). INDEXED_SINGLE coupled
fits (the areal / SVC trend path) are byte-identical.
spatial() gains a by = argument for replicated CAR -- one
independent copy of the whole varying-coefficient field per level of a factor,
with the hyperparameters shared across levels (INLA's replicate = /
mgcv's s(cell, by = ...), generalised to the bar). It is orthogonal to the
bar character: | / || sets the covariance among the coefficient columns
within a field, while by sets how many replicates exist. A by factor with
L levels builds the field over the block-diagonal Kronecker graph
I_L (x) Q (L disjoint copies, the node index offset into each level's
copy), so the replicates are independent and share one precision -- the outer
integration grid stays one axis. Supported for || and | (intrinsic) and
|| + proper = TRUE; | + proper stays out of scope, with or without
by. The new tulpa_bar_field_replicate() exposes the Kronecker remap for
consumer packages (the graph-side sibling of tulpa_bar_field_specs()).(n - 1) log-determinant normaliser -- assumed a single
connected graph; over a disconnected graph (the L-component block-diagonal
field a replicated CAR builds) the field constant of each component is its own
null direction. The kernels now apply one sum-to-zero pin per component and
normalise with (n - n_components), so a block-diagonal L-component
log-prior equals the sum of the L independent single-component log-priors.
A connected graph is the n_components = 1 case, byte-identical to before.(intercept, slope) field sharing a
free cross-covariance Sigma (x) Q^-1 (the within-arm covariance among the
fields) is copied onto a second linear predictor with one estimated amplitude
alpha (the cross-arm transfer): the donor arm sees the natural-parameter
field at amplitude 1, the copy arm at alpha, with alpha integrated over the
outer grid as a trailing axis alongside the Sigma log-Cholesky coordinates.
copy = list(arm =, block =, alpha_grid =) now accepts a type = "mcar"
block; the copy amplitude rides on the block's arm_scale (the natural-
parameter field stays the latent, so a single per-arm scalar carries the whole
correlated field). Previously MCAR rejected copy semantics. This is the engine
half of the cover-hurdle correlated-field consumer (gcol33/tulpaObs#64). The
single-arm MCAR path and every existing copy block are byte-identical
(no copy = empty arm_scale). Recovers Sigma (both SDs + the cross-
correlation) and the copy alpha against simulated truth
(test-nested-laplace-joint-multi-copy.R).tulpa_bar_field_specs(~ 1 + w || node, data) expands an lme4-style bar into
one spec per design-matrix column -- (column_name, weight, is_intercept),
where the intercept column is the unweighted (all-ones) field and each
covariate column carries its per-observation design value as the field weight.
tulpa_is_spatial_bar() recognizes such a bar. Both surface the single
expansion spatial() and the inline temporal() field constructor already
use internally (the one bar column-expansion helper the two paths share), so a
downstream package can offer a one-term spatial / temporal bar without
re-parsing the ~ 1 + w || node grammar. spatial() and the temporal field
constructor are refactored onto the shared bar recognizer so the engine and
any consumer cannot drift (gcol33/tulpa#93).fit_spde(method = "ccd") now actually runs the central-
composite design. fit_spde_nested_ccd() took optimHess() of the negative
log-posterior -- already the positive-definite precision of the Laplace
approximation -- and then negated it, so the degeneracy guard rejected every
usable mode and the integrator fell back to the rectangular grid on all
inputs. The negation is removed; the precision is used directly to orient the
design. method = "ccd" is the fit_spde() default, so fits over an
identified SPDE hyperparameter posterior now use the 9-node mode-centred
design instead of the 25-node grid. A weakly-identified axis still falls back
to the grid via the existing mode and Hessian guards (gcol33/tulpa#92).spatial(graph, ~ ... | cell)) now integrates its cross-covariance Sigma
on a mode-centred central-composite design over the log-Cholesky
coordinates, the same outer-integration recipe tulpa_re_cov_nested() uses
for random-effect covariances. The log-Cholesky coordinates (log L_ii on
the diagonal, raw strict-lower L_ij) are already unconstrained on all of R,
so they enter the joint CCD as identity axes: the integrator mode-finds the
marginal-likelihood mode in Sigma-space, orients the design by the Cholesky
of the posterior covariance, and weights with the corrected R-INLA design
weights -- no new mode-find or CCD code, the existing joint CCD machinery
drives it. This replaces the fixed log-Cholesky tensor that could land on the
nearest node and miss a sharp likelihood mode, and scales to general p at a
polynomial node count (1 + 2k + 2^k for k = p(p+1)/2 axes) where the fixed
tensor was exponential (p = 3 is 77 nodes vs ~1700 cells; p >= 4 no longer
overruns the grid cap). When the cross-correlation is weakly identified the
outer curvature is ill-conditioned and the CCD declines back to the fixed
log-Cholesky tensor, the correct net there. The outer Pareto-k accuracy
diagnostic (fit$pareto_k) is now reported for MCAR fits (it was NA while
the fit declined the CCD); a high k-hat on a small-group, weakly-identified
cross-correlation is a correct signal, not a defect. Recovers the fields and
every cross-correlation rho_ij with covering CIs at p = 2 and p = 3
(test-spatial-mcar.R).spatial(graph, ~ 1 + x | cell) (a single bar |) builds
correlated areal varying-coefficient fields -- a separable multivariate CAR
(MCAR) where the per-cell coefficient vector shares a cross-covariance
Sigma, with joint latent covariance Sigma (x) Q^-1 (#89). One coupled
block over the p fields assembles the Kronecker precision Sigma^-1 (x) Q
in the inner Laplace solve (the p per-field sum-to-zero constants are pinned
and folded by the block-Schur path), with the p design columns entering the
linear predictor as eta_i += sum_c X_{ic} u^{(c)}_{cell_i} via an
INDEXED_MULTI block. The outer grid integrates over Sigma in log-Cholesky
coordinates; the cross-field correlation rho and the per-field sigmas are
derived quantities, reconstructed per grid cell and weighted-quantiled (the
marginalize-derived-quantities rule). print() and fit$mcar_summary report
the marginalized Sigma (sigma_1, sigma_2, rho_12, ... with 95% CIs).
Recovers the fields and rho vs simulated truth with CI coverage
(test-spatial-mcar.R); the Sigma^-1 (x) Q assembly, gradient, and log-prior
(incl. the (n-1) log|Sigma^-1| normalizer) are locked by a direct algebra
check against kronecker(Sigma^-1, Q) (test-mcar-prior.R). The single bar |
no longer errors. A single | with proper = TRUE (correlated proper CAR) is
out of scope and errors. Tested for p = 2 (the headline intercept-plus-slope
case); general p > 2 fits through the same path with a coarser raw
log-Cholesky grid.temporal(formula = ~ 1 + x || time, structure = "rw1")
declares inline temporally varying-coefficient fields in a tulpa() model
formula, the temporal mirror of spatial() (#91). The bar's right-hand side
names the time index; the left-hand side expands (via model.matrix) into one
temporal field per design column -- the intercept column is a smooth temporal
level, a covariate column is a temporally varying slope on it
(eta_i += x_i * f(time_i)). structure selects the temporal GMRF: "rw1"
(default), "rw2", or "ar1" (which estimates its own correlation rho).
|| (independent fields) only; a single | (correlated) is reserved (the
temporal counterpart of the spatial MCAR). print() and
fit$temporal_field_hypers report each field's structure, sigma, and (ar1)
rho. The temporal() accessor on a fitted model and the bare temporal(col)
naming term are unchanged -- the constructor is reached only when temporal()
is given a formula.rw1 / rw2 / ar1) now carry the per-
row design weight (svc_weight) that the areal blocks already had, so a
covariate column scales the field per observation. The spatial and temporal
inline-field fitters share one engine (.bar_field_fit_core) and one bar
column-expansion helper, so the two paths cannot drift.spatial(graph, ~ ... || cell, proper = TRUE) builds proper
CAR varying-coefficient fields, where each field's precision is
Q = D - rho_car W with the spatial autocorrelation rho_car estimated from
the data instead of the intrinsic rho = 1 (#90). Each field stays
independent (||) but gains its own (sigma, rho_car) pair, so the per-field
outer grid is 2D (the car_proper registry derives the (tau, rho_car) axes
from the eigenvalue interval of D^-1 W); two proper fields give 4 axes and
CCD engages automatically. print() reports each field's structure (ICAR
vs proper CAR) plus the marginalized sigma and rho_car (median + 95% CI,
each a derived quantity weighted-quantiled over the outer grid, never a
plug-in of the modal hyperparameter); fit$spatial_field_hypers exposes them.
proper = FALSE (default) is unchanged (intrinsic ICAR, rho fixed at 1). A
single | with proper = TRUE (correlated proper CAR) remains a separate
model.B = A + sum_k coef_k 1_k 1_k' (the intrinsic field plus its sum-to-zero
rank-1 pins) the field sub-block A_FF is factored once, the pins fold via the
matrix-determinant lemma, and the field<->scalar coupling closes with a small
dense Schur complement -- the true Newton step with no perturbing ridge, so the
inner iteration converges quadratically and no longer drops ill-conditioned
high-(sigma, alpha) grid cells to -Inf (gcol33/tulpa#69). The Laplace
log-determinant uses the same partition (log|B| = log|B_FF| + log|Schur|),
which never factors the unpinned near-singular full matrix and so avoids the
matrix-determinant-lemma cancellation along the constant direction. The single-
species sparse oracle and the fused batched multi-response driver
(run_multi_block_nested_laplace_joint_batch) route their inner step through
one shared s2z_newton_step, so they stay bit-identical per species; the LM
ridge + Woodbury path remains the fallback when A_FF or the Schur complement
is indefinite far from the mode. Validated bit-identical (batched vs single,
max|dmode| = max|dlogmarg| = max|dQ| = 0 at a 412-cell field) and locked by a
direct block-Schur-vs-dense-LLT log-determinant + step unit test.spatial(graph, formula = ~ 1 + time || cell) declares areal
varying-coefficient fields inline in a tulpa() model formula, the way a
random-effect bar is written. The bar's left-hand side expands (via
model.matrix) into one independent CAR / Besag field per design column: the
intercept column is the spatial intercept field u_cell, a covariate column
is a spatially varying slope on it (a per-region trend time * s_cell). The
intercept is just the all-ones column, so the unweighted and weighted cases
share one path; factors, I(time^2), and splines expand for free. Each field
carries the sum-to-zero constraint and its own precision (independent fields).
A single response is fit through the single-arm joint nested-Laplace path,
which threads the per-row design weight (svc_weight), and summary() /
coef() report the marginalized fixed effects; the per-field posterior means
are on fit$spatial_fields.|| (independent fields) only -- a single |
(correlated fields, a multivariate CAR) errors as not-yet-implemented, as do
nested (a / b) or interaction grouping (the grouping must be a single
graph-node index; add ordinary nested random effects such as (1 | site)
separately), a missing bar, by = (reserved for replicated CAR), and
proper = TRUE (reserved). The bare spatial(col) areal-naming term and the
spatial = constructor path are unchanged.... | 0.06s/cells | 28 threads, whenever the
grid runs more than one cell at once, so "ran on N cores" is a property of the
fit log itself rather than something to read out separately (#88). The count
is the realised outer width stamped on the reporter by run_nested_laplace_grid
(after the sparse path's memory clamp), so it covers every model routing
through the joint engine -- the cover hurdle and occu_cover() included -- plus
the parallel NUTS sampler, which reports its concurrent-chain count the same
way. The mirrored R-side reporter (.tulpa_iter_progress()) gains an optional
threads argument carrying the same field. Serial loops leave the count at 1
and omit the field; their lines are byte-for-byte unchanged.Q on the fractional NUTS route (#87).n_x > 256; the log-determinant structure is now cached
across grid cells and species, restoring the densify-vs-rank-1 log-marginal
equivalence (guarded downstream by tulpaObs).logLik() / compare_models() description (a nested fit returns the
integrated-evidence scalar, not a per-grid vector) in the spatial and temporal
vignettes, refreshed the stale temporal front-door scope note (rw1/rw2/ar1 plus
panel and areal space-time are all wired), replaced an unexported
is_connected() reference with a base-R graph-Laplacian connectivity check, and
moved tgmrf_cpp() from a "forthcoming" framing to its shipped present-tense
description.slog_y / slog_1my, added in 0.0.15) so consumer packages can pin a
released engine; consumed by tulpaObs aggregate.pos. No engine code change.beta arm. When an arm carries slog_y /
slog_1my (the within-group sums of log(y) and log(1 - y), with
n_trials the per-row group count), a row collapses n exchangeable beta
observations sharing one linear predictor into a single row. The beta
log-density is linear in log(y) and log(1 - y), so the log-likelihood,
gradient and Fisher Hessian are pointwise unchanged and the aggregated fit is
byte-identical to the per-observation path (n = 1 reduces exactly to the
ungrouped branch). Read by the single-block and multi-arm joint drivers; the
shared log_lik_beta_grouped / grad_hess_beta_grouped helpers in
laplace_family_link.h are the single source. This is the engine backing for
tulpaObs's aggregate.pos cover-arm reduction (gcol33/tulpaObs#49)..is_multi_block_prior is now a single predicate (the
byte-identical _joint copy is dropped); the spatially- and
temporally-varying-coefficient print/summary methods delegate to shared
.print_varying_coef / .summary_varying_coef helpers; the column-major
matrix builder is one template over the element type (NumericMatrix vs
IntegerMatrix); and the internal CAR-proper and PC-variance log-prior helpers
follow the dominant log_prior_* naming (log_prior_car_proper,
log_prior_sigma2_pc), retiring a dead wrapper.R CMD check --as-cran ERROR and WARNING. The
joint-NUTS fractional-nu test now asserts the rejection at the fit call --
the spec constructor accepts fractional nu since the rational SPDE landed
(gcol33/tulpa#71); the performance-core test drops an assertion comparing a
hardware core count against the OpenMP-capped thread limit. Rd: a dangling
\link and lost-brace math in the rational-SPDE docs are fixed; NAMESPACE
gains importFrom(utils, flush.console) / importFrom(stats, vcov); the
tulpa_sample_glmm() offset is passed by its full name (offset_nullable).Makevars.win
changes, so a .gch left from an earlier flag set is no longer silently
rejected ("created and used with differing settings") and re-parsed in every
translation unit -- restoring the PCH speedup (cold compile ~70s -> ~61s).progress.file is set, the robust liveness signal for
detached runs where an Rcout flush does not survive the stdout buffer.
GridProgress gains emit_console + unit; an R-level
.tulpa_iter_progress() mirrors the same <done> <total> <elapsed_s>
<eta_s> wire format so a detached reader sees one file regardless of which
loop produced it. NUTS ticks a shared reporter across chains (the console
line self-suppresses inside the OpenMP region, the heartbeat file is the
parallel channel). The nested-Laplace console default flips ON; inner
refinement / EM / CCD-probe call sites pass progress = FALSE so only the
top-level fit ticks.tulpa_profile(), the inner sparse-Laplace phase timer.compute_param_layout() lays out: random effects (intercept,
slopes, correlated, multi-term), an areal spatial field (ICAR / BYM2), and a
temporal field (RW1 / RW2 / AR1). The variance-component hyperparameters
(sigma_re per term/coef, spatial tau / BYM2 sigma+rho, temporal tau
/ AR1 rho) are sampled JOINTLY with the latent and fixed effects -- full
Bayes over the variance components, the exact-MCMC counterpart of the Laplace
/ logpost backends that condition on them. The four default-link families
(gaussian / poisson / binomial / neg_binomial_2) get an analytic reverse-mode
AD gradient; other families fall back to the numerical gradient, which still
scores the full latent log-posterior. tulpa(mode = "hmc" / "smc" / "vi" /
...) reaches the new path for a structured formula; ess carries random
effects but declines a structured spatial / temporal field (its isotropic
Gaussian-prior block cannot hold the graph precision). The multi-term RE
ModelData marshalling is now shared (re_structure.h) between the Laplace
multi-RE fit and the sampler builder. Continuous spatial (gp / nngp / hsgp),
CAR_proper, and SPDE fields keep their dedicated paths.run_multi_block_nested_laplace_joint_sparse_impl). The cell-coupling
(coupled) arm's per-cell dispersion was read lock-free from the shared arms
during the inner Newton solve while a concurrent grid cell's prep_at_grid
rewrote it under the nl_sparse_phi critical -- every other arm already read
a thread-local snapshot, but the coupled arm did not. On a gridded coupled-arm
dispersion (e.g. the beta-cover precision on the phi axis) this corrupted
per-cell values across n_threads_outer > 1, causing wrong log-marginals /
non-convergence and the intermittent native crashes reported downstream
(gcol33/tulpaObs#42). Now the coupled arms' dispersion is snapshotted under the
same critical and read via a phi_override in the coupled scatter / log-lik;
serial and dense callers pass nullptr and are byte-unchanged. Verified: a
220-region BYM2 beta-cover fit is identical serial vs n_threads_outer = 6 to
~1e-10.feat(offset): thread offset() terms through the SPDE, spatial-Laplace, and
ModelData-sampler paths of tulpa() (gcol33/tulpa#72). A fixed log-exposure /
log-effort offset (the standard way to model rates) now enters the linear
predictor eta = offset + X beta + field on every fitter, where it was
previously honoured only on the non-spatial GLMM / EM paths and hard-errored
on the rest. The offset is carried as the per-process ProcessData::offset
(areal ICAR/CAR/BYM2 + NNGP Laplace and all seven sampler kernels --
hmc/ess/sghmc/sgld/mclmc/smc/vi -- which already consumed it via
compute_eta_spec / precompute_generic_fixed_eta), as a per-arm
ParsedArm::offset in the nested-Laplace joint engine (the nested SPDE path),
and as a raw additive vector in the single-point SPDE kernel. The three
tulpa() guards are removed; fit_spde() and tulpa_sample_glmm() gain an
offset argument.
fix(spde): gate fractional Matern smoothness (nu) instead of silently
fitting a mis-specified field (gcol33/tulpa#71). The wired rational assembly
builds Q = tau^2 sum_k w_k (L + r_k C)' C^{-1} (L + r_k C), whose spectral
symbol tau^2 sum_k w_k (l + r_k)^2 is a single quadratic in l for any
number of poles -- so no choice of poles/weights recovers a fractional
alpha; the construction collapses to an alpha = 2 field regardless of the
coefficients (verified to machine precision). The coefficient generator
(rational_spde_coefficients()) previously returned a self-derived
log-uniform approximation while documenting the published BRASIL / Bolin et
al. (2023) method. spatial_spde() / spatial_spde_custom() now require an
integer nu (0, 1, 2, ...) -- the exact FEM construction -- and reject
fractional nu with a clear error across every fit path (Laplace, nested,
NUTS, joint). A faithful rational SPDE precision assembly remains tracked
under gcol33/tulpa#71.
perf(nested-laplace-joint): budget the replicated per-outer-thread state
(the sparse-Hessian builders + numeric factor) against detected physical RAM
instead of a fixed 2 GB cap (gcol33/tulpa#64). The old cap clamped the outer
grid to ~15 threads at EVA scale on a 64 GB box even when 28 were requested;
a new standalone sysmem translation unit (total_ram_bytes(), Windows /
macOS / POSIX) lets a wide field use every requested outer thread when the
memory is there, falling back to the 2 GB cap only when the RAM query fails.
fix(nested-laplace-joint): the outer-grid progress ETA is now computed from
the realised parallel width, not a serial extrapolation of the pilot rate
(gcol33/tulpa#64). A ~21 min serial pilot on a 48-cell grid previously
projected ETA ~16 h; the ETA now estimates per-cell wall time from completed
waves (one serial pilot wave + (done-1)/width parallel waves) and projects
the remaining cells over ceil(remaining/width) waves. At outer width 1 this
is exactly the previous serial formula.
fix(nested-laplace): retire the unguarded .nl_normalise_weights softmax
entirely -- every outer-grid weight normaliser now routes through the
finite-guarded .nl_normalise_weights_safe (gcol33/tulpa#65). The remaining
unguarded call sites (the single-block grid, the multi-block grid, the
adaptive-refinement reweight, and the cheap-screen ESS gate) could still
collapse fit$weights to all-NaN when one outer cell returned a non-finite
log_marginal, breaking tulpa_posterior_draws() / predict() / WAIC on the
finite, precision-carrying cells. logLik()'s grid log-sum-exp is guarded the
same way so a non-finite cell no longer poisons the integrated evidence (and
AIC / compare_models downstream). Behaviour is unchanged on all-finite grids.
fix(nested-laplace-joint): the single-block joint path normalises the outer-grid
weights with the NaN-safe .nl_normalise_weights_safe (drop non-finite cells,
renormalise) instead of the bare max(lm) softmax. A single non-finite
log_marginal (an inner solve that diverges at an extreme hyperparameter cell)
previously poisoned the whole weights vector to NaN, so theta_* summaries
had to work around it and tulpa_posterior_draws() failed with "no outer-grid
cell has positive weight". Degenerate cells now get zero weight and the
remaining cells carry the mass, matching the multi-block path and the
hyper-grid path. Behaviour is unchanged on all-finite grids.
feat(gauss-hermite): export gauss_hermite.h (probabilist Golub-Welsch nodes)
under inst/include/tulpa/ so LinkingTo consumer packages reuse the engine's
one implementation instead of re-deriving the quadrature.
feat(nested-laplace): the multi-block joint path announces the engaged outer
integrator under control$verbose = TRUE, in one line at selection time
before the inner solves (gcol33/tulpa#63): e.g. outer integration: CCD
(4 latent axes, 25 nodes), tensor grid (72 cells), or CCD declined ->
tensor grid (72 cells). Previously the "auto" switch to the CCD at >= 4
latent axes was silent -- a consumer who omitted integration from the
control could end up on the CCD path (and, on a ridged posterior, in the
#62 thrash) with no signal, the only post-hoc tell being fit$...$integration.
The resolved integrator is still returned on the joint result as $integration.
control$integration for a multi-block joint prior
gains "auto" (the new default) and now takes "auto" / "ccd" / "grid"
(gcol33/tulpa#59). "auto" uses the CCD only at >= 4 transformable axes,
where the tensor product's k^d blow-up bites hardest, and keeps the cheaper,
more ridge-robust tensor grid at <= 3 axes; "ccd" lowers the CCD threshold
to >= 3 axes; "grid" always forces the tensor product. (Previously the
default engaged the CCD at >= 3 axes.)control$integration = "ccd" (default for >= 3
transformable axes) integrates the joint hyperparameter posterior on a central
composite design around its mode -- far fewer inner solves than the k^d tensor
product. Auto-declines to the tensor grid for <= 2 axes, an unguessable axis
(CAR_proper rho_car / non-BYM2 rho), or a degenerate mode-find.phi_grid (gcol33/tulpa#61).
An active per-arm dispersion axis no longer disables CCD: the design is built
over the >= 3 latent axes and the phi tensor is Cartesian-crossed on top,
with the CCD node weights replicated across the phi cells. A two-field beta
occu_cover() / cover() joint fit (4 latent axes + a phi grid) integrates
on 25 x phi cells instead of the 81 x phi dense tensor.1 1') Hessian; the sparse
path previously stored only its diagonal, so force_sparse / large-field ICAR
and BYM2 joint fits had a log-marginal off from the dense path by the missing
rank-1 log-det term. Now exact, size-gated: small fields densify the field
block and store 1 1' directly; large fields fold the rank-1 in at solve time
(Sherman-Morrison step + matrix-determinant-lemma log-det) via one reuse-solve
per field. TULPA_S2Z_DENSIFY_MAX tunes the cutoff.gibbs_spatial*, hmc_zi.h) and the trailing ModelType /
LegacyRatio references in the exported headers; rename stale numdenom
references to tulpaRatio in the docs.TULPA_FAST dev tier (skip_if_fast()) runs the structural /
closed-form / gradient unit tests only; the default still runs the full
recovery suite.src/checkpoint_io.h, CheckpointLog<Payload>) owns the file format,
load/append/torn-tail logic and fingerprinting once; a torn final record is
truncated and re-run, and a header fingerprint mismatch (different data /
settings / grid) errors rather than resuming onto a stale result. Wired
through:tulpa_nested_laplace()
multi-block, and fit_spde() -- control$checkpoint = list(path =,
resume =) (a checkpoint = arg on fit_spde());tulpa_re_cov_nested()'s CCD node integration (checkpoint = arg, an
atomic-RDS node cache);cpp_tulpa_fit_generic_chains(checkpoint_path=)):
a chain is deterministic in (seed, chain_id, data, settings), so a resumed
chain is bit-for-bit identical to the uninterrupted one.
Extends the joint-fit checkpoint shipped earlier to the rest of the engine;
the joint path was refactored onto the shared core with no file-format change.
Tests: test-checkpoint-universal.R.k_samples is below
the GPD-fit floor (.PSIS_MIN_EVAL, 25): a sub-floor budget can never reach
enough finite evaluations, so it now returns NA without paying a single
inner solve instead of evaluating the whole budget and discarding it.min(max_iter, 25). A
draw at an implausible hyperparameter (where the inner Newton would
otherwise stall to the full budget) carries negligible importance weight, so
the cap bounds its cost without moving the k-hat -- converged draws keep
their exact log-marginal; only the negligible-weight tail is truncated.
Covers tulpa_nested_laplace_joint() (single- and multi-block); the
sub-floor decline also covers tulpa_nested_laplace(), tulpa_re_cov_nested()
and fit_spde() through the shared PSIS cores.tulpa_re_cov_nested() documents its outer accuracy-diagnostic controls
diagnose_k (default TRUE) and k_samples (default 200) in the help page.ms_abun); the generic engine retains the
structure-agnostic make_site/make_group equivalence checks.fit$timing (gcol33/tulpa#48). A named numeric of seconds carrying
total plus a phase breakdown -- setup (validation / encoding / grid
construction), grid (the inner Laplace solves that scale with grid size and
core count, including adaptive-refinement and consistency passes), postproc
(weight / moment / marginal assembly), and diagnostics (the outer
Pareto-k-hat). Covers tulpa_nested_laplace() (single- and multi-block) and
tulpa_nested_laplace_joint() (single- and multi-block dispatch); consumer
fits riding on the joint object inherit it. A new print method for the
nested-Laplace classes surfaces a one-line summary ("fit in 5h 25m (grid 2h
09m)") alongside the hyperparameters, grid size, and outer Pareto-k-hat.control$n_threads_outer (gcol33/tulpa#46, lever 2). It previously
forced a serial outer loop, so on a large field only the inner per-observation
OpenMP parallelised and most cores idled. The sparse driver now allocates a
per-outer-thread pool (Hessian builder, Newton scratch, arm specs, scatter
index cache, DENSE_BASIS scratch) and dispatches grid cells across
n_threads_outer, matching the dense driver. The phi-grid dispersion axis is
parallel-safe: it rewrites the shared arms dispersion per cell, so each cell
snapshots it into its own thread's specs under a short critical before the
lock-free Newton solve. A memory guard clamps the thread count when the
replicated builders would be too large (very wide fields fall back to fewer
outer threads). Parallel-vs-serial parity (with and without a phi axis) is
covered in tests/testthat/test-nested-laplace-joint-sparse-parallel.R.inner_refresh
(gcol33/tulpa#46, lever 1b). On a factor-reuse step the Hessian is discarded,
so the cell-coupling scatter now passes a grad_only request to the
CellCouplingSpec (a new ABI-appended CellDerivs::grad_only): a spec may
skip its negative-Hessian work (e.g. a beta arm's digamma/trigamma) and emit
only the exact gradient, and the kernel skips the cross-arm Hessian scatter.
Specs that do not implement it write the full Hessian as before (correct, no
saving). The gradient stays exact on every step, so the converged mode is
unchanged -- validated by a grad-only-honoring cell-coupling reuse test in
tests/testthat/test-nested-laplace-joint-inner-refresh.R.control$inner_refresh (default 1L) adds
Shamanskii / chord-method Cholesky factor reuse to the sparse joint inner
Newton (gcol33/tulpa#46). For a non-quadratic positive arm (e.g. a beta cover
arm) the latent Hessian changes every inner iteration, so the default
re-factorizes the sparse Cholesky on each step -- the dominant per-grid-cell
cost. inner_refresh = m > 1 re-factorizes only every m-th inner step and
reuses the cached factor in between (refreshing early if a reused solve
fails). The gradient is exact on every step and each step is line-search
safeguarded, so the converged mode is unchanged and the final mode-pass
Hessian (log_det, SEs) is always fresh -- only the path to the mode uses a
stale curvature. Applies to the sparse LM path; the dense small-n_x path
re-factorizes cheaply and ignores it. Bit-equivalence to the every-step
default is covered in tests/testthat/test-nested-laplace-joint-inner-refresh.R.tulpa_posterior_draws(fit, idx, n) -- a generic
posterior sampler for the grid-integrated joint nested-Laplace backend (the
inla.posterior.sample() analogue, gcol33/tulpa#44). Draws from the outer-grid
mixture sum_k w_k N(m_k, V_k): each draw picks a grid cell from the
integration weights, then samples the inner latent vector from that cell's
constrained Gaussian via the stored sparse precision Q_csc_*_per_grid
(requires control$store_Q = TRUE). The ICAR / BYM2 field sum-to-zero
constraint is imposed by conditioning on kriging (Rue & Held 2005), so the
per-cell marginal matches the constrained inner-Laplace covariance exactly;
single-block and multi-block (multi-field trend) layouts are both handled.
Sampling the mixture -- rather than a single moment-matched Gaussian -- is the
faithful primitive for marginalizing nonlinear derived quantities (change in
occupancy, expected-cover products). Draws are tagged iid. Tests in
tests/testthat/test-posterior-draws-joint.R.control$prune = TRUE) no longer mis-ranks grid cells or drops the true
posterior mode (gcol33/tulpa#43). The screen previously ran a single Newton
step from one global pilot mode for every cell; when the inner latent mode
moves substantially across the outer grid (large spatial fields, wide
sigma/rho/alpha ranges) the one-step approximation mis-estimated far cells by
O(1e5) log-units and inverted the ranking, so the prune could skip the full
solve on the actual mode. The screen is now a rank-faithful chained sweep over
the lattice: each cell runs a short Newton run warm-started from the previous
screened cell's quasi-mode, so every cheap mode stays near its cell's true
mode and the cheap ranking agrees with the full-solve ranking.$prune_fallback_triggered,
$prune_fallback_reason) rather than silently returning a pruned answer. A
silently-wrong pruned posterior is now impossible.fit_spde() reports an outer Pareto-k-hat accuracy diagnostic
($pareto_k) over the integrated (range, sigma) hyperparameters -- the
iid-fit counterpart of Rhat. k-hat < 0.7 means the Gaussian proposal the
integrator orients its CCD/grid with fits the hyperparameter posterior;= 0.7 flags a skewed / heavy-tailed posterior the grid misfits. Controlled by
diagnose_k(default TRUE) /k_samples(default 200), RNG-restored so the fit's draws are unchanged.
tulpa_nested_laplace_joint,
single- and multi-block) reports the same outer Pareto-k-hat over its
heterogeneous hyperparameter space (gcol33/tulpa#42). A block-type-aware
per-axis transform unconstrains each axis -- positive scales by log, the
BYM2 mixing weight by logit, the copy coefficient by identity -- and the
summed log-Jacobians enter the importance target. A CAR_proper rho_car axis
(support is the adjacency eigenvalue interval, not guessable) declines to
quadrature ESS rather than apply a wrong transform.control$hessian selects the inner-Newton curvature for
the joint mixture Hessian -- "lm" (default, diagonal-ridge escalation until
CHOLMOD factorizes), "psd" (eigen-clamp the dense observed Hessian), or
"fisher" (complete-data expected information, PSD by construction).CellCouplingSpec registry
(gcol33/tulpa#32). Consumer packages register a per-cell coupling spec (e.g.
tulpaObs's cover-hurdle) so two arms share a latent field; the cross-arm block
is assembled into the joint Hessian (tulpa_register_cell_coupling C
callable, default "separable" always available).tulpa_re_aghq() -- a callback-driven adaptive Gauss-Hermite refinement
of a grouped random-effect covariance. Generalizes agq_fit() (intercept-only
RE, built-in binomial/poisson/gaussian) to random slopes and
correlated multi-coefficient blocks sharing one grouping factor, with the
per-observation marginal likelihood supplied by the caller through a
make_site callback. This lets a downstream package refine a custom marginal
(e.g. a latent-state-integrated occupancy / detection likelihood) through the
same quadrature. Reuses the existing log-Cholesky covariance parametrization
(.re_cov_*), gauss_hermite_prob(), and an optional LKJ correlation penalty;
the fixed parameters and chol(Sigma) are optimized jointly on the
exact-marginal log-likelihood, with SEs from the marginal Hessian. Recovery
tests in tests/testthat/test-re-aghq.R.feat(nested-laplace): tulpa_nested_laplace_joint() now reports the outer
Pareto-k-hat accuracy diagnostic ($pareto_k, $pareto_k_is_ess) over its
heterogeneous hyperparameter space, completing the nested-Laplace k-hat family
alongside the re-cov, generic single-axis, and SPDE paths (gcol33/tulpa#42). A
block-type-aware per-axis transform registry unconstrains each axis -- positive
scales (sigma, tau, phi_*, ...) by log, the BYM2 mixing weight (rho)
by logit, the copy coefficient (alpha) by identity -- with the summed
log-Jacobians in the importance target; the inner marginal is re-evaluated
through the same kernel the integrator used. A fit carrying an axis whose
support is the adjacency eigenvalue interval (CAR_proper's rho_car) declines
to the quadrature-ESS fallback rather than apply a guessed transform. Gated by
control$diagnose_k (default TRUE) / control$k_samples (200), RNG-restored
so draws are unchanged. Recovery + plumbing tests in
tests/testthat/test-nested-laplace-joint-pareto-k.R.
refactor(aghq): one compiled adaptive-Gauss-Hermite engine behind the whole
ML-II optimize family. The per-group marginal -- mode-find, quadrature grid,
log-Cholesky Sigma packing, LKJ penalty and marginal Hessian -- now lives in
C++ (src/aghq_re*.{h,cpp}, inst/include/tulpa/aghq_oracle.h), driven through
one structure-agnostic per-group oracle. tulpa_re_aghq() and agq_fit() are
thin wrappers over it (their R integration loops are gone; the optimizer takes
finite differences of the compiled objective, consistent at every n_quad).
The mode-find is a globally-convergent modified Newton (prefers the true
observed-info Hessian where PD; falls back to a caller-supplied PSD Fisher or an
eigenvalue-reflected curvature otherwise), so a latent-variable marginal whose
observed information is indefinite away from the mode no longer breaks it.
tulpa_re_aghq() gains theta_prior_sd (a Gaussian ridge on the fixed
parameters) and returns log_marginal.
refactor(aghq): agq_fit() builds its per-group marginal from the native
GLMM oracle (cpp_glmm_oracle_make, src/glmm_oracle.h) instead of an
R-closure oracle over an R family density. The built-in binomial /
poisson / gaussian densities now have a single C++ source of truth shared
with tulpa_re_aghq(), tulpa_re_cov_nested(n_quad > 1) and the Gibbs sweep;
this removes .agq_loglik_elt() / .agq_score_info(). Estimates,
covariances and n_quad-convergence are unchanged; the reported
log_marginal now carries the full likelihood normalizing constants (the
binomial coefficient and Poisson lgamma the R density previously dropped),
matching the other AGHQ fitters.
refactor(gibbs): the exact-target random-effect-covariance sampler
(tulpa_re_cov_gibbs()) runs its Metropolis-within-Gibbs sweep in compiled
code (src/re_cov_gibbs.cpp, src/re_cov_gibbs_sweep.h), driven by one native
per-row GLMM likelihood (src/glmm_oracle.h) rather than a duplicated R
density. This removes .re_obs_loglik: the family densities (binomial /
poisson / gaussian / negative-binomial-2) now have a single source, and the
engine owns the shared linear predictor with the cross-block eta coupling for
several terms. The estimator is unchanged -- the C++ sweep keeps the R
sampler's RNG-draw order, so a seeded run reproduces the previous sampler's
draws bit-for-bit (verified across the correlated, diagonal and multi-term
test-re-cov-gibbs.R cases). The R wrapper keeps the pilot Laplace solve
(starting values + proposal shapes) and the weighted-quantile summary.
feat(re-cov): tulpa_re_cov_nested() gains n_quad -- an adaptive
Gauss-Hermite refinement of the inner marginal. n_quad = 1 (default) keeps
the joint-field Laplace inner solve unchanged; n_quad > 1 routes the inner
solve through the shared compiled AGHQ engine (cpp_glmm_oracle_make +
cpp_aghq_objective), so each per-group integral inside the Sigma
integration is debiased by quadrature (the tulpa_re_aghq() correction applied
under the grid), reducing the small-cluster variance attenuation for binary /
low-count data. The fixed effects are integrated (profiled out + a fixed-effect
Laplace term), so the reported fixed-effect posterior is the marginal (ML-II)
one rather than the joint-mode (PQL) estimate. The per-group integral only
factorizes over one shared grouping factor, so AGHQ requires that; crossed RE
terms keep the joint-field Laplace (n_quad > 1 errors). Recovery + the
crossed-factor guard in test-re-cov-nested.R.
feat(nmix): community / multispecies N-mixture (tulpa_nmix_laplace_re(), the
spAbundance msNMix model) now fits through that shared engine -- it wraps
tulpa_nmix_site_marginal() as the per-species oracle (the marginal, the
abundance/detection score, and both the observed-info block with the
Var[N|y] coupling and the PSD complete-data Fisher for the mode-find) and
integrates the per-species coefficient random effects at n_quad = 1 (joint
Laplace). This replaces and removes the bespoke C++ Laplace-EM
(src/nmix_re.cpp); the community fit verified against an independent Laplace
marginal and recovered over seeds (tests/testthat/test-nmix-re.R).
Fixed-effect SEs are now the joint marginal Hessian (marginalizing the
community-covariance uncertainty, closer to the spAbundance posterior) rather
than a Sigma-plug-in Schur complement. The numerical knobs tol / inner_max
/ inner_tol are dropped (the engine owns the mode-find); sigma_beta is kept
as the fixed-effect ridge. Poisson only for now.
feat(nmix): tulpa_nmix_site_marginal() exposes the per-site N-mixture
marginal as a composable random-effect callback (eval / eval_beta /
obs_info_block), and tulpa_re_aghq() gained a make_group path for the
general / multi-arm case (a per-group b-space oracle), so a custom marginal
with coupled arms at different granularities -- the abundance / detection arms
of an N-mixture site sharing a species grouping -- integrates through the same
quadrature. (The C++ tulpa_nmix_laplace_re() above is the production fitter;
this is the composable / AGHQ-refinement path.)
perf(nmix): the per-site kernel (nmix_kernel.h) caches its eta-independent
lgamma combinatorial terms (NMixSiteCache / nmix_precompute_site /
compute_nmix_site_cached), so an iterative fitter that evaluates a site many
times at changing linear predictors skips the lgamma recompute (the dominant
cost). The single-shot compute_nmix_site() Poisson path delegates to the
cached helper -- single source of truth -- so existing single-species / spatial
fits are byte-identical (nmix regression suite unchanged).
refactor(nested-laplace)!: collapsed the 3 single-block temporal entries
(*_{rw1,rw2,ar1}) to one *_temporal entry that selects the kernel at
runtime via a temporal_type argument through the shared make_temporal_ops
registry -- the same collapse the spatio-temporal entries already use, so
adding a temporal kernel is O(1) at every layer (Rcpp entry, extern-C shim,
exported ABI). ABI break (TULPA_ABI_VERSION 26 -> 27):
tulpa_nested_laplace_{rw1,rw2,ar1} + their NestedLaplace{Rw1,Rw2,Ar1}Fn
typedefs become tulpa_nested_laplace_temporal / NestedLaplaceTemporalFn;
downstream packages rebuild. The R-level rw1 / rw2 / ar1 block types are
unchanged.
refactor(laplace)!: removed the 8 dead family-enum single-point Laplace
C-callables (tulpa_laplace_mode_{dense,spatial,dense_multi_re,bym2,gp,
multiscale_gp,multiscale_temporal,rsr}) and their LaplaceMode*Fn typedefs.
No package consumes them -- every model package routes single-point Laplace
through the LikelihoodSpec path (tulpa_laplace_spec_*). ABI break
(TULPA_ABI_VERSION 25 -> 26); downstream packages must rebuild. The shared
LaplaceShimResult POD is retained (reused by the spec shims).
refactor(nested-laplace)!: collapsed the 15 spatio-temporal nested-Laplace
entries (*_st_<spatial>_<temporal>) to 5 per-spatial-family entries
(*_st_{icar,car_proper,bym2,hsgp,nngp}) that select the temporal kernel
(rw1 / rw2 / ar1) at runtime via a temporal_type argument, dispatched
through a single make_temporal_ops registry. Adding a temporal kernel is
now O(1) -- one registry branch, no new cross-product function at any layer
(Rcpp entry, extern-C shim, or exported ABI). ABI break (TULPA_ABI_VERSION
24 -> 25): the 15 tulpa_nested_laplace_st_* registered callables + their
NestedLaplaceSt*Fn typedefs became 5; downstream packages must rebuild.
Dense/sparse per-kernel equivalence preserved (test-nested-laplace-st-sparse-equivalence.R).
feat(nmix): tulpa_nmix_laplace() gains mixture = c("P", "NB") -- a
negative-binomial abundance mixing distribution
(N_i ~ NegBin(mean = lambda_i, size = r), neg_binomial_2 convention) in
addition to the Royle (2004) Poisson kernel. The per-site marginal, its
scores (including the analytic dispersion score d log L / d log r), and the
full joint observed-information Hessian are closed form; the dispersion
log_r is profiled by block coordinate ascent outside the inner beta-Newton
and reported with its standard error in vcov. Matches
unmarked::pcount(mixture = "NB") on coefficients, log-likelihood, and
standard errors to machine precision, with the usual analytic-derivative
speed advantage. Poisson remains the default and is unchanged.
feat(nmix): the spatial nested-Laplace N-mixture fits
(tulpa_nmix_laplace_icar(), tulpa_nmix_laplace_car_proper(),
tulpa_nmix_laplace_bym2()) gain mixture = "NB". The NB size r is
integrated as an additional outer grid dimension alongside the spatial
hyperparameters (tau / rho / sigma); the posterior r_mean / r_sd
are reported from the grid weights. The inner (beta, z) / (beta, v, w)
Newton is unchanged in dimension -- only the likelihood pieces and the
NB-aware Var[N|y] rank-1 correction depend on r. Poisson remains the
default with identical behaviour and grid shape.
refactor(api): tulpa_nested_laplace() and tulpa_nested_laplace_joint()
collapse their perf/numerical knobs into a single control = list() argument,
matching tulpa(). The top-level signatures now carry only statistical
arguments (y/n_trials/X/prior/spec/family/phi/likelihood/...;
responses/prior/copy/phi_grid/prior_sigma/prior_alpha). Tuning
knobs move into control: single-arm max_iter, tol, n_threads, x_init,
keep_grid_hessians; joint additionally n_threads_outer, tile_warm,
prune, prune_tol, store_Q, adaptive_grid,
adaptive_grid_edge_thresh, adaptive_grid_max_passes,
var_of_means_consistency, force_sparse, verbose. Pre-release breaking
change -- pass these inside control = list(...) (no deprecation shim). The
dead single-arm verbose knob was dropped. Internal callers (em_laplace,
the tgmrf pilots) and the shipped examples were migrated.
feat(laplace): tulpa_laplace_beta() gains a beta_prior argument, forwarded
to the inner tulpa_laplace() fits (both the outer phi search and the final
refit) so the beta arm can carry a Gaussian fixed-effect penalty. Pure R
passthrough -- tulpa_laplace() already applied beta_prior for
family = "beta". Enables penalised beta-regression arms downstream
(tulpaObs cover_priors() positive arm). Rejected with spatial, matching
tulpa_laplace().
feat(laplace): correlated random slopes (1 + x | g) on the Laplace engine
(gcol33/tulpa#28). tulpa_laplace() RE terms accept a per-term covariance via
L (lower-triangular Cholesky, Sigma = L L') or cov; the off-diagonal now
enters both the joint Hessian (mode finding) and the marginal fixed-effect SE.
Previously a multi-coefficient RE block could only carry a per-coefficient
marginal-sigma vector (a diagonal covariance), so (1 + x | g) was
inexpressible under Laplace and downstream packages routed it to NUTS. The C++
multi-RE kernel already consumed a packed Cholesky; this wires the R API to it
through a single .re_cov_spec() helper and rebuilds the marginal Schur
complement with a block-diagonal precision. It also fixes a pre-existing bug
in the marginal-SE linear predictor -- the eta reconstruction treated every RE
term as intercept-only, so the returned H_beta silently ignored random
slopes (this affected (x || g) as well). Validated against an independent
full-precision Schur in tests/testthat/test-laplace-corr-re.R. Estimating the
covariance itself (the EM M-step for a full Sigma) is the follow-up; the
engine now fits correlated slopes at a supplied covariance.
feat(laplace): tulpa_laplace(return_re_cov = TRUE) returns per-group
posterior covariance blocks cov_blocks -- one n_coefs x n_coefs matrix per
(RE term, group) in term-major then group order, each a diagonal block of the
full inverse Hessian (fixed effects and other groups marginalized out), i.e.
Cov(u_g | y, Sigma), not the inverse of a diagonal block. Built by reusing
the Cholesky factor from the log-determinant (one back-solve per block column,
no refactorization). This is the primitive a full-covariance EM M-step
consumes to update Sigma_k <- mean_g [u_g u_g' + Cov(u_g)]; tulpaObs#11 uses
it to fit (1 + x | g) deterministically. Non-spatial multi-RE path only
(rejected with spatial).
feat(nuts): expose tulpa's across-chain OpenMP runner through the model-facing
C ABI (gcol33/tulpa#30). New registered callable tulpa_run_nuts_chains
(header accessor tulpa::get_nuts_chains_fn()) runs n_chains chains in one
call and fills a caller-allocated array of NUTSResult, so downstream packages
stop re-implementing chain orchestration (offset-seed loops / PSOCK clusters)
in R and get the engine's thread-parallel path for free. init and the
optional inv_metric_diag are chain-major [n_chains * n_params], so a fresh
fit broadcasts one init while a resume passes each chain's final_position +
inv_metric_out (with n_warmup = 0) — composing with #29 to continue a whole
multi-chain fit. The OpenMP loop now lives in one pure-C++ core
(run_hmc_parallel_chains_cpp) shared by the C ABI and the existing
Rcpp-returning run_hmc_parallel_chains. New generic R entry point
cpp_tulpa_fit_generic_chains() returns draws stacked chain-major with a
chain_id vector — the layout mcmc_diagnostics() (#26) consumes directly —
plus per-chain epsilon / inv_metric / final_position. Validated in
tests/testthat/test-generic-sampler.R, including a cross-chain Rhat/ESS check
through mcmc_diagnostics(). ABI bump 23 -> 24 (new callable only; no
struct layout change).
feat(nuts): the NUTS C-ABI now returns the state needed to resume or
warm-start a chain (gcol33/tulpa#29). NUTSResult gains inv_metric_out
(the adapted inverse-mass diagonal at end of warmup) and final_position
(the last raw sampler state); epsilon was already returned. Feeding them
back as init + inv_metric_diag with n_warmup = 0 continues the chain
from the previous fit's geometry instead of rediscovering it. The inputs
already existed on NUTSFn; only the result fields were missing. The
generic R entry point cpp_tulpa_fit_generic() gains optional init /
inv_metric_init arguments and returns inv_metric / final_position,
exercised in tests/testthat/test-generic-sampler.R. ABI bump 22 -> 23
(two trailing pointers appended to NUTSResult; NUTSFn unchanged).
feat(diagnostics): extend the native MCMC convergence surface
(R/convergence.R) toward posterior parity (gcol33/tulpa#26).
mcmc_diagnostics() gains measures and probs arguments selecting from
improved rhat (now the maximum of rank-normalized split-Rhat and folded
split-Rhat, matching posterior::rhat), rhat_bulk, rhat_fold,
ess_bulk, ess_tail, ess_mean, ess_sd, mcse_mean, mcse_sd, and
per-probability ess_quantile / mcse_quantile. The default columns
(rhat, ess_bulk, ess_tail) are unchanged. New measures are registered
in one table (.tulpa_diag_measures), so adding a statistic is a one-line
change. Two estimator bugs are fixed so the native code reproduces
posterior to machine precision (~1e-12): the rank-normalization divisor is
now the Blom S + 1/4 (was S - 1/4) and the Geyer tau_hat tail term no
longer double-counts. New exported helpers: tulpa_draws_array() (an
as_draws_array()-style [iter, chain, param] accessor), n_divergent(),
and check_diagnostics(). The plotting / summary layer (plot_rhat,
plot_ess, diagnostic_summary, plot_diagnostics, plot_acf,
plot_pairs) now resolves the previously undefined get_draws_array(),
grep_params(), and n_divergent() helpers and runs end-to-end on a
multi-chain fit. Validated in tests/testthat/test-convergence.R.
feat(re): random-effect blocks support an optional per-term intercept.
ModelData::re_has_intercept (default: all terms carry the implicit group
intercept) lets a term be slope-only (lme4 (0 + x | g)): every
re_n_coefs[t] coefficient is a slope read from the slope design matrix and
there is no z = 1 column. The change is threaded through the design lookup
(slope_at() / obs_re_contrib() in laplace_spec.cpp) and the autodiff
RE contribution (log_post_generic_impl.h), so the value and its gradient
stay consistent. re_term_has_intercept() (in model_data.h) centralises
the per-term test. ABI bump 21 -> 22 (new ModelData field). Enables
gcol33/tulpaObs#10 slope-only bar syntax.
fix(spatial): spatial_car() / spatial_bym2() /
spatial_car_proper() with level = "group" now accept datasets that
cover only a subset of adjacency cells. Closes gcol33/tulpa#25. The
Besag / ICAR / BYM2 / proper-CAR field is well-defined on every node
of the graph regardless of whether each node has an observation;
unobserved cells simply contribute no likelihood term (matching
INLA's f(cell, model = "besag", graph = g)). validate_spatial()
and prior_from_spec() now resolve group_var to 1-based adjacency
row indices via a new .resolve_spatial_idx() helper:
group_var -> 1-based row indices,
validated against [1, n_spatial_units].group_var with rownames(adjacency) set ->
matched by name (preserves cell identity for sparse subsets).character / factor group_var without rownames -> legacy
as.integer(as.factor(.)), retained for back-compat; errors with
an actionable message when level count differs from adjacency
size.
refactor(joint-laplace): unify single-block and multi-block joint
dispatch (Phase J-E). tulpa_nested_laplace_joint()'s single-block
path (prior = list(type = "bym2"/"icar"/"car_proper", ...)) now
packs the prior into a length-1 blocks_spec and dispatches through
the same cpp_nested_laplace_joint_multi entry that drives the
list-of-blocks API. The three legacy cpp_nested_laplace_joint_bym2
/ _icar / _car_proper R-facing wrappers (524 lines in
src/nested_laplace_joint.cpp) are deleted; the inner Newton driver
(run_multi_block_nested_laplace_joint) was already shared, so the
refactor is a routing change with bit-identical log_marginal on every
joint test (147/147 pass). User-facing R API and result shape are
unchanged. C ABI: the tulpa_nested_laplace_joint_bym2 shim
(R_RegisterCCallable entry in tulpa_shims.cpp) is removed;
external embedders should call tulpa_nested_laplace_joint() from
R or build a shim on top of cpp_nested_laplace_joint_multi.
TULPA_ABI_VERSION bumped 19 → 20.
fix: joint nested-Laplace reparam (sigma, alpha) → (sigma_occ, sigma_pos).
Closes gcol33/tulpa#18. The BYM2 / ICAR / CAR_proper backends of
tulpa_nested_laplace_joint() previously parameterized the joint
outer grid as (sigma, rho/rho_car, alpha), where sigma was the
shared field amplitude and alpha scaled the copy arm's contribution.
At small n_pos and low cover-arm sample fraction (e.g. d7 Cell B,
n_s = 25, n_pos ≈ 46), the cover-arm likelihood identified only the
product alpha * sigma; sigma was pulled toward its prior and alpha
inflated to compensate (~ −15% sigma bias, ~ +27% alpha bias on 30
seeds). The reparam scales each arm's contribution to a unit-precision
latent by its own sigma — eta_arm = X beta + sigma_arm * z_s, with
sigma_arm = sigma_occ on donor arms and sigma_arm = sigma_pos on
the copy arm. Each axis is now anchored by its own arm's likelihood,
so the posterior ridge along constant alpha * sigma disappears.
alpha = sigma_pos / sigma_occ is recovered post-hoc and attached to
theta_grid / theta_mean / theta_sd. API: copy$alpha_grid is
superseded by copy$sigma_pos_grid; alpha_grid still works with a
deprecation warning that translates it to
alpha_grid * median(prior$sigma_grid). ICAR / CAR_proper joint
kernels now take sigma_grid (donor sigma in sigma-space) instead of
tau_grid; tulpaObs callers translate tau_grid to
sigma_grid = 1/sqrt(tau_grid) internally. C ABI:
tulpa_nested_laplace_joint_bym2_impl switches its grid args from
sigma_spatial_grid / alpha_grid to sigma_occ_grid /
sigma_pos_grid. TULPA_ABI_VERSION bumped 18 → 19.
feat: EM+Laplace MI and Gibbs corrections. tulpa_em_laplace() gains two
post-EM correction modes (correction = "mi" / "gibbs") that replace
the previous "not yet implemented" stub. MI draws n_imputations hard
z's from the converged posterior weights P(z|y, theta_hat), refits
each block on the hard draws, and pools per-submodel coefficients via
rubins_pool(). Gibbs runs a warm-started z|theta -> theta|z Markov
chain of length n_gibbs starting from the EM fits — every step
refreshes weights via the user's e_step, draws hard z, refits — and
pools the chain via Rubin's rules. The fixed-effect (beta, se)
extraction is now a shared helper (.attach_beta_se) consumed by both
the new corrections and the existing tulpa_em_mc() MCEM driver, so
there is one source of truth for "Laplace fit -> Rubin pool input".
Bernoulli is the default per-observation hard-z draw; multi-class
latent structures supply their own via the new draw_z callback.
Return shape gains correction, pooled, and draws fields when a
correction is requested. No ABI bump (R-side only); closes
TODO.md P3.8.
feat: ABI v14 — SPDE nested-Laplace upgraded to the v10-style universal
shim (store_modes, store_Q, paired range/sigma grids, formula-side iid-RE
block). Replaces the v0 cpp_nested_laplace_spde entry and folds the
SPDE C-callable into the shared NestedLaplaceShimResult block used by
ICAR / BYM2 / NNGP / HSGP. The dedicated SpdeNestedLaplaceShimResult
struct is removed. Latent layout:
[beta (p)] [re (n_re_groups)] [w_mesh (n_mesh)].
TULPA_ABI_VERSION bumped 13 → 14; downstream packages must rebuild.
Closes the tulpaRatio migration tracker (gcol33/tulpa#15). After v12
gated the legacy ratio body of compute_log_post_impl<T> behind a
generic-layout check, Phase D removes the body itself and every
consumer of LegacyRatioData / LegacyRatioLayout.
TULPA_ABI_VERSION bumped 12 → 13. Downstream packages must
rebuild against the v13 headers.ModelData::LegacyRatioData legacy
(inst/include/tulpa/model_data.h) and
ParamLayout::LegacyRatioLayout legacy
(inst/include/tulpa/param_layout.h) are gone. n_processes > 0
with a non-null data.likelihood_spec is now the only supported
configuration.cpp_hmc_fit, cpp_hmc_fit_gp,
cpp_hmc_fit_gp_v2, cpp_ess_fit, cpp_ess_get_n_params,
cpp_vi_fit, cpp_vi_get_n_params, cpp_sghmc_fit, cpp_sgld_fit,
cpp_compute_log_post_test, cpp_compute_log_prior_test,
cpp_compute_log_lik_only_test, cpp_log_post_split_n_params.
Internal samplers (run_ess_sampler, run_sghmc_sampler, fit_vi,
run_mclmc_sampler) and their C-callable shims
(tulpa_run_ess_sampler, tulpa_sghmc_fit, tulpa_fit_vi,
tulpa_mclmc_fit) remain — downstream packages reach them via the
generic ModelData/ParamLayout API. Dev tools
tools/icar_collapsed_check.R and tools/bym2_gradient_check.R
also removed.resolve_gradient_fn
(src/hmc_gradient_dispatch.h) now only resolves the generic
spec->gradient_fn / compute_gradient_generic_arena /
compute_gradient_generic_numerical paths. Mode overrides
(AUTODIFF_TAPE, AUTODIFF_ARENA, AUTODIFF_FWD) and the H-mode
specialized fallthroughs are gone. Callers reaching the dispatcher
with n_processes == 0 get Rcpp::stop with a pointer to this
entry. hmc_gradient_dispatch_predicates.h deleted.compute_log_post, compute_log_prior, compute_log_lik_only
(src/hmc_sampler.cpp) now forward to
compute_log_post_generic_spec_double; the
accumulate_log_prior_and_state / accumulate_obs_log_lik body
and its 5 hmc_sampler_log_prior_*.h fragments are gone, along
with hmc_log_posterior_split.h. compute_log_post_impl<T>
(src/log_post_impl.h) reduces to the same forward for
T = double and a defensive T(0) no-op for autodiff T (arena
AD now routes through compute_log_post_generic<Var>).hmc_icar_collapsed_* ×9, hmc_gp_collapsed_* ×5),
the legacy ratio likelihood (hmc_likelihood.h,
hmc_observation_likelihood.h), and the legacy fallback gradients
(compute_gradient_numerical / _autodiff / _arena / _forward
/ _numerical_impl) are deleted. The 6 log_post_impl_*_block.h
fragments and the 2 Rcpp ModelData populators
(model_data_rcpp.h, hmc_modeldata_builders.h) follow.
verify_gradient_runtime now always uses
compute_gradient_generic_numerical as the reference.compute_param_layout
(src/hmc_param_layout.cpp) requires n_processes > 0; model
packages place model-specific scalars (overdispersion etc.) in the
LikelihoodSpec extra-parameter block at layout.extra_offset.
ESS's build_gaussian_priors and get_non_gaussian_params
(src/ess_sampler.h) walk process_beta_start and
extra_offset only.
hmc_nuts_mass_init.cpp drops the family-specific block-spec
heuristics (NB+ICAR / Bin+ICAR forced DENSE, NB phi-pair 2×2
block) — re-introducing them would need a LikelihoodSpec hint.src/hmc_nuts_chain_iter_nuts.h) reconstructed eta from
data.legacy.X_num_flat and branched on the legacy ModelType.
ST_IV chains now fall back to the adapted DIAG mass matrix until
the override is re-expressed through spec->eta_weights_fn. One
no-op per chain at warmup end; practical impact on sampling
efficiency is small.tests/testthat/test-log-post-split.R
and tests/testthat/test-hmc-modeldata-builders.R are deleted (every
test was a Phase-D skip). The legacy-ratio gradient-check test in
test-spatial-car-proper.R is removed; the two R-side
spatial_car_proper() construction tests stay.src/,
inst/include/tulpa/, tests/, and tools/. Net code reduction
before the v13 maintenance window starts.Downstream rebuild notes:
tulpaRatio already routes through the generic LikelihoodSpec
path via tulpa_bridge.cpp + per-family payloads in lik_specs/
(B1+B2 of the migration); rebuild against v13 headers, no logic
changes needed.
tulpaObs never used the legacy ratio path; rebuild against v13.
* tulpaGlmm Day-22+ already targets the generic path; rebuild
against v13.
TULPA_ABI_VERSION bumped 11 → 12. Downstream packages must
rebuild against the v12 headers.compute_log_post_impl<T> (src/log_post_impl.h)
now early-returns to compute_log_post_generic_spec_double when
the caller built ModelData with n_processes > 0 and a non-null
likelihood_spec. Previously the function reached lines 83-84 and
unconditionally read params[layout.legacy.beta_num_start], which
is params[-1] for generic-layout callers — segfault. This was the
blocker for tulpaGlmm Day-22 inference = "ess" (see deferred
fix.md entry from 2026-05-06). The early-return makes the function
safe for both layouts; the legacy ratio body remains in place for
n_processes == 0 callers (i.e. nobody outside this file at the
moment, but tulpaRatio's hmc_sampler.cpp keeps its own copy).tulpa_ess::build_gaussian_priors
(src/ess_sampler.h) now walks every process's β block
(layout.process_beta_start[k] for k in 0..n_processes) when
data.n_processes > 0, instead of only layout.legacy.beta_num_start
/ beta_denom_start. Previously generic-layout ESS produced an
empty β prior block and β was never sampled.tulpa_ess::get_non_gaussian_params now appends every parameter
in [layout.extra_offset, layout.extra_offset + n_extra_params)
to the RWMH list. LikelihoodSpec authors pack their model-specific
scalars (e.g. log_phi for negative-binomial, log_sigma for Gaussian)
into that block; ESS now walks them. Legacy ratio log_phi_num /
log_phi_denom indices remain in the list for n_processes == 0.LegacyRatioData / LegacyRatioLayout (inst/include/tulpa/model_data.h,
param_layout.h) are still exported but stay deprecated — the
in-engine consumers are the H-mode gradient kernels, the legacy
AD fallback (hmc_gradient_fallback.cpp), the composite gradient,
and tulpa_hmc::compute_log_post inside hmc_sampler.cpp. None
of those are reached when the dispatcher (hmc_gradient_dispatch.h)
sees n_processes > 0. Full removal is a follow-up cut after the
collapsed-spatial double-evaluator (MCLMC / SGHMC consumer) is
reworked.tulpa_bridge.cpp + per-family payloads in
lik_specs/; it never touched ModelData::legacy and rebuilds
cleanly against v12 headers. tulpaGlmm Day-22 ESS shim can now
call tulpa::get_ess_fn()(...) end-to-end on a generic-layout
ModelData without segfaulting.TULPA_ABI_VERSION bumped 10 → 11. Downstream packages must
rebuild against the v11 headers.tulpa_run_nuts_generic (nuts_api.h
NUTSFn) gains a new positional parameter const double*
inv_metric_diag immediately before NUTSResult* result_out. Pass
nullptr to keep the v10 behaviour (default structural warm-start
of the mass matrix). Pass a length-n_params vector to seed the
diagonal inverse-mass — useful for warm-starting NUTS from an
analytical-approximation method (Laplace, VI, etc.).run_hmc_chain_cpp / run_hmc_chain (hmc_sampler_funcs.h) take a
matching trailing inv_metric_init std::vector<double> (default
empty). Within run_hmc_chain_cpp (hmc_nuts_chain_setup.h), a
non-empty caller diagonal overrides the structural diagonal set by
warm_start_mass_matrix. Values are clamped to [1e-3, 1e3]
before being installed via mass.set_diagonal, then
find_reasonable_epsilon re-runs against the seeded metric.run_hmc_chain_cpp (hmc_nuts_parallel.cpp ×3,
tulpa_generic_sampler.cpp ×1) continue to pass no inv_metric_init
via the default empty vector; the local forward declaration in
tulpa_generic_sampler.cpp was updated to match the canonical
declaration's parameter list.tulpaGlmm exercises this end-to-end via
Day-32's hmc_warm_start = "laplace" argument. tulpaObs and
tulpaRatio need to be reinstalled against v11 — both already
updated to pass nullptr for the new parameter (no logic change).laplace_mode_gp (and the spatial-only / ST-combo NNGP entries in
nested_laplace.cpp) now assemble the full NNGP precision matrix
Λ = (I - A)' D⁻¹ (I - A) in every Newton iteration, replacing the
diagonal-on-w approximation that only kept 1/v_i on the focal
diagonal of each row.+a_{i,k}·q_i/v_i), the off-diagonal Hessian entries
(focal, neighbour_k) and (neighbour_k, neighbour_kp), and the
pairwise precision between members of every conditioning set. The
Newton mode for w was therefore shrunk toward zero and pointwise
field recovery on smooth latent fields collapsed (cor ≈ 0).gpu_nngp_laplace.h:batch_nngp_scatter(..., alpha_out = nullptr) — backward-compatible
extra optional output capturing the per-row conditional regression
weights (already computed internally; just exposed).apply_nngp_full_prior_dense — scatters the full precision
contribution into a dense (grad, H) pair via the alpha + cv
bundle.apply_nngp_full_prior_sparse — same, into a SparseHessianBuilder.make_nngp_prior_sparsity_pattern — emits the (row, col) pairs
required to back the sparse path.laplace_mode_gp dense Newton,
laplace_mode_gp sparse Newton (with pattern expansion), the
spatial-only cpp_nested_laplace_nngp scatter lambda, and
make_nngp_spatial_ops::add_prior_at_k (the ST-combo NNGP block).
Log-prior calls (log_prior lambdas) are unchanged — they only need
cm and cv, and the existing batch_nngp_scatter signature still
supports that without alpha_out.cor(w_mean, f_true) jumps from near zero to ≈ 0.81 Pearson on
a 120-location Poisson + smooth-GP simulation. β recovery unchanged.TULPA_ABI_VERSION bump (still v10). Public shim
signatures are unchanged; only the inner Laplace scatter is upgraded.
Downstream packages must rebuild against this commit to pick up the
new behaviour (no source changes required).run_two_indexed_nested_laplace driver and joint
inner Newton introduced earlier today:tulpa_nested_laplace_st_icar_rw1tulpa_nested_laplace_st_icar_rw2tulpa_nested_laplace_st_car_proper_rw1tulpa_nested_laplace_st_car_proper_rw2tulpa_nested_laplace_st_car_proper_ar1
Each routes through a per-combo Rcpp entry plus a C-callable
_impl wrapper; matching typedefs + getters live in
nested_laplace_api.h.IndexedPriorOps in nested_laplace.cpp
with per-kind builders make_icar_ops, make_car_proper_ops,
make_rw1_ops, make_rw2_ops, make_ar1_ops. The shared
run_two_indexed_nested_laplace driver now consumes
std::function-typed callbacks, so adding the next indexed × indexed
combination is a few lines of Rcpp glue rather than a re-derivation.cpp_nested_laplace_st_icar_ar1 to use the new factories
(identical behavior; just dropped the inline lambdas).TULPA_ABI_VERSION bump (still v9). The new shims are
resolved via R_GetCCallable at first use; downstream packages
rebuilt against ABI v9 pick them up automatically.tulpa_nested_laplace_st_icar_ar1 for joint nested-Laplace
inference with an ICAR spatial field AND an AR1 temporal field in the
same fit. The joint inner Newton solves over the full latent vector
[beta] [re] [w_spatial (n_s)] [w_temporal (n_t)] at each grid point;
the cross-block H[w_s, w_t] is non-zero, so the two fields cannot be
Laplace-marginalized separately. The hyperparameter grid is supplied
caller-side as paired vectors of length n_grid (Cartesian product of
τ_spatial × τ_temporal × ρ_temporal built on the R side).NestedLaplaceStIcarAr1Fn + getter
get_nested_laplace_st_icar_ar1_fn() in nested_laplace_api.h.nested_laplace.cpp: the templated
run_two_indexed_nested_laplace driver and helpers
nl_compute_eta_two_indexed / nl_scatter_obs_two_indexed. These are
the shared substrate for the remaining 11 (spatial_kind × temporal_kind)
combinations.TULPA_ABI_VERSION bumped 8 → 9. Downstream packages must be rebuilt
against this header set.tulpa_nested_laplace_hsgp now sets store_modes = 1 (was 0) and
gained a store_Q flag matching the rest of the nested-Laplace family.
The basis-coefficient latent [beta] [re] [beta_M (n_basis)] is
returned per (σ², ℓ) grid point, and with store_Q = 1 the joint Q
at the mode is retained in the standard NestedLaplaceShimResult::Q_*_flat
slots.cpp_nested_laplace_hsgp gained a trailing bool store_Q = false
argument and now passes store_modes = true to the grid driver. The
C-callable tulpa_nested_laplace_hsgp_impl signature picks up the
matching int store_Q parameter; the public typedef
NestedLaplaceHsgpFn in nested_laplace_api.h is updated to match.f_i = Σ_j Φ_ij · √S(λ_j; σ²_k, ℓ_k) · β_M_j
can be reconstructed caller-side from modes + posterior draws over the
basis coefficients plus the per-draw grid index.TULPA_ABI_VERSION bumped 7 → 8. Downstream packages (tulpaGlmm,
tulpaObs) must be rebuilt against the updated headers.tulpa_nested_laplace_bym2 now sets store_modes = 1 (was 0) and
gained a store_Q flag matching the rest of the nested-Laplace family.
The reparameterised latent [beta] [re] [phi (n_spatial)] [theta
(n_spatial)] is returned per grid point, and with store_Q = 1 the
joint Q at the mode is retained in the standard
NestedLaplaceShimResult::Q_*_flat slots.cpp_nested_laplace_bym2 gained a trailing bool store_Q = false
argument and now passes store_modes = true to the grid driver. The
C-callable tulpa_nested_laplace_bym2_impl signature picks up the
matching int store_Q parameter; the public typedef
NestedLaplaceBym2Fn in nested_laplace_api.h is updated to match.w_s = σ·(√ρ · scale · φ_s + √(1−ρ) · θ_s) can be
reconstructed caller-side from modes + posterior draws over the
(σ, ρ) grid.TULPA_ABI_VERSION bumped 6 → 7. Downstream packages (tulpaGlmm,
tulpaObs) must be rebuilt against the updated headers.tulpa_nested_laplace_rw1, tulpa_nested_laplace_rw2,
tulpa_nested_laplace_ar1, and tulpa_nested_laplace_car_proper now
accept a store_Q flag (matching the ICAR shim added in v5). When set
the shim retains the joint negative-Hessian Q at each grid point's mode
in NestedLaplaceShimResult::Q_*_flat, so downstream packages can draw
mixture-of-MVN posteriors sum_k w_k · N(mode_k, Q_k^{-1}) without
re-doing the Newton assembly R-side.cpp_nested_laplace_<rw1|rw2|ar1|car_proper> entries
gained a trailing bool store_Q = false argument. Default is false,
so existing callers that don't ask for Q keep the previous behaviour
and footprint.TULPA_ABI_VERSION bumped 5 → 6. Downstream packages (tulpaGlmm,
tulpaObs) must be rebuilt against the updated headers.tulpa::takahashi_partial_inverse_dense(n, Lp, Li, Lx,
Z_out) in sparse_cholesky.{h,cpp} runs the Takahashi recursion on a
caller-supplied lower-triangular L (CSC) and writes a dense column-major
n*n Z with Q^{-1} on pattern(L + L^T) and zeros elsewhere. A
matching takahashi_partial_inverse_csc returns just the Zx values on
pattern(L). The existing SparseCholeskySolver::selected_inversion_diagonal
now routes through the new helper so there is one source of truth for the
recursion (no copy-paste).tulpa_takahashi_partial_inverse_dense exposes the
pure-function variant to downstream packages. Resolved via
tulpa::get_takahashi_partial_inverse_dense_fn() in
inst/include/tulpa/sparse_solver_api.h; the getter Rf_errors if the
symbol is missing (i.e. caller built against newer headers than the loaded
tulpa).TULPA_ABI_VERSION stays at 4. Downstream
packages that want the new shim need only rebuild against the updated
sparse_solver_api.h.tulpa_laplace_spec_dense (and its public C ABI shim) now accepts the
full multi-term, multi-coefficient RE structure populated by populate_re
in downstream model packages: K = data.n_re_terms random-effect terms,
each with q_t = re_n_coefs[t] coefficients per group (intercept-only
when q_t == 1, intercept + slopes when q_t > 1), uncorrelated
((x||g)) or correlated ((x|g)) prior covariance. Per-process
sharing is uniform across terms via data.sharing.re.layout.re_start / layout.re_end /
log_sigma_re_idx with data.n_re_terms == 0) is preserved and stays
bit-identical numerically.result_out->mode writeout from tulpa_laplace_spec_dense_impl
now concatenates every RE term's block in term order
(re_start_multi[t]..re_end_multi[t]), matching the new
SpecLatentLayout ordering.cpp_laplace_spec_test_multi_re exercises
the multi-term path end-to-end against hand-derived linear-Gaussian
reference solutions; new tests in tests/testthat/test-laplace-spec.R
cover (a) two crossed intercept-only terms, (b) one correlated random
slope, (c) one uncorrelated random slope.TULPA_ABI_VERSION bumped from 3 → 4. Downstream packages that ship
with the spec-Laplace dispatcher (e.g. tulpaGlmm) must be rebuilt
against the new headers.Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.