kedd: kernel estimation of a density and its derivatives

knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 6,
                      fig.height = 3.8, fig.align = "center", dpi = 96)
library(kedd)
## plot() methods print their figure; quiet() keeps the returned ggplot without
## drawing it, so that layers can be added before display.
quiet <- function(expr) { grDevices::pdf(NULL); on.exit(grDevices::dev.off()); expr }

kedd logo

1. Introduction

Given an i.i.d. sample $X_1, \dots, X_n$ from a density $f$, the kernel estimator of the $r$-th derivative $f^{(r)}$ is $$ \hat f_h^{(r)}(x) \;=\; \frac{1}{n\,h^{r+1}} \sum_{i=1}^{n} K^{(r)}!\left(\frac{x - X_i}{h}\right), \qquad r = 0, 1, 2, \dots $$ where $K$ is a kernel and $h > 0$ the bandwidth. Its asymptotic mean integrated squared error is $$ \mathrm{AMISE}(h; r) \;=\; \frac{R\big(K^{(r)}\big)}{n\,h^{2r+1}} + \frac{\mu_2(K)^2}{4}\, h^4\, R\big(f^{(r+2)}\big), \qquad R(g) = \int g^2,\quad \mu_2(K) = \int x^2 K(x)\,dx, $$ so the optimal bandwidth is of order $n^{-1/(2r+5)}$: estimating a derivative requires a larger bandwidth than estimating the density itself. The whole difficulty is the choice of $h$, which is what the package is about.

kedd provides

All results are S4 objects with show() and ggplot2 plot() methods, and the $O(n^2)$ criteria run in C++ (Rcpp) in parallel (OpenMP).

2. Kernels

2.1 The kernels

Ten named kernels and the power family Pow(s,t) (section 2.2) are available in every function through the argument kernel. "r max" is the highest derivative order that is not identically zero.

| kernel | $K(x)$ | support | r max | |---|---|---|:---:| | gaussian | $\phi(x) = \frac{1}{\sqrt{2\pi}} e^{-x^2/2}$ | $\mathbb{R}$ | any | | epanechnikov | $\frac34 (1-x^2)$ | $[-1,1]$ | 2 | | uniform | $\frac12$ | $[-1,1]$ | 0 | | triangular | $1-\lvert x\rvert$ | $[-1,1]$ | 1 | | biweight | $\frac{15}{16}(1-x^2)^2$ | $[-1,1]$ | 4 | | triweight | $\frac{35}{32}(1-x^2)^3$ | $[-1,1]$ | 6 | | tricube | $\frac{70}{81}(1-\lvert x\rvert^3)^3$ | $[-1,1]$ | 9 | | cosine | $\frac{\pi}{4}\cos\frac{\pi x}{2}$ | $[-1,1]$ | any | | silverman | $\frac12 e^{-\lvert x\rvert/\sqrt2}\sin!\big(\frac{\lvert x\rvert}{\sqrt2}+\frac{\pi}{4}\big)$ | $\mathbb{R}$ | any | | logistic | $\frac{e^{-x}}{(1+e^{-x})^2}$ | $\mathbb{R}$ | 20 | | pow(s,t) | $c_{s,t}(1-\lvert x\rvert^s)^t$, $c_{s,t} = \frac{s}{2B(1/s,\,t+1)}$ | $[-1,1]$ | $\min(st,16)$ |

For pow(s,t), $\min(st, 16)$ applies when $s$ and $t$ are integers (16 otherwise). The bandwidth selectors need $R(K^{(r)}) < \infty$ (and MCV, CCV use $K^{(r+1)}$, $K^{(r+2)}$), so their maximum order is lower. Above it they return h = NA with a warning; "—" means not available even at $r = 0$:

| kernel | h.ucv, h.bcv, h.tcv, h.amise | h.mcv | h.ccv | |---|:---:|:---:|:---:| | gaussian, silverman | any | any | any | | cosine | 1 | 0 | — | | logistic | 8 | 8 | 8 | | epanechnikov | 1 | 0 | — | | uniform | 0 | — | — | | triangular | 0 | — | — | | biweight | 2 | 1 | 0 | | triweight | 3 | 2 | 1 | | tricube | 3 | 2 | 1 | | pow(s,t) | $\min(r^\ast, 6)$ | $\min(r^\ast-1, 6)$ | $\min(r^\ast-2, 6)$ |

Here $r^\ast$ is the largest integer $r < t + \frac12$ (with $2r \le st$ when $s$ and $t$ are integers, and $r < s + \frac12$ when $s$ is not an even integer); e.g. pow(2,3): 3, 2, 1. h.scv (Gaussian kernel) has no limit and h.mlcv is for $r = 0$ only.

kernel.fun() evaluates $K^{(r)}$ on a grid (or at given points) and returns an object of class kfun:

kf <- kernel.fun(kernel = "biweight", deriv.order = 1L)
kf
plot(kf)

kernel.conv() returns $K^{(r)} * K^{(r)}$ (class kconv), the building block of the cross-validation criteria. At $0$ it equals $(-1)^r R(K^{(r)})$:

kc <- kernel.conv(kernel = "logistic", deriv.order = 1L)
plot(kc)
kernel.conv(x = 0, kernel = "logistic", deriv.order = 1L)@y   # -R(K') = -1/30

The constants used by the selectors, $\mu_2(K)$, $\mu_4(K)$ and $R(K^{(r)})$, are available from the low-level routine Cpp_kernel_constants():

kers <- c("gaussian", "epanechnikov", "uniform", "triangular", "biweight",
          "triweight", "tricube", "cosine", "silverman", "logistic",
          "pow(2,4)", "pow(1.5,2.5)")
round(t(sapply(kers, function(k) Cpp_kernel_constants(k, 0L)[c("mu2", "mu4", "R")])), 5)

The Silverman kernel is of order 4 ($\mu_2 = 0$, $\mu_4 < 0$); all the others are second-order kernels.

2.2 The power family Pow(s,t)

For $s > 0$ and $t \ge 0$, $$ K_{s,t}(x) = c_{s,t}\,(1 - \lvert x\rvert^s)^t\, \mathbf 1_{\lvert x\rvert \le 1}, \qquad c_{s,t} = \frac{s}{2\,B(1/s,\, t+1)}, $$ is passed as the string "pow(s,t)", or built with kernel.pow(s, t). It contains uniform (pow(s,0)), triangular (pow(1,1)), Epanechnikov (pow(2,1)), biweight (pow(2,2)), triweight (pow(2,3)) and tricube (pow(3,3)). Its constants are exact: $\mu_k(K) = B\big(\tfrac{k+1}{s}, t+1\big) / B\big(\tfrac1s, t+1\big)$ and $R(K) = s\,B(1/s, 2t+1) / \big(2\,B(1/s, t+1)^2\big)$.

kernel.pow(2, 3)
all.equal(kernel.fun(kernel = "pow(2,2)")@y, kernel.fun(kernel = "biweight")@y)
Cpp_kernel_constants(kernel.pow(1.5, 2.5), 0L)
plot(kernel.fun(kernel = kernel.pow(1.5, 2.5)))

Increasing $t$ (or decreasing $s$) concentrates the kernel around $0$:

grid <- seq(-1, 1, length.out = 201)
shapes <- do.call(rbind, lapply(c("pow(2,1)", "pow(2,4)", "pow(4,1)", "pow(1,2)"),
  function(k) data.frame(x = grid, K = Cpp_kernel_deriv(k, grid, 0L), kernel = k)))
ggplot2::ggplot(shapes, ggplot2::aes(x, K, colour = kernel)) +
  ggplot2::geom_line(linewidth = 0.9) + ggplot2::theme_minimal()

3. Estimating a density and its derivatives

We use a bimodal sample whose true density is known, $f = 0.6\,N(-1, 0.5^2) + 0.4\,N(1.5, 0.7^2)$:

set.seed(2027)
n <- 300
x <- c(rnorm(0.6 * n, -1, 0.5), rnorm(0.4 * n, 1.5, 0.7))
f_true <- function(t, r = 0L) {
  comp <- function(m, s) {
    z  <- (t - m) / s
    He <- switch(r + 1L, 1, z, z^2 - 1)        # Hermite polynomials
    (-1)^r * He * dnorm(z) / s^(r + 1)
  }
  0.6 * comp(-1, 0.5) + 0.4 * comp(1.5, 0.7)
}

dkde() computes $\hat f_h^{(r)}$ (class dkde). When h is missing, the UCV bandwidth is used.

f0 <- dkde(data = x)
f0
p <- quiet(plot(f0))
p + ggplot2::stat_function(fun = f_true, linetype = "dashed")

The first derivative, estimated with its own (SCV) bandwidth (dashed: true $f'$):

h1 <- h.scv(x, deriv.order = 1L)
p1 <- quiet(plot(dkde(data = x, h = h1@h, deriv.order = 1L)))
p1 + ggplot2::stat_function(fun = f_true, args = list(r = 1L), linetype = "dashed")

Compare with the density bandwidth, which is too small for $f'$:

c(h_density = f0@h, h_derivative = h1@h)
plot(dkde(data = x, h = f0@h, deriv.order = 1L), band = FALSE)

Every kernel can be used, each with its own bandwidth. Here the Gaussian, logistic and power pow(2,4) kernels with UCV bandwidths, against the true density (dashed):

grid <- seq(-3, 4, length.out = 300)
est <- do.call(rbind, lapply(c("gaussian", "logistic", "pow(2,4)"), function(k) {
  d <- dkde(x = grid, data = x, h = h.ucv(x, kernel = k)@h, kernel = k)
  data.frame(x = d@x, y = d@y, kernel = k)
}))
ggplot2::ggplot(est, ggplot2::aes(x, y, colour = kernel)) +
  ggplot2::geom_line(linewidth = 0.8) +
  ggplot2::stat_function(fun = f_true, colour = "black", linetype = "dashed") +
  ggplot2::theme_minimal()

4. Bandwidth selectors

All selectors return an object of class bandwidth (slots h, min.obj, hos, method, ...). Writing $\psi_h(u) = u/h$ for the scaled differences $X_j - X_i$, the criteria minimised are:

bw <- h.ucv(x)
bw
plot(bw)

lines() returns a ggplot2 layer, so several criteria can be drawn together:

p <- quiet(plot(h.ucv(x)))
p$plot +
  lines(h.bcv(x), color = "darkred") +
  lines(h.ccv(x), color = "darkorange") +
  lines(h.mcv(x), color = "purple")

4.1 Comparison on the bimodal sample

The integrated squared error $\mathrm{ISE}(h) = \int (\hat f_h^{(r)} - f^{(r)})^2$ measures the quality of each selected bandwidth:

ise <- function(h, r) {
  d <- dkde(x = seq(-4, 5, length.out = 400), data = x, h = h, deriv.order = r)
  sum((d@y - f_true(d@x, r))^2) * diff(d@x[1:2])
}
sel <- list(AMISE = h.amise, UCV = h.ucv, BCV1 = h.bcv,
            BCV2 = function(x, ...) h.bcv(x, whichbcv = 2, ...),
            CCV = h.ccv, MCV = h.mcv, TCV = h.tcv, SCV = h.scv)
tab <- sapply(0:1, function(r) {
  h <- sapply(sel, function(f) f(x, deriv.order = r)@h)
  c(rbind(h = h, ISE = sapply(h, ise, r = r)))
})
dimnames(tab) <- list(paste(rep(names(sel), each = 2), c("h", "ISE")),
                      c("r = 0", "r = 1"))
round(tab, 4)
c(MLCV = h.mlcv(x)@h)   # density only

Two well-known behaviours show up in such comparisons:

4.2 Same criterion, different kernels

Bandwidths are not comparable across kernels: each kernel has its own scale ($\mu_2(K)$, $R(K)$). The selectors take this into account:

ks <- c("gaussian", "epanechnikov", "uniform", "triangular", "biweight", "triweight",
        "tricube", "cosine", "silverman", "logistic", "pow(2,4)", "pow(1.5,2.5)")
round(sapply(ks, function(k) h.ucv(x, kernel = k)@h), 4)

5. Example datasets

The package contains five samples of size 200 from the Marron and Wand (1992) normal mixtures (?"kedd-data"): kurtotic (#4), outlier (#5), bimodal (#7, separated bimodal), trimodal (#9) and claw (#10). Their true densities are known, which makes them convenient benchmarks. The claw density, for instance, is hard to estimate: the five narrow peaks need a small bandwidth.

f_claw <- function(x) 0.5 * dnorm(x) +
  rowSums(sapply(0:4, function(l) 0.1 * dnorm(x, l / 2 - 1, 0.1)))
bw_claw <- sapply(list(UCV = h.ucv, BCV1 = h.bcv, MCV = h.mcv, SCV = h.scv,
                       AMISE = h.amise), function(f) suppressWarnings(f(claw))@h)
round(bw_claw, 4)
p <- quiet(plot(dkde(data = claw, h = bw_claw[["UCV"]])))
p + ggplot2::stat_function(fun = f_claw, linetype = "dashed")

The same selectors on the five samples:

samples <- list(kurtotic = kurtotic, outlier = outlier, bimodal = bimodal,
                trimodal = trimodal, claw = claw)
round(sapply(samples, function(s)
  sapply(list(AMISE = h.amise, UCV = h.ucv, BCV1 = h.bcv, MCV = h.mcv, SCV = h.scv),
         function(f) suppressWarnings(f(s))@h)), 4)

6. Parallel computation

The $O(n^2)$ criteria (sums over all pairs $X_i, X_j$) are computed in C++ and split between the processor cores with OpenMP. The argument nthreads of every selector and of dkde() sets the number of threads of that call:

Cpp_omp_threads() reports how many threads a call actually gets. The results do not depend on nthreads, only the computing time does:

c(all = Cpp_omp_threads(0L), two = Cpp_omp_threads(2L))
y <- rnorm(1500)
system.time(h_seq <- h.ucv(y, nthreads = 1L)@h)
system.time(h_par <- h.ucv(y, nthreads = 0L)@h)
all.equal(h_seq, h_par)

7. Summary of functions

| function | purpose | returns | |---|---|---| | kernel.fun(x, deriv.order, kernel, n.grid) | $K^{(r)}(x)$ | kfun | | kernel.conv(x, deriv.order, kernel, n.grid) | $(K^{(r)} * K^{(r)})(x)$ | kconv | | kernel.pow(s, t) | name of a Pow(s,t) kernel | character | | dkde(x, data, h, deriv.order, kernel, nthreads) | $\hat f_h^{(r)}(x)$ | dkde | | h.amise(x, deriv.order, kernel) | AMISE rule | bandwidth | | h.ucv, h.bcv, h.ccv, h.mcv, h.tcv, h.scv (x, deriv.order, kernel, lower, upper, tol, nthreads) | cross-validation selectors | bandwidth | | h.mlcv(x, kernel, lower, upper, tol, nthreads) | likelihood CV (density) | bandwidth | | show(), plot() | print / ggplot2 figure | — | | lines(<bandwidth>) | criterion curve as a ggplot2 layer | layer | | kurtotic, outlier, bimodal, trimodal, claw | example samples | numeric | | Cpp_kernel_constants(), Cpp_omp_threads(), Cpp_*() | low-level C++ routines (?"kedd-lowlevel") | numeric |

8. Resources

References



Try the kedd package in your browser

Any scripts or data that you put into this service are public.

kedd documentation built on Nov. 12, 2026, 5:10 p.m.