# R output pre blocks are styled by default to indicate output knitr::opts_chunk$set(comment = NA, cache=TRUE) set.seed(97310) # shorthand for rd_link() - see ?packagedocs::rd_link for more information rdl <- function(x) packagedocs::rd_link(deparse(substitute(x)))
The dbmss package allows simple computation of spatial statistic functions of distance to characterize the spatial structures of mapped objects, including classical ones (Ripley's K and others) and more recent ones used by spatial economists (Duranton and Overman's $K_d$, Marcon and Puech's $M$). It relies on spatstat for some core calculation.
This vignette contains a quick introduction to the package (first section) and its reference guide (second section).
The main data format is wmppp
for weighted, marked point pattern.
It inherits from the ppp
class of the spatstat package.
A wmppp
object can be created from the coordinates of points, their type and their weight.
library("dbmss") # Draw the coordinates of 10 points X <- runif(10) Y <- runif(10) # Draw the point types. PointType <- sample(c("A", "B"), 10, replace=TRUE) # Plot the point pattern. Weights are set to 1 ant the window is adjusted. plot(wmppp(data.frame(X, Y, PointType)), which.marks=2)
An example dataset is provided: it is a point pattern from the Paracou forest in French Guiana. Two species of trees are identified, other trees are of type "Other". Point weights are their basal area, in square centimeters.
data(paracou16) # Plot (second column of marks is Point Types) plot(paracou16, which.marks=2, leg.side="right")
The main functions of the packages are designed to calculate distance-based measures of spatial structure. Those are non-parametric statistics able to summarize and test the spatial distribution (concentration, dispersion) of points.
The classical, topographic functions such as Ripley's K are provided by the spatstat package and supported by dbmss for convenience.
Relative functions are available in dbmss only. These are the $M$ and $m$ and $K_d$ functions.
The bivariate $M$ function can be calculated for Q. Rosea trees around V. Americana trees:
plot(Mhat(paracou16, , "V. Americana", "Q. Rosea"))
Confidence envelopes of various null hypotheses can be calculated. The univariate distribution of Q. Rosea is tested against the null hypothesis of random location.
plot(KdEnvelope(paracou16, , ReferenceType="Q. Rosea", Global=TRUE), main="")
Significant concentration is detected between about 10 and 20 meters.
Characterizing spatial distributions of objects concerns numerous researchers in various fields. Amongst many questions, ecologists have been addressing the spatial attraction between species [@Duncan1991] or the non-independence of the location of dead trees in a forest [@Haase1997]. Analyzing the spatial distribution of plants, economists may be concerned by the location of new entrants [@Duranton2008] or by the location of shops according to the types of good sold [@Picone2009]. In epidemiology, researchers want to identify the spatial distribution of sick individuals in comparison to the population [@Diggle1991]. In these fields of research, the point process theory undoubtedly helps dealing with these questions. Exploratory statistics of point patterns widely rely on Ripley's seminal work [@Ripley1976; @Ripley1977], namely the $K$ function. A review of similar methods has been made by @Marcon2012c who called them distance-based measures of spatial concentration. We will write spatial structures here since both dispersion and concentration can be characterized. They are considered as novel and promising tools in spatial economics [@Combes2008]. The traditional approach to detect localization, i.e. the degree of dissimilarity between the geographical distribution of an industry and that of a reference [@Hoover1936], relies on discrete space (a country is divided in regions for example) and measures of inequality between zones, such as the classical Gini [[email protected]] index or the more advanced Ellison and Glaeser [[email protected]] index. This approach suffers several limitations, mainly the Modifiable Areal Unit Problem (MAUP): results depend on the way zones are delimited and on the scale of observation [@Openshaw1979]. Distance-based methods have the advantage to consider space as continuous, i.e. without any zoning, allowing detecting spatial structures at all scales simultaneously and solve MAUP issues.
These methods estimate the value of a function of distance to each point calculated on a planar point pattern, typically objects on a map. They all consist in counting neighbors (up to or exactly at the chosen distance) around each reference point and transform their number into a meaningful statistic. There are basically three possible approaches: just count neighbors, count neighbors per surface area or calculate the proportion of neighbors of interest among all neighbors. They define the three families of functions: absolute (how many neighbors are there?), topographic (how many neighbors per unit of area?) and relative (what is the ratio of neighbors of interest?). The function values are not the main motivation. The purpose is rather to test the point pattern against the null hypothesis that it is a realization of a known point process which does not integrate a property of interest. The basic purpose of Ripley's $K$ is to test the observed point pattern against complete spatial randomness (CSR), i.e. a homogeneous Poisson process, to detect dependence between point locations (the null hypothesis supposes independent points) assuming homogeneity (i.e. the probability to find a point is the same everywhere). Ripley-like functions, available in the dbmss package, can be classified in three families:
Topographic measures such as $K$ take space as their reference. They have been widely used in ecology [@Fortin2005]. They have been built from the point process theory and have a strong mathematical background.
Relative measures such as $M$ [@Marcon2010] compare the structure of a point type to that of another type (they can be considered as cases and controls). They have been developed in economics, where comparing the distribution of a sector of activity to that of the whole economic activity is a classical approach [@Combes2008], but introduced only recently in ecology [@Marcon2012].
Absolute functions such as $K_d$ [@Duranton2005] have no reference at all but their value can be compared to the appropriate null hypothesis to test it.
Relative and absolute functions have been built from descriptive statistics of point patterns, not related to the underlying point processes, so they are seen as heuristic and ignored by the statistical literature [@Illian2008]. Topographic functions are implemented in the spatstat package [@Baddeley2005} but absolute and relative functions are missing. We fill this gap by proposing the dbmss package. It makes the computation of the whole set of distance-based methods simple for empirical researchers by introducing measures that are not available elsewhere and wrapping some topographic measures available in spatstat so that all can be used the same way.
Estimated values of the functions must be tested against a null hypothesis. The usual empirical way to characterize a spatial structure consists in computing the appropriate function and comparing it to the quantiles of a large number of simulations of the null hypothesis to reject [@Kenkel1988]. We propose extended possibilities to evaluate confidence envelopes, including global envelopes [@Duranton2005], a goodness-of-fit test [@Diggle1983] and an analytical test [@Lang2010; @Marcon2013].
Definitions of all functions and formulas for their estimation can be found in @Marcon2012c and are not repeated here, but they are summarized in the statistical background section. Their implementation is presented in the package content section.
We consider a map of points which often represents establishments in economic geography or trees in vegetation science. These points have two marks: a type (an industrial sector, a species...) and a weight (a number of employees, a basal area...). We want to apply to this point pattern a variety of exploratory statistics which are functions of distance between points, able to test the null hypothesis of independence between point locations. These functions are either topographic, absolute or relative. They can be interpreted as the ratio between the observed number of neighbors and the expected number of neighbors if points where located independently from each other. If reference and neighbor points are of the same type, the functions are univariate, to study concentration or dispersion. They are bivariate if the types differ, to address the colocation of types. We detail this approach below.
Topographic, homogeneous functions are Ripley's $K$ and its derivative $g$. Their null hypothesis is a Poisson homogeneous process: rejecting it means that the process underlying the observed pattern is either not homogeneous or not independent. These functions are applied when homogeneity is assumed so independence only is tested by comparing the observed values of the function to their confidence envelope under CSR. Bivariate functions are tested against the null hypothesis of random labeling (points locations are kept unchanged but marks are redistributed randomly) or population independence (the reference point type is kept unchanged, the neighbor point type is shifted) following @Goreaud2003. The random labeling hypothesis considers that points preexist and their marks are the result of a process to test (e.g. are dead trees independently distributed in a forest?). The population independence one considers that points belong to two different populations with their own spatial structure and wants to test whether they are independent from each other.
Edge effect correction is compulsory to compute topographic functions: points located close to boundaries have less neighbors because of the lack of knowledge outside the observation window. The spatstat package provides corrections following @Ripley1988, we use them.
$K_{inhom}$ [@Baddeley2000] is the generalization of $K$ to inhomogeneous processes: it tests independence of points assuming the intensity of the process is known. Empirically, it generally has to be estimated from the data after assumptions on the way to do it relying on the theoretical knowledge of the process. The null hypothesis (random position) is that the pattern comes from an inhomogeneous Poisson process of this intensity, which can be simulated. Applying $K_{inhom}$ to a single point type allows using the random location null hypothesis, following @Duranton2005: observed points (with their marks) are shuffled among observed locations to test for independence. Bivariate $K_{inhom}$ null hypotheses may be random labeling or population independence as defined by @Marcon2010: reference points are kept unchanged, other points are redistributed across observed locations.
$K_{mm}$ [@Penttinen2006; @Penttinen1992] generalizes $K$ to weighted points (weights are continuous marks of the points). Its null hypothesis in dbmss is random location. @Penttinen1992 inferred the point process from the point pattern, and used the inferred process to simulate the null hypothesis patterns. This requires advanced spatial statistics techniques and knowledge about the process that is generally not available. The random location hypothesis is a way to draw null patterns simply, but ignoring the stochasticity of the point process.
The $D$ [@Diggle1991] function compares the $K$ function of points of interest (cases) to that of other points (controls). Its null hypothesis is random labeling.
In their seminal paper, Duranton and Overman [@Duranton2005] study the distribution of industrial establishments in Great Britain. Every establishment, represented by a point, is characterized by its position (geographic coordinates), its sector of activity (point type) and its number of employees (point weight). The $K_d$ function [@Duranton2005] is the probability density to find a neighbor a given distance apart from a point of interest in a finite point process. The $K^{emp}$ function integrates the weights of points: it is the density probability to find an employee $r$ apart from an employee of interest.
$K_d$ and $K^{emp}$ are absolute measures since their value is not normalized by the measure of space or any other reference: for a binomial process, $K_d$ increases proportionally to $r$ if the window is large enough to ignore edge effects [the probability density is proportional to the perimeter of the circle of radius $r$, @Thomas2013], then edge effects make it decrease to 0 when $r$ becomes larger than the window's size: it is a bell-shaped curve. $K_d$ values are not interpreted but compared to the confidence envelope of the null hypothesis, which is random location. Bivariate functions null hypotheses are random labeling, following @Duranton2005, i.e. point types are redistributed across locations while weights are kept unchanged, or population independence (as for $K_{inhom}$). It is not corrected for edge effects. $K_d$ was designed to characterize the spatial structure of an economic sector, comparing it to the distribution of the whole activity. From this point of view, it has been considered as a relative function @Marcon2010. We prefer to be more accurate and distinguish it from strict relative functions which directly calculate a ratio or a difference between the number of points of the type of interest and the total number of points. What makes it relative is only its null hypothesis: changing it for random location (that of univariate $K_{inhom}$) would make univariate $K_d$ behave as a topographic function (testing independence of the distribution supposing its intensity is that of the whole activity).
$K_d$ is a leading tool in spatial economics. A great number of its applications can be found in the literature that confirms the recent interest for distance-based methods in spatial economics. A recent major study can be found in @Ellison2010.
The univariate and bivariate $M$ function [@Marcon2010] are the ratio of neighbors of interest up to distance $r$ normalized by its value over the whole domain. Their null hypotheses are the same as $K_d$'s. They do not suffer edge effects. @Marcon2010 show that the $M$ function respect most of the axioms generally accepted as the good properties to evaluate geographic concentration in spatial economics [@Combes2004; @Duranton2005].
$m$ [@Lang2014] is the density function measuring the same ratio as $M$, at distance $r$.
Empirically, all estimators can be seen as variations in a unique framework: neighbors of each reference point are counted, their number is averaged and divided by a reference measure. Last, this average local result is divided by its reference value, calculated over the whole point pattern instead of around each point.
Choosing reference and neighbor point types allows defining univariate or bivariate functions, counting neighbors up to or at a distance defines cumulative or density functions, taking an area or a number of points as the reference measure defines topographic or relative functions. These steps are detailed for two functions to clarify them: we focus on Ripley's $g$ and Marcon and Puech's $M$ bivariate function. See @Marcon2012c for a full review.
Reference points are denoted $x_i$, neighbor points are $x_j$. For density functions such as $g$, neighbors of $x_i$ are counted at a chosen distance $r$:
$$n\left(x_i,r\right){\rm =}\sum_{j{\rm ,\ }i\ne j}{k\left(\left\|x_i{\rm -}x_j\right\|,r\right)c\left(i,j\right)}$$
$k(\left\|x_i{\rm -}x_j\right\|,r)$ is a kernel estimator, necessary to evaluate the number of neighbors at distance $r$, and $c(i,j)$ is an edge-effect correction (points located close to boundaries have less neighbors because of the lack of knowledge outside the observation window).
To compute the bivariate $M$ function, reference points are of a particular type in a marked point pattern: $x_i \in \mathcal{R}$, where $\mathcal{R}$ is the set of points of the reference type. Neighbors of the chosen type are denoted $x_j \in \mathcal{N}$. In cumulative functions such as $M$, neighbors are counted up to $r$:
$$n\left(x_i,r\right){\rm =}\sum_{x_j \in \mathcal{N}, i \ne j}{{\mathbf 1}\left(\left\|x_i{\rm -}x_j\right\|\le r\right)w\left(x_j\right)}$$
Points can be weighted, $w(x_j)$ is the neighbor's weight.
The number of neighbors is averaged then. $n$ is the number of reference points:
$$\bar{n}\left(r\right)=\frac{1}{n}\sum_i{n\left(x_i,r\right)}$$
The average number of neighbors is compared to a reference measure. It may be a measure of space (the perimeter of the circle of radius $r$ for $g$), defining topographic functions:
$$z\left(r\right)=2 \pi r$$
It may also be the average number of neighbors of all types in a relative function such as $M$:
$$z\left(r\right)=\frac{1}{n}\sum_{j,i\ne j}{{\mathbf 1}\left(\left\|x_i{\rm -}x_j\right\|\le r\right)w\left(x_j\right)}$$
Finally, $\frac{\bar{n}(r)}{z(r)}$ is compared to the same ratio computed on the whole window. For $g$:
$$\frac{\bar{n}_0}{z_0}{\rm =}\frac{n-1}{A}$$
$A$ is the area of the window, $\bar{n}_0$ and $z_0$ are the limit values of $\bar{n}(r)$ and $z(r)$ when $r$ gets larger than the window's size. For $M$, it comes:
$$\frac{{\bar{n}}0}{z_0}=\sum_i{\frac{W\mathcal{N}}{W-w\left(x_i\right)}}$$
$W_\mathcal{N}$ is the total weight of neighbor points, $W$ that of all points. Finally, despite the functions are quite different (density vs. cumulative, topographic vs. relative, univariate vs. bivariate), both estimators can be written as ${\frac{{\bar{n}}}{z}}/{\frac{{\bar{n}}_0}{z_0}}$. Their value (except for absolute functions) can be interpreted as a location quotient: $g(r)=2$ or $M(r)=2$ means than twice more neighbors are observed at (or up to) distance $r$ than expected on average, i.e. ignoring the point locations in the window. The appropriate function will be chosen among the toolbox according to the question raised.
The dbmss package contains a full (within the limits of the literature reviewed in section 2) set of functions to characterize the spatial structure of a point pattern, including tools to compute the confidence interval of the counterfactual. It allows addressing big datasets thanks to C++ code used to calculate distances between pairs of points [using Rcpp and RcppParallel infrastructure: @Eddelbuettel2011; @Allaire2015]. Computational requirements actually are an issue starting from say 10,000 points [see @Ellison2010 for instance]. Memory requirement is $O(n)$, i.e. proportional to the number of points to store their location and type. We use loops to calculate distances and increment summary statistics rather than store a distance matrix which is $O(n^2)$, following @Scholl2013. Computation time is $O(n^2)$ because $n(n-1)/2$ pair distances must be calculated.
We consider planar points patterns (sets of points in a 2-dimensional space) with marks of a special kind: each point comes with a continuous mark (its weight) and a discrete one (its type).
We call this special type of point pattern weighted, marked, planar point patterns and define objects of class wmppp
, which derives from ppp
class as defined in spatstat.
Marks are a dataframe with two columns, PointWeight
containing the weights of points, and PointTypes
containing the types, as factors.
A wmppp
object can be created by the wmppp()
function which accepts a dataframe as argument, or converted from a ppp
object by as.wmppp()
.
Starting from a csv file containing point coordinates, their type and their weight in four columns, a wmppp
object can be created by just reading the file with read.csv()
and applying wmppp()
to the result.
Options are available to specify the observation window or guess it from the point coordinates and set default weights or types to points when they are not in the dataframe, see the package help for details.
The simplest code to create a wmppp
with 100 points is as follows.
It draws point coordinates between 0 and 1, and creates a wmppp
with a default window, all points are of the same type named All and their weight is 1.
Pattern <- wmppp(data.frame(X=runif(100), Y=runif(100))) summary(Pattern)
All functions are named Xhat
where X
is the name of the function:
Ripley's $g$ and $K$, and $K$'s normalization, Besag's $L$ [[email protected]];
Penttinen's $K_{mm}$ and $L_{mm}$;
Diggle and Chetwynd's $D$;
Baddeley et al.'s $K_{inhom}$ and its derivative $g_{inhom}$;
Marcon and Puech's $M$;
Duranton and Overman's $K_d$ (including its weighted version $K^{emp}$).
The suffix hat
has been used to avoid confusion with other functions in R: D
exists in the stats package for example.
Arguments are:
A weighted, marked planar point pattern (a wmppp
class object). The window can be a polygon or a binary image, as in spatstat.
A vector of distances.
Optionally a reference and a neighbor point type to calculate bivariate functions, or equivalently the types of cases and controls for the $D$ function.
Some optional arguments, specific to some functions.
Topographic functions require edge-effect corrections, provided by spatstat: the best
correction is systematically used.
Relative functions ignore the window.
Technical details are provided in help files.
These functions return an fv
object, as defined in spatstat, which can be plotted.
The classical confidence intervals, calculated by Monte-Carlo simulations [@Kenkel1988] are obtained by the XEnvelope
functions, where X
is the function's name.
Arguments are the number of simulations to run, the risk level, those of the function and the null hypothesis to simulate.
These functions return a dbmssEnvelope
object which can be plotted.
Null hypotheses have been discussed by @Goreaud2003 for topographic functions such as $K$ and by @Marcon2010 for relative functions. The null hypothesis for univariate functions is random position (points are drawn in a Poisson process for topographic functions) or random location (points are redistributed across actual locations for relative functions). Bivariate functions support random labeling and population independence null hypotheses. The possible values of arguments are detailed in the help file of each function.
Building a confidence envelope this way is problematic because the test is repeated at each distance.
The underestimation of the risk has been discussed by @Loosmore2006.
@Duranton2005 proposed a global envelope computed by the repeated elimination of simulations reaching an extreme value at any distance until the desired level is reached.
The argument Global = TRUE
is used to obtain it instead of the local one.
We illustrate the main features of the package by two examples. The first one comes from the economic literature [@Bonneu2007] [^1]
[^1]: The dataset can be downloaded from http://publications-sfds.fr/index.php/csbigs/article/downloadSuppFile/376/69
load("CSBIGS.Rdata") Category <- cut(Emergencies$M, quantile(Emergencies$M, c(0, 0.9, 1)), labels = c("Other", "Biggest"), include.lowest = TRUE) X <- wmppp(data.frame(X=Emergencies$X, Y=Emergencies$Y, PointType=Category), win=Region) X$window$units <- c("meter","meters") X2 <- split(X) marks(X2$Other) <- rep(1, X2$Other$n) marks(X2$Biggest) <- rep(1, X2$Biggest$n) par(mfrow=c(1,2), mar=c(0,0,0,0)) plot(X2$Other, main="", maxsize=1, legend=FALSE) text(514300, 1826800, "a") plot(X2$Biggest, main="", maxsize=1, legend=FALSE) text(514300, 1826800, "b") par(mfrow=c(1,1))
The map of emergencies in the urban area of Toulouse, France, during year 2004 (about 33 km from south to north) shows (a) 20,820 emergencies recorded (many points are confused at the figure scale) and (b) the locations of the 10 percent most serious ones. The workload associated to each emergency (the number of men $\times$ hours it required) is known. The original study tested the dependence between workload and location of emergencies: it did not exclude the null hypothesis of random labeling. We have a complementary approach here: we consider the 10 percent more serious emergencies, i.e. those which caused the highest workload. $K_d$ may detect concentration (or dispersion) if, at a distance $r$ from a serious emergency, the probability to find another serious emergency is greater (or lower) than that of finding an emergency whatever its workload:
The Emergencies
data frame contains point coordinates (in meters) in columns X
and Y
and workload in column M
.
The second line of the code creates a vector containing a factor describing the workload to separate its 10% higher values.
A wmppp
object is created then, containing the points and their mark.
The KdEnvelope
function is run from 0 to 10km by steps of 100m for the most serious emergencies.
The figure below shows that the 10% most serious emergencies are more dispersed than the distribution of all emergencies at all distances up to 10km.
KdE <- KdEnvelope(X, r=seq(0, 10000, 100), NumberOfSimulations=1000, ReferenceType="Biggest", Global=TRUE) plot(KdE, main="")
The solid, black curve is $K_d$. The dotted red curve is the average simulated value and the shaded area is the confidence envelope under the null hypothesis of random location. The risk level is 5%, 1000 simulations have been run. distances are in meters.
This opens the way to discuss on the optimal location of fire stations.
The second example uses the paracou16
point pattern provided in the package.
It represents the distribution of trees in a 4.1-ha tropical forest plot in the Paracou field station in French Guiana [@Gourlet2004].
It contains 2426 trees, whose species is either Qualea rosea, Vouacapoua americana or Other (one of more than 300 species).
Weights are basal areas (the area of the stems virtually cut 1.3 meter above ground), measured in square centimeters.
data("paracou16") plot(paracou16, which.marks="PointWeight", main="", legend=FALSE)
On the map, circles are centered on trees in the forest plot (the containing rectangle is 200m wide by 250m long). Circle sizes are proportional to the basal areas of trees.
The question to test is dependence between the distributions of the two species of interest. Bivariate $M(r)$ is calculated for $r$ between 0 and 30 meters. 1000 simulations are run to build the global confidence envelope.
Envelope <- MEnvelope(paracou16, r = seq(0, 30, 2), NumberOfSimulations = 1000, Alpha = 0.05, ReferenceType = "V. Americana", NeighborType = "Q. Rosea", SimulationType = "RandomLabeling", Global = TRUE) plot(Envelope, main="", ylim=c(0, 20))
$M(r)$ values of Qualea rosea around Vouacapoua Americana are plotted. The solid, black curve is $M$. The dotted red curve is the average simulated value. The shaded area is the confidence envelope. $M=1$ is expected if points are independently distributed. The risk level is 5%, 1000 simulations have been run. Distances are in meters.
The calculated function is $M$, showing the repulsion between V. Americana and Q. rosea up to 30 m. Significance is unclear, since the observed values of the function are very close to the lower bound of the envelope. The complete study, with a larger dataset allowing significant results, can be found in @Marcon2012.
A Goodness-of-fit test for $K$ has been proposed by @Diggle1983, applied to $K$ by @Loosmore2006 and to $M$ by @Marcon2012.
It calculates the distance between the actual values of the function and its average value obtained in simulations of the null hypothesis.
The same distance is calculated for each simulated point pattern, and the returned $p$-value of the test if the ratio of simulations whose distance is larger than that of the real point pattern.
The test is performed by the GoFtest
function whose argument is the envelope previously calculated (actually, the function uses the simulation values).
Applied to the example of Paracou trees, the $p$-value is:
GoFtest(Envelope)
The Ktest
has been developed by Lang and Marcon [@Lang2010; @Marcon2013].
It does not rely on simulations and returns the $p$-value to erroneously reject complete spatial randomness (CSR) given the values of $K$.
It relies on the exact variance of $K$ calculated with edge-effect corrections.
It only works in a rectangular window.
The following example tests a 1.5-ha subset of paracou16
(100m by 150m, origin at the southwestern corner).
data("paracou16") RectWindow <- owin(c(300, 400), c(0, 150)) X <- paracou16[RectWindow] (pKtest <- Ktest(X, seq(5, 50, 5)))
It rejects CSR (p=r round(pKtest, 4)
).
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.