Spatial data and topology"
In anglr: Mesh Topology and Visualization for Spatial Data

knitr::opts_chunk$set(echo = TRUE)

library(spbabel)
library(rgdal)
library(dplyr)

Terminology note: an early version of this document used the term branch to refer to what is now called a path, where a path is a linked sequence of coordinates. Technically a path can be composed of a single vertex, which is partly why a different term was used originally. There are some abbreviations "b" and "bXv" below that still refer to this old usage, they stand for "path" and "paths-link-vertex" respectively.

Spatial normal forms

In this document I describe a "normal-form" that provides a very general way of extending the traditional GIS forms, and is a bridge between vector and raster rather than being a different form altogether. The purpose of this document is to advocate for this general form of data organization that can be used for new extended uses of GIS. I'm not arguing that this be used in place of other optimized forms, although it can be: I am interested in operations that simply cannot be performed already.

When we talk about vector data in geo-spatial, we have at least three levels of hierarchy to consider in the data structures.

objects (or features) - these are the things we primarily interact with and store data against
coordinates - these are the specific vertices, the dimensionless points that place our data in a geometry
paths - these are the parts that different objects are made of, they describe the way the coordinates are linked together in sequence

GIS tools typically only provides direct access to the objects, though the relations between paths and coordinates can sometimes be seen.

We generally cannot store information against the paths or the coordinates, beyond what they inherently are defined by. For coordinates this is the X and Y values, and/or the longitude and latitudes, but simple features does provide the ability to store a "third" coordinate "Z" and or a measure coordinate "M". M is typically used for "linear referencing", and not a more general multidimensional geometry (like time).

I'll use the countries example from a GeoPackage file provided here. I use R because I can tell the complete story with it, in a concrete and reproducible way.

Read in a polygon vector layer in traditional GIS form, and plot it.

#library(rworldmap)
#data(countriesLow)
#p <- countriesLow
library(rgdal)
p <- readOGR(system.file("extdata", "small_world.gpkg", package = "anglr"), "ne_110m_admin_0_countries")
plot(p, col = viridis::viridis(nrow(p)))

This object p presents a "data frame" (i.e. a table) front-end that we can query and use at the objects level, much like in some GIS software we can easily select a single object or row in a table.

library(spbabel)
pnganz <- subset(p, name %in% c("Australia", "Indonesia", "New Zealand", "Papua 
New Guinea"))
pnganz

plot(pnganz, col = viridis::viridis(nrow(pnganz)))

Looking at the object's underlying geometric structure shows nested lists of matrixes of x,y coordinates. There is one matrix per branch, analogous to the way that feature parts are nested in standard Geom binary forms like WKB. Each component path stores extra information about whether it is a hole, the ring direction, a coordinate for label plotting and so on. We otherwise cannot store any more information on the component parts though.

NOTE: the Spatial classes here are pre-simple features, so they are more analogous to the structures in a shapefile in that a polygon hole's "island parent"" may be ambiguous, but this is not so important to this story. R now has simple features in the sfr project here, which adds Z, M and the possibility of some of the exotic types as well. https://github.com/edzer/sfr

These hierarchical structures can be serialized and stored in different ways, typically they are stored as binary geoms and stored directly in a table.

An interesting aspect here is that these structures don't describe the topology of the objects in any special way, these are just paths of coordinates, and when they are plotted or used in analysis there's a rule about how space is enclosed by a closed path. If we treat them as lines (as SpatialLinesDataFrame), the only difference is to not treat them as enclosed paths. Literally the only difference in the structure of this object from the polygons version is the name of the class, and the behaviour that methods invoked for this object will provide.

plot(as(pnganz, "SpatialLinesDataFrame"))
plot(as(pnganz, "SpatialLinesDataFrame"), col = viridis::viridis(nrow(pnganz), alpha = 0.7), lwd = c(2, 4), add = TRUE)

str(as(geometry(pnganz[1,]), "SpatialLines"))

If we convert these data to "normal form", we actually need at least three tables, one each for the objects, the paths, and the coordinates (vertices). The map_table function in the spbabel package creates these but also adds another link table between paths and vertices to enable de-duplication of shared vertices. The de-duplication is required for triangulating the polygons, and other topological operations.

## please note, map_table was a very early silicate::PATH
ptabs <- spbabel::map_table(pnganz)
print(names(ptabs))
print(sapply(ptabs, nrow))

Now it's a bit clearer how the underlying entities are put together. Each table here has a unique relational id, this allows us to subset and recombine these tables without having to convert indexes each time.

The objects.

ptabs$o

The paths record which object they belong to.

ptabs$b

The paths-link-vertex table records the relationship between vertices and paths (by default the de-duplication is done in X-Y but it could be done in other geometric spaces, e.g. in 1D time or 3D X-Y-Z or X-Y-Time).

This is the instances of vertices as opposed to the unique paired values of coordinates themselves.

ptabs$bXv

And finally the vertices. In this example there are fewer unique x-y vertex than there are instance of the vertices, not a one-to-one match. This discrepancy obviously increases greatly for layers with shared boundaries, though in this example it is mostly due to the final closing coordinate on each polygon path - it's a repeated instance, but not a repeated vertex value. There is at least one shared edge in this layer, clearly the one between Indonesia and Papua New Guinea.

ptabs$v

Polygons are just lines

From this form we can see clearly that polygons and lines in GIS are really the same thing, we have paths of coordinates and then rules about how they are used.

If we compare each entity table side by side it's clear the only difference is whether a path is badged as an island vs. a hole.

For points we don't need the paths or the order data, though for multipoints we do need branch.

ltabs <- spbabel::map_table(as(pnganz, "SpatialLinesDataFrame"))

for (i in seq_along(ltabs)) {
  writeLines("------------------------------")
  print(ptabs[i])
 writeLines("")
}

What makes polygons different to lines?

The coordinate-path structures used above for polygons and lines are very explicit, and in traditional form cannot be used in a more abstract way. By collecting the attributes of the entities in use into their own tables we start to build this abstraction. The paths are represented as a sequence of identifiers, rather than the actual coordinate values themselves. Why do this? We can abstract the choice of what do with those coordinate away from their storage. We also get a limited form of topology, in that a change made to one vertex coordinate attribute is reflected in all of the paths that use that vertex, analogous the Shared Edit mode in Manifold 8.0.

The next step in topological relationships is to represent each segment of a line rather than the entire path. To do this we need a table of segments, and a link table to store the identity of the two vertices used by those segments.

This has been implemented in the package anglr, but (more recently) replaced by models in silicate.

lsegment <- silicate::SC(as(pnganz, "SpatialLinesDataFrame"))
as.data.frame(lapply(lsegment, nrow))

rgl::rgl.clear()
library(geosphere)
library(anglr)
xyz <- proj4::ptransform(randomCoordinates(5e4) * pi/180, 
                   "+init=epsg:4326", 
           "+proj=geocent +a=1")


tri <- geometry::convhulln(xyz)
rgl::triangles3d(xyz[t(tri), ], 
                 specular = "black", 
                 color = "skyblue", alpha = 0.4)

plot3d(globe(lsegment, "+proj=geocent +a=1.05"), add = TRUE)
rgl::rglwidget()

This is no different for polygons when we store them as polygon paths, so then why is the segment/edge model useful? It provides a table to store metrics such as the length of the segment, its duration in time, and other information. The segment/edge model is also a required precursor for building a triangulated mesh. This brings us to an important stage of the story.

Polygons are not composed of primitives

WIP everything from here needs updating for the new silicate regime. MDS 2018-03-06.

Lines and polygons are stored as paths of coordinates, but lines can be decomposed to a more abstract form. Once in this form we can (in R) plot the lines much more quickly as segments, each with their own individual properties.

par(mar = rep(0, 4))
plot(lsegment$v$x_, lsegment$v$y_, asp = 1, pch = ".", axes = FALSE)
lines(lsegment$v$x_, lsegment$v$y_, col = viridis::viridis(4))

Not surprisingly, our connected line doesn't make much sense, but worse our attempts at applying multiple colours was completely unsuccessful. Segments to the rescue.

par(mar = rep(0, 4))
plot(lsegment$vertex$x_, lsegment$vertex$y_, asp = 1, pch = ".", axes = FALSE)

lsegment$object$color <- viridis::viridis(nrow(lsegment$object))
plot(lsegment)
## we used to use this
# segs <- 
#     lsegment$ %>%
#     inner_join(lsegment$l) %>% 
#     inner_join(lsegment$o) %>% 
#     inner_join(lsegment$v) %>%
#     dplyr::select(color, vertex_, segment_, x_, y_) %>%
#     group_by(segment_) %>%
#     mutate(x__ = lead(x_), y__ = lead(y_)) %>%
#     filter(row_number() == 1)
#     
# segments(segs$x_, segs$y_, segs$x__, segs$y__, col = segs$color, lwd = 4)

This is not lovely code, though it is straight forward and systematic. Treated as segments we automatically get the right "topology" of our lines, we joined the object attribute down to the actual pairs of coordinates and plotted all the segments individually. We managed to keep our object-part-coordinate hierarchy, though we've chosen primitives belonging to objects rather than paths as the model. This is also convenient for the next step because line segments are what we need for generating primitives to represent the polygons as surfaces.

Constrained polygon triangulation starts with line primitives

Treat the polygon as segments build a triangulation, a surface of 2D triangle primitives.

prim2D <- DEL(pnganz)
plot3d(pnganz, border = "black", col = "transparent", lwd = 4)
prim2D$object$color_ <- viridis::viridis(nrow(prim2D$object))

library(purrr)
library(tidyr)
tri <-
  prim2D$t %>%
  inner_join(prim2D$tXv) %>%
  inner_join(prim2D$v) %>%
  inner_join(prim2D$o) %>%
  select(-object_, -name, -vertex_) %>%
  nest(x_, y_, color,.key = "triangle")

walk(tri$triangle, ~polygon(x = .$x_, 
                            y = .$y_,
                            border = .$color)
                    )

The plot walk above may be inefficient, but it's purely to illustrate that we have the shapes in the right form. This is used in anglr to plot the shapes in 3D, either in native planar form or as a surface of a globe.

library(rgl)
rgl.clear()
plot(prim2D, specular = "black")
rglwidget()

rgl.clear()
plot(anglr::globe(prim2D), specular = "black")
rglwidget()

Why do this? It's not just to plot a globe, but to see why it's helpful to see what the function globe() does.

Run the layer through globe() and print out the vertices table.

prim2D$v
anglr::globe(prim2D)$v

The only thing that happened was that the input x_ and y_ were converted to geocentric "x, y, z" coordinates. Under the hood this is done by driving the transformation with PROJ.4 (via the R package proj4). The PROJ.4 family in use is "geocent", i.e. here the meta table simply records the history of transformations.

anglr::globe(prim2D)$meta[, c("proj", "ctime")]

We can otherwise do anything we like with the vertices, including reprojecting them and copying on other attributes.

A relief map of North Carolina counties.

f <- system.file("extdata/gebco1.tif", package = "anglr")
## ad hoc scaling as x,y and  z are different units
r <- raster::raster(f)/1000

library(sf)
nc <- read_sf(system.file("shape/nc.shp", package="sf"))
library(raster)
library(anglr)
## objects
## a relief map, triangles grouped by polygon with interpolated raster elevation 
p <- anglr(nc, max_area = 0.008) ## make small triangles (0.2 sq lon-lat degree)
g <- anglr(graticule::graticule(-85:-74, 32:37))
p$v$z_ <- raster::extract(r, cbind(p$v$x_, p$v$y_), method = "bilinear")

## plot the scene
library(rgl)
rgl.clear(); 
plot(p); plot(g, color = "white"); 
bg3d("black"); material3d(specular = "black")
rglwidget()

What if we didn't set the max_area and only got the triangulation needed to stay within the polygons?

p <- anglr(nc, max_area = 0.008) ## make small triangles (0.2 sq lon-lat degree)
g <- anglr(graticule::graticule(-85:-74, 32:37))
p$v$z_ <- raster::extract(r, cbind(p$v$x_, p$v$y_), method = "bilinear")

## plot the scene
library(rgl)
rgl.clear(); 
plot(p); plot(g, color = "white"); 
bg3d("black"); material3d(specular = "black")
rglwidget()