whitebox Demo

In whitebox: 'WhiteboxTools' R Frontend

knitr::opts_chunk$set(
  eval = whitebox::check_whitebox_binary(),
  echo = TRUE,
  collapse = TRUE,
  comment = "#>",
  fig.width = 6,
  fig.height = 6 
)

# setup so inline stats on version/tools show up
library(whitebox)
data("wbttoolparameters", package="whitebox")

# # sample code to check and install whitebox to a custom path
# if (!whitebox::check_whitebox_binary()) {
#   wd <- tempdir()
#   whitebox::install_whitebox(wd)
#   whitebox::wbt_init(wd = file.path(wd, "WBT", basename(whitebox::wbt_default_path())))
# }

# system and package dependencies must be met to build the vignette
stopifnot(requireNamespace("terra"))

Introduction

whitebox is an R frontend for the 'WhiteboxTools' library, which is an advanced geospatial data analysis platform developed by Prof. John Lindsay at the University of Guelph's Geomorphometry and Hydrogeomatics Research Group.

'WhiteboxTools' can be used to perform common geographical information systems (GIS) analysis operations, such as cost-distance analysis, distance buffering, and raster reclassification. Remote sensing and image processing tasks include image enhancement (e.g. panchromatic sharpening, contrast adjustments), image mosaicing, numerous filtering operations, simple classification (k-means), and common image transformations. 'WhiteboxTools' also contains advanced tooling for spatial hydrological analysis (e.g. flow-accumulation, watershed delineation, stream network analysis, sink removal), terrain analysis (e.g. common terrain indices such as slope, curvatures, wetness index, hillshading; hypsometric analysis; multi-scale topographic position analysis), and LiDAR data processing.

WhiteboxTools is not a cartographic or spatial data visualization package; instead it is meant to serve as an analytical backend for other data visualization software, mainly GIS.

This vignette shows how to use the whitebox R package to run WhiteboxTools.

Suggested citation: Lindsay, J. B. (2016). Whitebox GAT: A case study in geomorphometric analysis. Computers & Geosciences, 95, 75-84. doi: http://dx.doi.org/10.1016/j.cageo.2016.07.003

Setup

Load the whitebox library.

library(whitebox)

How whitebox works

whitebox generates system() calls to a local WhiteboxTools binary: whitebox_tools or whitebox_tools.exe

You can find the binary path that the package is going to use with wbt_exe_path()

wbt_exe_path(shell_quote = FALSE)

This command always returns a "default" path, whether or not you have WhiteboxTools installed.

Interfacing with R spatial packages

WhiteboxTools input and output are specified as file paths to rasters in GeoTIFF format, shapefiles, HTML output, LiDAR-related files, and more.

In this vignette we will use the terra package for visualization. Just as easily we could have used raster, stars or other options available in the R ecosystem for handling the GeoTIFF output.

The main way to view your output is to save "output" file paths as a variable so that you can use them after processing to load the result into an R spatial object.

Working with Raster Data

A demonstration employing the {terra} package follows:

library(terra)
library(whitebox)

# DEMO: calculate slope with WhiteboxTools and raster

# Typically the input/output paths are stored as variables

# sample DEM input GeoTIFF
input <- sample_dem_data()

# output file (to be created)
output <- file.path(tempdir(), "slope.tif")

Run a tool such as wbt_slope() or "Slope".

WhiteboxTools reads from input and writes to output.

wbt_slope(input, output, units = 'radians')

if (file.exists(output)) {
  # create a SpatRaster from file output path
  outputras <- terra::rast(output)
}

In this case, we can achieve a similar slope map result using terra::terrain(), so we will create and plot a SpatRaster from output and compare the two.

if (file.exists(input) && file.exists(output) && !is.null(outputras)) {
  # par(mfrow = c(2, 1), mar = c(1, 1, 1, 1))

  # inspect the output graphically
  plot(
    outputras,
    main = "whitebox::wbt_slope() [radians]",
    axes = FALSE
  )

  # calculate equivalent using raster::terrain() on input
  plot(
    terra::terrain(terra::rast(input)),
    main = "terra::terrain() [radians]",
    axes = FALSE
  )
}

The SpatRaster, RasterLayer and related classes in the terra and raster packages are perfect for maintaining the linkage between file output and an R object with the data in or out of memory.

Use terra::sources() to get the "source" file name(s). If you are using a raster RasterLayer objects the equivalent method is raster::filename().

# the SpatRaster retains a reference to the input file name
terra::sources(outputras)

WhiteboxTools R setup

Installing WhiteboxTools

In whitebox wbt_init() is the standard way to set the "exe_path" for a session.

If you do not have WhiteboxTools installed in the default location, do not have whitebox_tools on your PATH, and have not set up your package options, the package will not be able to find your WhiteboxTools installation on load.

Usually you can use whitebox::install_whitebox() to download the latest binaries that correspond to the available version of the R package. However, this is not required. You may download/compile WhiteboxTools yourself and install anywhere for use with the whitebox R package.

For general information consult the WhiteboxTools User Manual: https://www.whiteboxgeo.com/manual/wbt_book/install.html

For more details on building from source see: https://github.com/jblindsay/whitebox-tools

Package Settings with wbt_init()

wbt_init() is used to set and check the path of the binary executable that commands are passed to.

The executable path and other options are stored as package options, and can be overridden by system environment variables. A default value wbt_exe_path(shell_quote = FALSE) is passed when the exe_path argument is unspecified.

# inspect where wbt_init() will be checking
wbt_exe_path(shell_quote = FALSE)

# TRUE when file is found at one of the user specified paths or package default
# FALSE when whitebox_tools does not exist at path
wbt_init()

This section will cover optional arguments to wbt_init() (exe_path, wd and verbose) and their corresponding options and helper functions.

exe_path argument

The exe_path argument to wbt_init() sets the whitebox.exe_path package option. exe_path is the path to a WhiteboxTools executable file. The default value is the package installation directory, subdirectory "WBT", followed by whitebox_tools.exe or whitebox_tools depending on your operating system.

# set path manually to whitebox_tools executable, for instance:
wbt_init(exe_path = '/home/andrew/workspace/whitebox-tools/target/release/whitebox_tools')

The package will automatically find an existing installation when whitebox_tools is in a directory on your PATH.

Package options other than exe_path (as detailed in ?whitebox::whitebox and ?wbt_init) can be set with wbt_init(exe_path, ...), where ... is additional named arguments corresponding to the * suffix in whitebox.* package options names. Use wbt_options() or specific methods like wbt_verbose(), wbt_wd() to get all values or set specific values.

wd argument

The wd argument can be used to set the WhiteboxTools working directory.

A working directory specifies a base folder path where WhiteboxTools can find inputs and create outputs. Setting the whitebox.wd package option (via the wd argument to wbt_init() or wbt_wd()) aids the process of setting file paths. If a value is set for the option the --wd directory flag is added for tools that support it.

Before you set the working directory in a session the default output will be in your current R working directory unless directory is specified in your input/output arguments. You can change working directory at any time by setting the wd argument to wbt_wd() and running a tool.

NOTE: once you have set a working directory in a session, the directory needs to be set somewhere new to "replace" the old value; just dropping the flag will not automatically change the working directory back to your R working directory* and your output will show up in whatever folder you set initially.

A helper method for setting the whitebox.wd option is wbt_wd().

To "unset" the option in the R package you can use wbt_wd("") which is equivalent to wbt_wd(getwd()). The next tool call will change the WhiteboxTools working directory setting to the new path. After this point the flag need not be specified [until you wish to change again].

wbt_wd("") # "" equivalent to getwd()

verbose argument

The verbose argument is used to set the package option related to tool "verbosity": whitebox.verbose. When whitebox.verbose is FALSE no output will be cat() to the console by running tools.

A helper method for getting and setting the whitebox.verbose option is wbt_verbose(). wbt_verbose() is used throughout the package to check what level of verbosity should be used.

By default, the result of wbt_verbose() is the result of interactive() so tools will print extra console output when you are there to see it. This is used in a variety of wbt_* methods to allow the package option to control output for many functions in a consistent manner, hide output in your automated tests, markdown documents, vignettes etc.

In this vignette we use wbt_verbose(TRUE) so the package option whitebox.verbose is set to TRUE

# force output when run non-interactively (knitr)
wbt_verbose(TRUE)

This is mainly to print out the tool name and elapsed time whenever we run a tool:

#> wbt_breach_depressions - Elapsed Time (excluding I/O): 0.12s

This package-level verbose option can also control the verbose_mode values passed to wbt_* tool functions. Turning on "full" output requires a third option to be set for this argument: "all". Use wbt_verbose("all"). wbt_verbose() will still return TRUE when the whitebox.verbose option is "all".

Long-term Package Option Settings

The package will detect when you have added the WhiteboxTools directory to your PATH environment variable. For long-term package option settings you can put whitebox_tools on $PATH or set R_WHITEBOX_EXE_PATH in your user .Rprofile or .Renviron.

On Windows you can add the path to whitebox_tools.exe as a new entry R_WHITEBOX_EXE_PATH in User or System Environment variable.

On Linux/Mac you can set R_WHITEBOX_EXE_PATH directly with export R_WHITEBOX_EXE_PATH="/path/to/whitebox_tools".

• Note that if you set your $PATH in you regular shell profile (.profile/.bashrc/.zshrc) then it will not be sourced by RStudio for your R session by default.

This requires that you have the Unix tool which or one of its analogues.

You can also set R_WHITEBOX_EXE_PATH manually in R:

Sys.setenv(R_WHITEBOX_EXE_PATH = Sys.which("whitebox_tools"))

• Replace the Sys.which() call with a custom path string as needed such as "C:/path/to/whitebox_tools.exe".

All of the other package options can similarly be set as environment variables. They are prefixed with R_WHITEBOX_*. See ?whitebox for details.

Running tools

Specify input and output paths, and any other options, as specified in package reference:

• https://whiteboxR.gishub.org/reference/index.html

For instance, "BreachDepressions" is used below to process a Digital Elevation Model (DEM) before we identify flow pathways. This tool uses uses Lindsay's (2016) algorithm, which is preferred over depression filling in most cases.

# sample DEM file path in package extdata folder
input <- sample_dem_data()

# output file name
output <- file.path(tempdir(), "output.tif")

# run breach_depressions tool
wbt_breach_depressions(dem = input, output = output)

For more info see: ?wbt_breach_depressions

These wbt_*_tool_name_*() functions are wrappers around the wbt_run_tool() function that does the system() call given a function-specific argument string.

# sample DEM file path in package extdata folder
input <- sample_dem_data()

# output file name
output <- file.path(tempdir(), "output.tif")

# run breach_depressions tool
wbt_run_tool(tool_name = "BreachDepressions", args = paste0("--dem=", input, " --output=", output))

The above method of creating wbt_breach_depressions(dem = ..., output = ...) to handle wbt_run_tool("BreachDepressions", args = ...) makes it easy to generate static methods that have parity with the latest WhiteboxTools interface.

Example: Compare input v.s. output with terra

We use the {terra} package to read the GeoTIFF file output by WhiteboxTools.

Setup

library(terra)

# sample DEM file path in package extdata folder
input <- sample_dem_data()

# output file name
output <- file.path(tempdir(), "output.tif")

Run wbt_breach_depressions() (BreachDepressions tool)

# run breach_depressions tool
wbt_breach_depressions(dem = input, output = output)

Visualize results with terra

# create raster object from input file
input <- rast(input)

if (file.exists(output)) {
  # create raster object from output file
  output <- rast(output)

  # par(mar = c(2, 1, 2, 1))
  # inspect input v.s. output
  plot(input, axes = FALSE, main = "DEM")
  plot(output, axes = FALSE, main = "DEM (Breached Depressions)")

  # inspect numeric difference (output - input) 
  plot(output - input, axes = FALSE,  main = "Difference ([Breached Depressions] - [DEM])")
}

Example: Identifying Tributaries

Here we will take our processing of DEMs a bit further by performing several WhiteboxTools operations in sequence.

We are interested in identifying and ranking tributaries of watercourses (streams and rivers).

An R package that makes use of the whitebox R package is hydroweight. Here is a brief example on the beginning of the hydroweight README showing how the breached DEM above can be used in a spatial analysis of stream networks.

Setup

library(whitebox)
library(terra)

## Sample DEM from whitebox package
toy_file <- sample_dem_data()
toy_dem <- rast(x = toy_file)

## Generate wd as a temporary directory. 
## Replace with your own path, or "." for current directory
wd <- tempdir()

## Write toy_dem to working directory
writeRaster(
  x = toy_dem, filename = file.path(wd, "toy_dem.tif"),
  overwrite = TRUE
)

wbt_breach_depressions() -- Breach DEM Depressions

First we pre-process by breaching depressions in the DEM

## Breach depressions to ensure continuous flow
wbt_breach_depressions(
  dem = file.path(wd, "toy_dem.tif"),
  output = file.path(wd, "toy_dem_breached.tif")
)

wbt_d8_pointer() -- Calculate Flow Direction

Then we generate the direction of flow on the DEM surface using the "D8" flow pointer method

## Generate d8 flow pointer (note: other flow directions are available)
wbt_d8_pointer(
  dem = file.path(wd, "toy_dem_breached.tif"),
  output = file.path(wd, "toy_dem_breached_d8.tif")
)

wbt_d8_flow_accumulation() -- Flow Accumulation

Once we calculate the direction of flow by some method, we calculate cumulative flow

For example with wbt_d8_flow_accumulation():

## Generate d8 flow accumulation in units of cells (note: other flow directions are available)
wbt_d8_flow_accumulation(
  input = file.path(wd, "toy_dem_breached.tif"),
  output = file.path(wd, "toy_dem_breached_accum.tif"),
  out_type = "cells"
)

Additional Flow Direction and Accumulation Tools

In addition to D8 flow pointers (flow direction), there are several other options for both direction and accumulation such as FD8, D-infinity, and D-infinity.

• Keyword "Pointer" tools: "D8Pointer", "DInfPointer", "FD8Pointer", "Rho8Pointer"

• Keyword "FlowAccumulation" tools: "D8FlowAccumulation", "DInfFlowAccumulation", "FD8FlowAccumulation", "Rho8FlowAccumulation", "MDInfFlowAccumulation"

Search for more tools involving "flow pointer" by key word: wbt_list_tools(keyword = "flow pointer")

wbt_list_tools(keyword = "flow pointer")

This is just an example of the wealth of tool options made available by the WhiteboxTools platform.

wbt_extract_streams() -- Extract Stream Network

With our flow accumulation raster in hand, we can extract a stream network with wbt_extract_streams() based on a threshold (e.g. 100) of accumulated flow. This threshold value you choose will depend on analysis goals, the choice of flow accumulation algorithm used, local topography, as well as resolution and extent of DEM.

## Generate streams with a stream initiation threshold of 100 cells
wbt_extract_streams(
  flow_accum = file.path(wd, "toy_dem_breached_accum.tif"),
  output = file.path(wd, "toy_dem_streams.tif"),
  threshold = 100
)

wbt_tributary_identifier() -- Identify Tributaries

Next, let's identify tributaries. This function wbt_tributary_identifier() is a little more complicated because it takes takes two inputs:

• Our raster D8 pointer file.

• And our raster streams file.

wbt_tributary_identifier(
  d8_pntr = file.path(wd, "toy_dem_breached_d8.tif"),
  streams = file.path(wd, "toy_dem_streams.tif"),
  output = file.path(wd, "toy_dem_tributaries.tif")
)

Compare results

Finally, we compare results of wbt_extract_streams() with wbt_tributary_identifier()

if (file.exists(file.path(wd, "toy_dem_streams.tif"))) {
  par(mfrow = c(2, 1), mar = c(3, 1, 2, 1))

  plot(
    rast(file.path(wd, "toy_dem_streams.tif")),
    main = "Streams",
    col = "black"
  )

  plot(
    rast(file.path(wd, "toy_dem_tributaries.tif")),
    main = "TributaryIdentifier"
  )
}

Appendix: wbt_* utility functions

These methods provide access to WhiteboxTools executable parameters and metadata.

wbt_help()

wbt_help() prints the WhiteboxTools help: a listing of available commands for executable

wbt_help()

wbt_license()

wbt_license() prints the WhiteboxTools license

wbt_license()

wbt_version()

Prints the WhiteboxTools version

wbt_version()

wbt_list_tools()

Use wbt_list_tools() to list all available tools in WhiteboxTools. In version r attr(wbttoolparameters, 'version') there are over r (length(whitebox::wbt_list_tools()) - 1) tools! See all the available toolboxes and extensions.

wbt_list_tools()

The full list can be an overwhelming amount of output, so you pass the keywords argument to search and filter.

For example we list tools with keyword 'flowaccumulation' in tool name or description.

wbt_list_tools(keywords = "flowaccumulation")

wbt_tool_help()

Once we find a tool that we are interested in using, we can investigate what sort of parameters it takes. The R methods generally take the same named parameters.

R functions have the naming scheme wbt_tool_name where _ is used for spaces, whereas the tools themselves have no spaces.

wbt_tool_help("tributaryidentifier") shows the command line help for a tool by name.

wbt_tool_help("tributaryidentifier")

?wbt_tributary_identifier shows the corresponding R function help, which is derived from the command line help page and other metadata.

wbt_toolbox()

Another way that tools are organized in WhiteboxTools is by "toolbox."

wbt_toolbox() prints the toolbox for a specific tool (or all tools if none specified)

wbt_toolbox(tool_name = "aspect")

Print the full list by not specifying tool_name

wbt_toolbox()

wbt_tool_parameters()

wbt_tool_parameters() retrieves the tool parameter descriptions for a specific tool as JSON formatted string.

wbt_tool_parameters("slope")

wbt_view_code()

WhiteboxTools is written in Rust and is open source. You can view the source code for a specific tool on the source code repository.

wbt_view_code("breach_depressions")

Use the argument viewer=TRUE to use browseURL() to open a browser window to the corresponding GitHub page.

# cleanup temp files
wd <- tempdir()
unlink(file.path(wd, "slope.tif"))
unlink(file.path(wd, "output.tif"))
unlink(list.files(wd, "^toy_dem.*tif*$", full.names = TRUE))

whitebox documentation built on Nov. 19, 2023, 1:07 a.m.