In NOAA-OWP/hydrofabric: Easily Install and Load 'hydrofabric' tools

library(dplyr)
library(leaflet)
library(DT)
eval = TRUE

current_version = 'v1.0'

path = '/Volumes/Transcend/ngen/CONUS-hydrofabric/pre-release/nextgen'

meta = data.frame(path = list.files(path,full.name = TRUE, pattern = "gpkg$")) %>% 
  filter(!grepl("_ext", path)) %>% 
  mutate(file = basename(path),
         v = gsub(".gpkg", "", gsub("nextgen_", "", file)),
         gpkg_link = glue::glue('https://nextgen-hydrofabric.s3.amazonaws.com/{current_version}/nextgen_{v}.gpkg'),
         vpu = paste0("<a href='",gpkg_link,"'>",v,"</a>")) 

vpus = nhdplusTools::get_boundaries() %>%
  filter(VPUID %in% meta$v) %>% 
  sf::st_transform('+proj=longlat +datum=WGS84')

for(i in 1:nrow(meta)){
  t = sf::st_layers(meta$path[i])
  meta$flowpaths[i] =  prettyNum(t$features[which(t$name == "flowpaths")],big.mark=",",scientific=FALSE)
  meta$divides[i] =  prettyNum(t$features[which(t$name == "divides")],big.mark=",",scientific=FALSE)
  meta$hydrolocations[i] =  prettyNum(t$features[which(t$name == "hydrolocations")],big.mark=",",scientific=FALSE)
  meta$nexus[i] =  prettyNum(t$features[which(t$name == "nexus")],big.mark=",",scientific=FALSE)
  meta$size[i] = paste0(round(file.size(meta$path[i]) / 1e6, 2), " Mb")
}

labels = paste("VPU", vpus$VPUID)

pop <- paste(
      paste0('<strong>GPKG: </strong>', meta$vpu ),
      paste("<strong>Flowpath:</strong>", meta$flowpaths),
      paste("<strong>Divides:</strong>", meta$divides),
      paste("<strong>Size:</strong>", meta$size),
      sep = "<br/>"
    )

bbox = as.numeric(sf::st_bbox(vpus))

NOAA OWP Next Generation Modeling Framework Hydrofabric

What is it?

Next Generation Modeling Framework (NextGen) hydrofabric artifacts are distributed by NHDPlusV2 Vector Processing Units. They are generated from a set of national reference datasets built in collaboration between NOAA, the USGS, and Lynker for federal water modeling efforts. These artifacts are designed to be easily updated, manipulated, and quality controlled to meet the needs of a wide range of modeling tasks while leveraging the best possible input data.

::: {#hello .blackbox .left} Citation:

Johnson, J. M. (2022). National Hydrologic Geospatial Fabric (hydrofabric) for the Next Generation (NextGen) Hydrologic Modeling Framework, HydroShare, http://www.hydroshare.org/resource/129787b468aa4d55ace7b124ed27dbde :::

How do I get it?

NextGen artifacts are publicly available through a partnership with Lynker and the NOAA OWP. For each VPU a geopackage that contains all tables, spatial data, and lookups relvant to a hydrofabric data model

These can be browsed here, or, programatically accessed using the respective URL patterns:

s3

s3://nextgen-hydrofabric/{version}/nextgen_{VPU}.gpkg
s3://nextgen-hydrofabric/{version}/nextgen_{VPU}.zip

#pal <- colorNumeric("BuPu", domain  = AOI$count, n = 10)

leaflet(width='100%') %>% 
  addProviderTiles(providers$CartoDB.Positron) %>% 
  addPolygons(data = vpus, 
              fillColor  = "gray", 
              color = "navy",
              fillOpacity = .3,
              weight = 1, 
              label = labels,
              popup = pop,
              highlightOptions = highlightOptions(color = "#FEBC11", weight = 5, bringToFront = FALSE, opacity = 1)) %>% 
  setMaxBounds(lng1 = bbox[1], lng2 = bbox[3], lat1 = bbox[2], lat2 = bbox[4])

library(DT)

getTotal <- function(index, data){

  if(index < 1 || index > ncol(data)){
    return("")
  }  

  col <- data[,index]
  col <- gsub("[Mb]","",col)
  col <- suppressWarnings(as.numeric(gsub(",", "", col)))
  if(all(is.na(col))){
    return("Totals")
  } else {
    return(prettyNum(sum(col), big.mark=",",scientific=FALSE))
  }
}

m = meta %>% 
  select(-file, -path, -gpkg_link, -v) 


m = rbind(m, sapply(1:ncol(m), function(x){ getTotal(x, m) }))
m[nrow(m), ncol(m)] = paste0(round(as.numeric(gsub(",", "", m[nrow(m), ncol(m)])) / 1000, 2), " GB")

  DT::datatable(m, escape = FALSE, width="100%",  filter = "none", 
                  rownames = T,
                options = list(autoWidth = T, 
                              pageLength = 25, 
                              scrollCollapse = T,
                              dom = 'lftp',
                              columnDefs = list(list(visible = F, targets = 0)))) %>% 
  DT::formatStyle(0, target = "row", fontWeight = styleEqual(dim(m)[1], "bold"))

:::{ .main-container}

A resource built on collaboration

The NextGen artifacts are a model application dataset built to meet the aims of NextGen. By design, these artifacts are derived from a set of general authoritative data products outlined in figure 1 that have been built in close collaboration with the USGS.

knitr::include_graphics("../man/figures/roadmap.png")

These include a set of base data that improves the network topology and geometry validity while defining a set of community hydrolocations (POIs). These 4 data products are used to build an intermediate refactored network from which one hydrofabric network has been aggregated to a set of community hydrolocations (minimal network), and one has been aggregated to a more consistent size (3-10 sqkm) with enforced POI locations (target distribution). NextGen specifically is derived from the target size aggregated product while the upcoming developments on the National Hydrologic Model (NHM) will be built from the community minimal network. While these two aggregations serve a wide range of federal modeling needs, our focus on open source software development and workflows allow interested parties to build there own networks starting with either the 4 reference datasets, or the refactored network!

Resources

• The hydrofabric builds on the OGC HY_Features conceptual model, the in prep Hydrofabric Logical model, and the proposed Hydrofabric Data Model.

• The base software for general hydrofabric development is based on nhdplusTools and hydrofab. The tools for extending these to be NextGen ready are found at ngen.hydrofab.

• The reference, refactor, minimal, and target hydrofabrics can all be accessed here. A high level introduction to these resources can be found on the USGS Water Data blog.

::: {#hello .blackbox .left} Disclaimer: These data are preliminary or provisional and are subject to revision. They are being provided to meet the need for timely best science. The data have not received final approval by the National Oceanic and Atmospheric Administration (NOAA) or the U.S. Geological Survey (USGS) and are provided on the condition that neither NOAA, the USGS, nor the U.S. Government shall be held liable for any damages resulting from the authorized or unauthorized use of the data. :::

:::

NOAA-OWP/hydrofabric documentation built on Dec. 7, 2024, 11:24 a.m.