In ISCN/SOCDRaHR2: Scripts to read in and harmonize different digital soil data archives

Purpose:

The International Soil Carbon Network Database vs 3.1 (ISCN3.1) is a collection of layer-level soil observations pulled together to explain and investigate soil carbon stocks world wide. ISCN 3.0 was published in 2015 (archived with EDI 2022), vs 3.1 addressed some known shortcomings and removed the ISCN inferred soil organic carbon to emphasize fit-for-purpose decisions needed for end use.

There are 29 contributing data sets with 434,119 layer-level observations across 56,926 geo-referenced locations and 10,934 un-referenced locations.

In this document we will present summary statistics for each contributing data set and illustrate how to work with the ISCN3.

library(SOCDRaH2)
library(tidyverse)
library(knitr)
library(ggplot2)
library(ggmap)
library(kableExtra)
dataDir <- '~/Documents/Datasets/ISCN'

knitr::opts_chunk$set(eval=TRUE, echo=FALSE)

#compare and confirm changes that cleaning things up did
ISCN_old <- ISCN3(dataDir=dataDir, orginalFormat = TRUE)

#read in the 'new' clean data set that we are basing these figures on
ISCN3.ls <- ISCN3_3(data_dir = dataDir)

#Create an overall summary of the entire collection.

site_locations <- ISCN3.ls$layer %>%
  #reduce size of dataset by only looking at necessary columns
  select(dataset_name_sub, site_name, #site identifiers
         `lat (dec. deg)`, `long (dec. deg)`) %>% #location information
  #remove duplicate entries of multiple layer observations at a given site
  unique() %>% 
  #create column as flag for lat/long entries to create counting groups that partitions locations into geolocated vs unlocated layers
  mutate(has_lat_long = is.finite(`lat (dec. deg)` + `long (dec. deg)`)) %>% 
  #for each of the datasets, count number of sites that are geolocated vs unlocated
  group_by(dataset_name_sub, has_lat_long) %>%
  tally() %>%
  #create informative column names and move tallies into appropriate columns
  mutate(my_label = if_else(has_lat_long, "geolocated_layer", "unlocated_layer")) %>%
  #remove unneeded column
  select(-has_lat_long) %>%
  #make the table more human readable
  pivot_wider(names_from = my_label, values_from = n) %>% 
  #do the same thing for profile and join
  full_join(ISCN3.ls$profile %>%
              select(dataset_name_sub, site_name, `lat (dec. deg)`, `long (dec. deg)`) %>%
              unique() %>%
              mutate(has_lat_long = is.finite(`lat (dec. deg)` + `long (dec. deg)`)) %>%
              group_by(dataset_name_sub, has_lat_long) %>%
              tally() %>%
              mutate(my_label = if_else(has_lat_long, "geolocated_profile", "unlocated_profile")) %>% 
              select(-has_lat_long) %>%
              pivot_wider(names_from = my_label, values_from = n), by = "dataset_name_sub")

table_summary <- ISCN3.ls$study %>% 
  #reduce size of dataset by only looking at necessary columns
  select(dataset_name, `dataset_type (dataset_type)`, dataset_description) %>% 
  #remove duplicate entries of datasets
  unique() %>% 
  #combine lat/long counts from site_locations table with respective datasets
  right_join(site_locations, by=c('dataset_name' = 'dataset_name_sub')) %>% 
  #create column showing total layer site count
  mutate(layer_site_count = sum(unlocated_layer, geolocated_layer, na.rm = TRUE)) %>% 
    #sort rows in ascending order based on layer site count
    arrange(layer_site_count) 


# sum(table_summary$geolocated_layer, na.rm = TRUE)
# sum(table_summary$unlocated_layer, na.rm = TRUE)

knitr::kable(table_summary, 
  caption = 'Site counts for each contributing dataset submision (dataname_sub) in the profile and layer data table.')

National Cooperative Soil Characterization Database (11 September 2014) (NRCS Sept/2014)

Layer-level soils data of pedons from the NRCS-Soil Survey Lab, National Cooperative Soil Characterization Database (version date: 11 September 2014). Sites are concentrated in the United states with a smaller number of international sites. NRCS Sept/2014 has 373,326 layer-level and 26,096 profile-level observations across 44,896 and 13,016 sites, respectively. Please see @Nrcs2014a and @Nrcs2014b for additional details.

ISCN vs 3 to 3.3 changes

• Columns were with soil organic carbon density (volume and area) were removed in the profile and layer tables since they were entirely filled in by ISCN.
• In the profile tables 14,756 rows were removed from ISCN3 that match the count of the ISCN SOC stock to 1m computation flag in the dataset_name_soc column implying that when that 1m computation was calculated, the other values in the ISCN contribution rows were duplicated from the other row in this original dataset.
• After cleaning for ISCN 3.3 there were no reduction in the number of layer level observations.
• There are some layers (525) with multiple rows many of which represent different organic carbon processing codes (cNRCS_prep).

Future issues

• Max/min bounds need to be worked out for this data.
• Profile information in layer table does not match all of the profile information in profile table

#This first data collection will be extensively commented, subsequent data sets will follow similar patterns.

#identify the current dataset
current_dataset <- "NRCS Sept/2014"

study <- ISCN3.ls$study %>% 
  #separate the dataset
  filter(dataset_name == current_dataset) %>%
  #remove columns with no information
  select(where(~any(!is.na(.)))) %>%
  #flip table to make more readable
  t()

profile <- ISCN3.ls$profile %>%
  #separate the data set based on the submission name (dataset_name_sub)
  filter(dataset_name_sub == current_dataset)%>% 
  #remove columns with no information
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>% 
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

##Pull out layers with more then one row
# duplicate_layers <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  # filter(length(layer_name) >= 2)

#random_slice <- layer %>%
#  slice_sample(n=100)

#ordered_slice <- layer[200:240,]

##Check to see if there is different/new information in the layer table that is not in the profile
##This code is left in for manual checks
# profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
# 
# series <- profile_info_layer %>%
#  select(profile_name, soil_taxon, soil_series) %>%
#  unique() %>%
#  anti_join(profile)



# layer_sites <- ISCN3.ls$layer %>% 
# filter(dataset_name_sub == current_dataset) %>%
# select(site_name)
# layer_sites_unique <- unique(layer_sites)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.')) 

#create factor table for layer data
layer %>% select_if(is.factor) %>% 
  #modify to be more readable
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  #count number of entries for all columns
  group_by_all() %>%
  #count how many of each factor there is
  tally() %>% 
  #list the most common first
  arrange(-n) %>%
  #this is over 42000 rows so trimming this down a bit
  slice_head(n = 3) %>% 
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

#if profile data exists, generate factor table with profile data
if(nrow(profile) != 0) { 
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    #count how many of each factor there is
    tally() %>%
    #list the most common first
    arrange(-n) %>%
    #this is over 17000 rows so trimming this down a bit
    slice_head(n = 3) %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts (subset of all catagories).')
}

#pull the unique locations
locations <- layer %>%
  #reduce size of dataset by only looking at necessary columns
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>% 
  #remove duplicate lat/long entries
  unique() %>%
  #remove entries with no information
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`)) 

#if profile data exists, assign locations using profile data instead
if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) { 
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group), fill = 'white', color = 'grey') +
  geom_hex(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), bins = 200)+#, color = 'red', size = 3) +
  scale_fill_gradient(name = 'Site count', trans = 'log10') +
  theme(axis.title = element_blank()) +
  labs(title = current_dataset)

#Layer level histograms and depth plots

#NRCS is so large that we only look at a representative subset here
set.seed(42)
subset_site <- sample(x = unique(c(layer$site_name, profile$site_name)), size = 1000)

#temp <- layer %>%
#  filter(site_name %in% subset_site)


#Again, the dataset is so large that we are subsetting

#pull out bd c and loi
oc_bd <- names(layer %>% select(starts_with('bd'), starts_with('c_tot'), starts_with('loi'), starts_with('oc')))

##oc_bd
plot.df <- layer %>%
         select(dataset_name_sub, site_name, profile_name,
                `layer_top (cm)`, `layer_bot (cm)`,
                all_of(oc_bd)) %>%
         filter(site_name %in% subset_site) %>%
         pivot_longer(cols = where(is.numeric) & one_of(oc_bd), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')

ggplot(data = plot.df %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = plot.df %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(depth + measurements))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name)),
            alpha = 0.8) +
  facet_wrap(~variable,scales = 'free')

#pull out p, n, and ph
p_n_ph <- names(layer %>% select(starts_with('p_'), 
                                 starts_with('n_'), starts_with('ph_'),
                                 starts_with('cec'), starts_with('bs'),
                                 starts_with('ecec')))

plot.df <- layer %>%
         select(dataset_name_sub, site_name, profile_name,
                `layer_top (cm)`, `layer_bot (cm)`,
                all_of(p_n_ph)) %>%
         filter(site_name %in% subset_site) %>%
         pivot_longer(cols = where(is.numeric) & one_of(p_n_ph), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')

ggplot(data = plot.df %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = plot.df %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(depth + measurements))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name)),
            alpha = 0.8) +
  facet_wrap(~variable,scales = 'free')

#everything else
other <- setdiff(names(layer %>% select(where(is.numeric))), c(oc_bd, p_n_ph,
                                                               "layer_top (cm)", "layer_bot (cm)",
                                                               "lat (dec. deg)", "long (dec. deg)"))
plot.df <- layer %>%
         select(dataset_name_sub, site_name, profile_name,
                `layer_top (cm)`, `layer_bot (cm)`,
                all_of(other)) %>%
         filter(site_name %in% subset_site) %>%
         pivot_longer(cols = where(is.numeric) & one_of(other), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')

ggplot(data = plot.df %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = plot.df %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(depth + measurements))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name)),
            alpha = 0.8) +
  facet_wrap(~variable,scales = 'free')

rm(plot.df, oc_bd, other, p_n_ph, subset_site)

Worldwide soil carbon and nitrogen data (Worldwide soil carbon and nitrogen data)

This data set is located in the United States. Worldwide soil carbon and nitrogen data has 21972 layer-level and 7423 profile-level observations across 1930 and 1834 sites, respectively. Please see @Zinke1986 for additional details and if you are using ISCN3 please cite.

ISCN 3 to 3.3

• 2401 rows were removed between ISCN3 and ISCN3.3 due to ??duplication??
• SOC calculation from ISCN were removed. (total of 3353 observations are SOC stock computation by ISCN )
• 466 duplicate observations with multiple rows of different organic carbon %, BD, pH, CEC values in the layer table
• 3296 observations in the profile table were duplicates => resulting in 4127 observations in profile table after removing the duplicates with 26 variables

Future issues

• Download and re-ingest from https://dx.doi.org/10.3334/CDIAC/lue.ndp018

#subset the data
current_dataset <- "Worldwide soil carbon and nitrogen data"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_WSCN <- layer %>%
# group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
# filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  arrange(-n) %>%
  slice_head(n = 3) %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    arrange(-n) %>%
    slice_head(n = 3) %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group), fill = 'white', color = 'grey') +
  geom_hex(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), bins = 200)+#, color = 'red', size = 3) +
  scale_fill_gradient(name = 'Site count', trans = 'log10') +
  theme(axis.title = element_blank()) +
  labs(title = current_dataset)

ggplot(data = profile %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)') & !starts_with('layer_'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins=20) + #make a histrogram
  facet_wrap(~variable,scales = 'free') +
  labs(title = 'Profile measurements')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins=20) + #make a histrogram
  facet_wrap(~variable,scales = 'free') +
  labs(title = 'Layer measurements')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')%>%
         filter(is.finite(measurements + depth))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name),
                alpha = 0.5)) +
  facet_wrap(~variable,scales = 'free') +
  labs(title = 'Depth profile')

USGS Site-specific soil C (USGS_S3C)

This data set is located in the United States. USGS_S3C has 10434 layer-level and 1674 profile-level observations across 1068 and 748 geolocated sites, respectively. Please see @Buell2004 for additional details.

ISCN vs 3 to 3.3 changes

• ISCN calculated soil organic carbon stocks were removed and duplicate data entries collapsed.
• 729 observations in the profile table were duplicates => resulting in 945 observations in profile table after removing the duplicates with 16 variables
• the ISCN SOC stock to 1m computation were found in the profile datatable
• No layer duplicates with length >= 2 were found

Future issues

• download and re-ingest from @Buell2004
• Clean up pH values (3 extraneous), clean up bulk density total (1 extraneous)
• Values in the country (country) column of both the dataset_layer and dataset_profile tables were reassigned to "United States" because this is the known location of the data points collected.

#subset the data
current_dataset <- "USGS_S3C"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

##Pull out layers with more then one row
# duplicate_layers_usgs <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  arrange(-n) %>%
  slice_head(n=3) %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    arrange(-n) %>%
    slice_head(n=3) %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('state'), aes(x=long, y = lat, group = group), fill = 'white', color = 'grey') +
  geom_hex(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), bins = 200)+#, color = 'red', size = 3) +
  scale_fill_gradient(name = 'Site count', trans = 'log10') +
  theme(axis.title = element_blank()) +
  labs(title = current_dataset)

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = profile %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)') &
                        !starts_with('layer'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(measurements + depth))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Alaska Deep Soil Carbon Project (AK DSC Project SOC stock computation)

This data set is located in the United States. AK DSC Project SOC stock computation has 9002 layer-level and 3159 profile-level observations across 743 and 743 sites, respectively. Please see @Johnson2011 for additional details.

ISCN 3 to 3.3 changes

• ISCN calculated soil organic carbon stocks were removed and the resulting duplicates removed. ?
• 1095 observations in the profile table were duplicates => resulting in 2064 observations in profile table after removing the duplicates with 26 variables
• No layer duplicates with length >= 2 were found

Future issues

• Check possible data downloads

Tables and figures

#subset the data
current_dataset <- "AK DSC Project SOC stock computation"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

##Pull out layers with more then one row
# duplicate_layers_akdsc <- layer %>%
 # group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
 # filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  arrange(-n) %>%
  slice_head(n=3) %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    arrange(-n) %>%
    slice_head(n=3) %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('state'), aes(x=long, y = lat, group = group), fill = 'white', color = 'grey') +
  geom_hex(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), bins = 200)+#, color = 'red', size = 3) +
  scale_fill_gradient(name = 'Site count', trans = 'log10') +
  theme(axis.title = element_blank()) +
  labs(title = current_dataset)

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = profile %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)') &
                        !starts_with('layer'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(measurements + depth))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Jorgensen_NPS (Jorgensen_NPS)

This data set is located in the United States. Jorgensen_NPS has 2531 layer-level and 0 profile-level observations across 529 and 0 sites, respectively. Please see @Jorgenson2008 for additional details and cite.

ISCN 3 to 3.3 changes

• No changes noted from 3 to 3.3
• No layer duplicates with length >= 2 were found

Future issues

• Check ISCN SOC calculations to see if any were removed.
• Go to https://www.nps.gov/im/vmi-wrst.htm and re-ingest the associated soil data.

#subset the data
current_dataset <- "Jorgensen_NPS"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

##Pull out layers with more then one row
# duplicate_layers_j_nps <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  arrange(-n) %>%
  slice_head(n=3) %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world') %>% filter(subregion == 'Alaska'), 
               aes(x=long, y = lat, group = group), fill = 'white', color = 'grey') +
  geom_hex(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), bins = 20)+#, color = 'red', size = 3) +
  scale_fill_gradient(name = 'Site count', trans = 'log10') +
  xlim(-200, -100)  +
  theme(axis.title = element_blank()) +
  labs(title = current_dataset)

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable') %>%
         filter(is.finite(measurements))) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

# ggplot(data = profile %>%
#          pivot_longer(cols = where(is.numeric) &
#                         !ends_with('(dec. deg)') &
#                         !starts_with('layer'), #pull out the numerics that aren't lat/long
#                       values_to = 'measurements', names_to = 'variable') %>%
#          filter(is.finite(measurements))) +
#   geom_histogram(aes(x = measurements), bins = 20) + #make a histrogram
#  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable') %>%
         filter(is.finite(measurements + depth))) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Northern Circumpolar Soil Carbon Database (Northern Circumpolar Soil Carbon Database (NCSCD))

This data set is located in the United States. Northern Circumpolar Soil Carbon Database (NCSCD) has 1907 layer-level and 1020 profile-level observations across 476 and 471 sites, respectively.

obs

• 510 observations in the profile table were duplicates => resulting in 510 observations in profile table after removing the duplicates with 22 variables
• 21 layer duplicates with length >= 2 were found in layer table
• 1875 missing rows in the layer table in ISCN3.3

Please see @Hugelius2013 for additional details and if you are using ISCN3 please cite.

#subset the data
current_dataset <- "Northern Circumpolar Soil Carbon Database (NCSCD)"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_ncscd <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >=2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows were removed from ISCN3 because of duplicate data entries in both layer and profile.
• Columns were removed from ISCN3 if they were SOC calculations.
• Special character '?' in hzn column in layer was reassigned to NA (so the column was then removed entirely.)

Future issues

• check for updated database then download, and re-ingest from @Hugelius2013

Jorgensen_ARCN (Jorgensen_ARCN)

This data set is located in the United States. Jorgensen_ARCN has 1108 layer-level and 0 profile-level observations across 316 and 0 sites, respectively. Please see @Jorgenson2008 for additional details.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Jorgensen_ARCN"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_j_arcn <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows weren't removed from ISCN3 because there were no duplicated entries within the layer-level and profile-level information.
• Columns were removed from ISCN3 in places where information was marked as 'NA' throughout the entirety of the column.
• Ggplot maps were modified to display a map centered around the Pacific Northwest and a second map focusing on the data collection site in Alaska.

Future issues

• Go to https://www.nps.gov/im/vmi-wrst.htm and re-ingest the associated soil data.

Bockheim (Bockheim)

Please see @Bockheim1998, @Bockheim1999, @Bockheim2001, @Bockheim2003 and @Bockheim2004 for additional details. This data set is located in the United States. Bockheim has 1611 layer-level and 46 profile-level observations across 126 and 24 sites, respectively.

• 20 observations in the profile table were duplicates => resulting in 26 observations in profile table after removing the duplicates with 22 variables
• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Bockheim"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_bock <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows were removed from ISCN3 for being duplicates.
• Columns were removed from ISCN3 for being SOC calculated.

Future issues

• Check that maps are reasonable for the location (wrong because of longitudes)
• Check longitude that looks like it might be wrong (dropped the negative for some)

Permafrost_RCN (Permafrost_RCN)

This data set is located in the United States. Permafrost_RCN has 3696 layer-level and 0 profile-level observations across XX and 0 sites, respectively. Please see @Harden2012 for additional details and if you are using ISCN3 please cite. Cite the individual author if using their data, in addition cite Harden if using that data as part of the broader Permafrost RCN dataset.

• No profile level observations are recorded for this dataset
• Consider site names instead for counting the number of sites for layer level observations (== 211 sites for layer level data?)
• No layer duplicates with length >= 2 were found
• No lat and long values are provided

#subset the data
current_dataset <- "Permafrost_RCN"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

# locations <- layer %>%
#               select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>%
#   unique() %>%
#   filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_p_rcn <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

# ggplot(locations) +
#   geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
#   geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
#   theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows were removed from ISCN3 because of gap-filling.
• Columns were removed from ISCN3 because they were entirely NA values, and the elevation (m) column in the profile-level data was innacurately filled with "10000."

Future issues

• fill in the lat-long and other data from the profile level information
• contact Schuur and Harden, see if they published the unpublished data they cited
• address elevation being incorrect

Lehmann_NE_US_soils (Lehmann NE US soils)

This data set is located in the United States. Lehmann NE US soils has 172 layer-level and 0 profile-level observations across 172 and 0 sites, respectively. The Lehmann NE US soils dataset references @Schmidt2000.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Lehmann NE US soils"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_leh_ne <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

# ggplot(data = layer %>%
#          pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
#                         !starts_with('layer') & #as long as they aren't layer bounds
#                         !ends_with('(dec. deg)'), #and aren't lat/lon
#                       #create the paired measurement-variable tables
#                       values_to = 'measurements', names_to = 'variable') %>%
#          pivot_longer(cols = c('layer_top (cm)',
#                                'layer_bot (cm)',), #pull out the layers
#                       #create a paired depth-variable
#                       values_to='depth', names_to = 'layer_variable')) +
#   #make nifty line plots that span the layer boundaries
#   geom_line(aes(x = depth, y = measurements,
#                  group = paste(dataset_name_sub, site_name, profile_name))) +
#   facet_wrap(~variable,scales = 'free')

Fixed issues

• Soil carbon stock and method were previously filled in ISCN3 and were removed in ISCN3.5
• ISCN3 had country listed as Unknown; ISCN3.5 now has country as United States since data is from NE US.

Future issues

None we are aware of

USDA-FS_NRS_LandscapeCarbonInventory (USDA-FS NRS Landscape Carbon Inventory)

This data set is located in the United States. USDA-FS NRS Landscape Carbon Inventory has 1510 layer-level and 0 profile-level observations across 143 and 0 sites, respectively. The USDA-FS NRS Landscape Carbon Inventory dataset references @Cole2013.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "USDA-FS NRS Landscape Carbon Inventory"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_fs_nrs <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows weren't removed from ISCN3 because there were no duplicated entries within the layer-level and profile-level information.
• Columns were removed from ISCN3 in places where information was marked as 'NA' throughout the entirety of the column.
• Values in the country (country) column of the layer-level and profile-level rows were replaced with 'United States', as they were previously marked as unknown (it is not observed in this ISCN3.3 version!). Values in the hzn and hzn_desgn columns of the profile level rows were originally filled with '?' and therefore, were replaced with a NA_character.
• Ggplot maps were modified to display a map centered around the North America and a second map focusing on the data collection sites in the United States.

Future issues

None we are aware of

Jorgensen_YKDE (Jorgensen_YKDE)

This data set is located in the United States. Jorgensen_YKDE has 697 layer-level and 0 profile-level observations across 60 and 0 sites, respectively. The Jorgensen_YKDE dataset references @Jorgenson2000.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Jorgensen_YKDE"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_j_ykde <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

# ggplot(data = layer %>%
#          pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
#                         !starts_with('layer') & #as long as they aren't layer bounds
#                         !ends_with('(dec. deg)'), #and aren't lat/lon
#                       #create the paired measurement-variable tables
#                       values_to = 'measurements', names_to = 'variable') %>%
#          pivot_longer(cols = c('layer_top (cm)',
#                                'layer_bot (cm)',), #pull out the layers
#                       #create a paired depth-variable
#                       values_to='depth', names_to = 'layer_variable')) +
#   #make nifty line plots that span the layer boundaries
#   geom_line(aes(x = depth, y = measurements,
#                  group = paste(dataset_name_sub, site_name, profile_name))) +
#   facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows weren't removed from ISCN3 because there were no SOC computations that needed removing.
• Columns were removed from ISCN3 to remove the three empty SOC columns.
• Citation rows were merged into one citation to fill NA values.
• This data set was set in Alaska, so the Alaska Location code from the Template was used instead of the generic.

Future issues

• For ISCN5, Jorgenson is spelled wrong in the actual data

Boby_Mack (Boby_Mack)

This data set is located in the United States. Boby_Mack has 482 layer-level and 0 profile-level observations across 38 and 0 sites, respectively. See @Boby2010 for additional details and if you are using ISCN3 please cite.

• No layer duplicates with length >= 2 were found
• ? is not replaced by NA in the hzn column of the layer table

#subset the data
current_dataset <- "Boby_Mack"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if (any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){
locations <- profile %>% 
  select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
  bind_rows(locations) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_boby_mack <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if (nrow(profile) != 0){
profile %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

Below is a list of know issues that were addressed between ISCN 3 and ISCN 3.3 + Repeated information on soil carbon stocks was removed from the layer and profile information of the data set. + There are no duplicate rows that needed to be removed, however, the soil carbon stock columns gap filled by ISCN were removed by ISCN3.5 cleaning. + The site note in the data set profile was expanded to a formal citation and added to the ISCN bib file. + Note that data recorded as "Unknown" or "?" was left as is and NOT shifted to an NA value. + In profile, the ISCN SOC to 1m computation rows were removed, getting rid of duplicate data

Future issues

None we are aware of

Lehmann_Soil_CBC_1 (Lehmann Soil C&BC #1)

This data set is located in the United States. Lehmann Soil C&BC #1 has 139 layer-level and 0 profile-level observations across 31 and 0 sites, respectively. The Lehmann Soil C&BC #1 dataset references @Schmidt2000.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Lehmann Soil C&BC #1"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_leh_cbc <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• 3 columns in layer and 4 columns in profile were removed where ISCN 3 tried to fill in SOC values.
• Special character '?' was reassigned with 'NA' in hzn column in layer.
• The country was reassigned from an unknown value to United States as all data is from the US.

Future issues

None we are aware of

USGS_Harden (USGS Harden)

This data set is located in the United States. USGS Harden has 1203 layer-level and 18 profile-level observations across 23 and 6 sites, respectively. The USGS Harden dataset is references @Harden2008a, @ODonnell2011, @Manies2004, @Harden2008b, @Harden2006 and @Neff2005.

• 8 observations in the profile table were duplicates => resulting in 10 observations in profile table after removing the duplicates with 21 variables
• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "USGS Harden"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_usgs_harden <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows weren't removed the dataset_layer table from ISCN3 because of there were no duplicated gap-filled entries.
• As it is such a large data set, the lack of duplicates was verified using a length(summary()) function.
• 4 columns were removed from the dataset_layer table from ISCN3 because these columns contained values from ISCN soc stock correction.
• 10 rows were removed from the dataset_profile table from ISCN3 because of ISCN soc gap-filling.
• 5 columns were removed from the dataset_profile table from ISCN3 because these columns contained values from ISCN soc stock correction.
• Ggplot maps were adjusted to be centered upon Alaska and narrowed in on location of specific data points.

Future issues

• Contains data citations (@Harden2008a @Harden2008b) that need to be revisited in ISCN 4 Both of the URL's provided lead to 404 error pages. [2] URL for dataone of Harden2008a: https://search.dataone.org/view/knb-lter-bnz.336.19 actual download URL leads to an error: http://www.lter.uaf.edu/404.php Contacting of researchers is necessary. [4] URL for dataone of Harden2008b: https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-bnz%2F334%2F18 actual download URL: http://www.lter.uaf.edu/data/data-detail/id/334

Bonanza_LTER (Bonanza LTER)

This data set is located in the United States. Bonanza LTER has 583 layer-level and 44 profile-level observations across 20 and 15 sites, respectively. The Bonanza_LTER dataset references @Cleve1993 and @Yarie2013.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Bonanza LTER"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- profile %>% 
  select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
  bind_rows(layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

# #Pull out layers with more then one row
# duplicate_layers_bonanza_lter <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

profile %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Profile-level catagorical variable counts.')

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

Below is a list of know issues that were addressed between ISCN 3 and ISCN 3.3

• In the dataset_profile, SOC computed values were removed and duplicate entries were removed.
• In the dataset_layer, SOC computed values were removed.

Future issues

• Data citation present, import directly from EDI for ISCN4

Lu_LTER(Lu_LTER)

The Lu_LTER data set in ISCN3 contains 103 layer-level observations and 14 profile-level observations after cleaning for ISCN3.5. The Lu_LTER data set references @Ping2000, @Ping2002, @Ping2004, @Ping2005, and @Michaelson1996.

• No layer duplicates with length >= 2 were found

current_dataset <- "Lu_LTER"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_lu_lter <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) > 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

Below is a list of know issues that were addressed between ISCN 3 and ISCN 3.3

• In profile, the ISCN SOC to 1m computation rows were removed, as well as any resulting duplicate data. (it is actually the other way round, the ISCN SOC to 1m computation rows were kept!) confirm again(vm)

Future issues

None we are aware of

HeckmanSwanstonBiscuitBurn (Heckman/Swanston Biscuit Burn)

This data set is located in the United States. Heckman/Swanston Biscuit Burn has 28 layer-level and 0 profile-level observations across 13 and 0 sites, respectively. The Heckman/Swanston Biscuit Burn dataset references @Heckman2013.

• No layer duplicates with length >= 2 were found
• ? in hzn column has not been replaced with NA

#subset the data
current_dataset <- "Heckman/Swanston Biscuit Burn"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_heck_swanston <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• This data set compared data for burned and unburned soils at the site of the 2002 Biscuit Fire.
• In ISCN3, the country was not set to United States despite the location of the collected data being identified as the state of Oregon. ISCN3.5 now identifies this as 'United States'.
• ISCN3 filled soil carbon stock computations that created duplicated data entries, which were then removed in ISCN3.5.

Future issues

• Follow up with Heckman to see if control at same lat/lon as Burn 7
• Follow up with Heckman about curator name discrepancy (101 says Kate but this document, 102, says Katherine)

Kane (Kane)

This data set is located in the United States. Kane has 574 layer-level and 0 profile-level observations across 12 and 0 sites, respectively. Please see @Kane2004, @Kane2005, @Kane2009, and @Valentine2004 for additional details.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Kane"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_kane <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

Below is a list of know issues that were addressed between ISCN 3 and ISCN 3.3 + Rows weren't removed from ISCN3 because there were no duplicated entries within the layer-level and profile-level information. + Columns were removed from ISCN3 where information was not available throughout the data set. + Unclear information on vegetation remains in the vegclass_local section of the layer-level and profile-level information. + This data set required the use of a map centered around Alaska. + In profile, the ISCN SOC to 1m computation rows were removed, getting rid of duplicate data

Future issues

• Inquire about cryptic vegetation information located in the vegclass_local columns
• Include the two data citations The two urls provided lead to 404 error pages. citation [2]: https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-bnz%2F133%2F19 or https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-bnz%2F133%2F18 Looks like same datasets with two different DOI's - contact repository to see if they want this handled a certain way citation [4]: https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-bnz%2F132%2F18 Download URL: https://pasta.lternet.edu/package/data/eml/knb-lter-bnz/132/18/e9bb75742dedc686e224c41ea6e33ad8 - the twelve pits data AND https://pasta.lternet.edu/package/data/eml/knb-lter-bnz/132/18/fc7153b558a9fa8d1f02ff61f8231fde - biophysical data and GPS coordinates

Lu_PIMA (Lu_PIMA)

This dataset contains 69 layer-level information and 2 profile-level observations of 11 and 2 unique latitude-longitude locations in the layer-level and profile-level tables. The Lu_PIMA data set references @Ping2010.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Lu_PIMA"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_lu_pima <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• In profile, the ISCN SOC to 1m computation rows were removed, getting rid of duplicate data.
• The layer tables were adjusted to reassign the value "NA" to a column with "?".
• The clean layer and clean profile table were checked for accuracy of information.
• A citation was added to the bibliography file.
• Four columns were removed from the original dataset that remove the soc columns from both the dataset_layer table and dataset_profile table.

Future issues

• Cite a database referenced in the dataset_study table
• The url links provided lead to a 404 error page. The correct url for dataone: https://search.dataone.org/view/knb-lter-bnz.366.16
• The datasite download link provided: http://www.lter.uaf.edu/404.php This link leads to a 404 error, contacting the researchers is necessary.

USGS Harden Yazoo (USGS Harden Yazoo)

This data set is located in the United States. USGS Harden Yazoo has 234 layer-level and 17 profile-level observations across 4 and 3 sites, respectively. The USGS Harden Yazoo dataset references @Harden1999 and @Huntington1998.

• No layer duplicates with length >= 2 were found
• In layer table, the 234 rows of the data are categorized based upon the USGS Harden Yazoo calculated SOC data

#subset the data
current_dataset <- "USGS Harden Yazoo"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_usgs_yazoo <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Half of the rows (234) of dataset_layer were removed from ISCN3 because they were repetitive entries from ISCN stock computation.
• 3 columns of dataset_layer were removed from ISCN3 because they were determined using soc.
• 17 rows of dataset_profile were removed from ISCN3 because they were filled using ISCN stock computation.
• 5 columns of dataset_profile were removed from ISCN 3 because they were determined using soc.
• Values in column country (country) were reassigned to "united States" in the dataset_layer and dataset_profile tables because it was the known location of the collection state. (it is not updated in the layer table - vm)
• Values in the "hzn" column of the dataset_layer table were reassigned to an NA character from "?", a nonspecific character. (hzn column is actually missing in this dataset! - vm)
• Ggplot maps were modified to ensure datasets accurately focused on Southeastern United States.
• Cryptic site notes in dataset_profile table were resolved to contain full citations.

Future issues

• Fix remaining SOC column in Dataset_layer table?

Vogel (Vogel)

This data set is located in the United States. Vogel has 230 layer-level and 0 profile-level observations across 8 and 0 sites, respectively. The Vogel dataset references @Kane2009, @Vogel2007, and @Vogel2008.

• No layer duplicates with length >= 2 were found
• ? is not replaced with NA in the hzn column in the layer table

#subset the data
current_dataset <- "Vogel"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if (any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_vogel <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if (nrow(profile) != 0)
{
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

Below is a list of know issues that were addressed between ISCN 3 and ISCN 3.3

• ISCN3 removed repeated information in citations, which was corrected and filled for ISCN3.5.
• ISCN3 also filled in the soil carbon stocks where bulk density and organic carbon was provided, this was removed for ISCN3.5.
• This data set was set in Alaska, so the Alaska Location code from the Template was used instead of the generic.

Future issues

• Find out where temperature and precipitation data is (currently not on ISCN)

USGS Muhs (USGS Muhs)

This data set is located in the United States. USGS Muhs has 395 layer-level and 0 profile-level observations across 6 and 0 sites, respectively. The USGS Muhs dataset references @Muhs2003.

• No layer duplicates with length >= 2 were found
• ? is not replaced with NA in the hzn column in the layer table

#subset the data
current_dataset <- "USGS Muhs"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_usgs_muhs <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Repeated information on soil carbon stocks was removed from the layer and profile information of the data set.
• There are no duplicate rows that needed to be removed, however, the soil carbon stock columns gap filled by ISCN were removed by ISCN3.5 cleaning.
• In addition, the site note in the data set profile was expanded to a formal citation and added to the ISCN bib file.

Future issues

None we are aware of

HeckmanLithosequence (Heckman lithosequence)

This data set is located in the United States. Heckman lithosequence has 46 layer-level and 12 profile-level observations across 4 and 4 sites, respectively. The Heckman lithosequence dataset is references @Heckman2009.

#subset the data
current_dataset <- "Heckman lithosequence"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){

  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_heck_lith <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• In ISCN3 the country was not set to United states originally despite the state being identified as Arizona, ISCN3.5 now identifies this as 'United States'.
• ISCN3 also filled in the soil carbon stocks where bulk density and organic carbon was provided, this was removed for ISCN3.5.

Future issues

• Follow up with Heckman to get key for root_quant_size and color.

Schuur (Schuur)

This data set is located in the United States. Schuur has 153 layer-level and 4 profile-level observations across 1 and 1 sites, respectively. The Schuur data set references @Schuur2007.

• No layer duplicates with length >= 2 were found
• 2 observations in the profile table were duplicates => resulting in 2 observations in profile table after removing the duplicates with 19 variables
• ? is not replaced with NA in the hzn column in the layer table

current_dataset <- "Schuur"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_schuur <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

In profile, the ISCN SOC to 1m computation rows were removed, getting rid of duplicate data.

Future issues

None that we are aware of

Myers_Smith (Myers-Smith)

This data set is located in the United States. Myers-Smith has 231 layer-level and 0 profile-level observations across 1 and 0 sites, respectively. The Myers-Smith dataset references @MyersSmithHarden2007 and @MyersSmithMcGuire2007.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Myers-Smith"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_myer_smith <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())


ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• ISCN3 removed repeated information in citations, so these column values were corrected and filled for ISCN3.5.
• ISCN3 filled in the soil carbon stocks where bulk density and organic carbon was provided, so these columns were removed for ISCN3.5.
• This data set was set in Alaska, so the Alaska Location code from the Template was used instead of the generic.

Future issues

• This dataset contains an actual data citation to be re-ingested in next ISCN update

OakRidgeNationalLab_Loblolly_DWJ (Oak Ridge National Lab_Loblolly_DWJ)

The Oak Ridge National Lab_Loblolly_DWJ dataset references @Parr2006. This data set is located in the United States. Oak Ridge National Lab_Loblolly_DWJ has 102 layer-level and 0 profile-level observations across 1 and 0 sites, respectively.

• No layer duplicates with length >= 2 were found

#subset the data
current_dataset <- "Oak Ridge National Lab_Loblolly_DWJ"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_oak_ridge <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• This dataset requires the country to be set to US. Additionally, the hzn in layer and profile needs to be reset to "NA" when the symbol "?" is present.
• In addition, gapfilled bd with sampling was set to "NA" based on method noted.
• ISCN SOC stock computation's were removed reducing the duplicated data entries.
• Dataset reference noted in a cryptic message, needs to be formly cited using bibtext. (https://doi.org/10.2172/921773)

Future issues

• reformat graphs and rest of document

OakRidgeNationalLab_TDE (Oak Ridge National Lab_TDE)

This data set is located in the United States. Oak Ridge National Lab_TDE has 1176 layer-level and 0 profile-level observations across 1 and 0 sites, respectively. The Oak Ridge National Lab_TDE dataset references @Froberg2008.

• No layer duplicates with length >= 2 were found
• ? in the hzn column needs to be replaced with NA

#subset the data
current_dataset <- "Oak Ridge National Lab_TDE"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>%
              select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)%>% #) %>%
  unique() %>%
  filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)){
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_oak_tde <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

Fixed issues

• Rows weren't removed from ISCN3 because there were no duplicated entries within the layer-level and profile-level information.
• Columns were removed from ISCN3 in places where information was marked as 'NA' throughout the entirety of the column.
• A reference and citation in the dataset_study information were replaced with a more expanded version. Additionally, cryptic site_note information in the profile level was also replaced with a more formal citation.
• Ggplot maps were modified to display a map centered around the United States and a second map focusing on the data collection site in Tennessee.

Future issues

None that we are aware of

University of Michigan Biological Station Forest Accelerated Succession ExperimenT (UMBS_FASET)

This data set is located in the United States. UMBS_FASET has 405 layer-level and 137 profile-level observations across 1 and 1 sites, respectively. The University of Michigan Biological Station Forest Accelerated Succession Experiment (UMBS_FASET) data set is uncited and unpublished.

• No layer duplicates with length >= 2 were found
• 364 duplicates were found in the layer table in ISCN3

#subset the data
current_dataset <- "UMBS_FASET"

study <- ISCN3.ls$study %>% 
  filter(dataset_name == current_dataset) %>%
  select(where(~any(!is.na(.)))) %>%
  t()

profile <- ISCN3.ls$profile %>%
  filter(dataset_name_sub == current_dataset)%>%
  select(where(~any(!is.na(.))))

layer <- ISCN3.ls$layer %>%
  filter(dataset_name_sub == current_dataset) %>%
  select(where(~any(!is.na(.))))

locations <- layer %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(layer %>%
                select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`)) %>%

    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))

if(any(ISCN3.ls$profile$dataset_name_sub == current_dataset)) {
  locations <- profile %>% 
    select(`lat (dec. deg)`, `long (dec. deg)`, `datum (datum)`) %>%
    bind_rows(locations) %>%
    unique() %>%
    filter(is.finite(`lat (dec. deg)` + `long (dec. deg)`))
}

# #Pull out layers with more then one row
# duplicate_layers_umbs <- layer %>%
#  group_by(dataset_name_sub, site_name, profile_name, layer_name) %>%
#  filter(length(layer_name) >= 2)

#Check to see if there is different/new information in the layer table that is not in the profile
#profile_info_layer <- layer %>%
#  select(any_of(setdiff(names(profile), c("layer_top (cm)", "layer_bot (cm)", "soc (g cm-2)")))) %>%
#  unique()
#
#series <- profile_info_layer %>% 
#  select(profile_name, soil_taxon, soil_series) %>% 
#  unique() %>%
#  anti_join(profile)

study %>%
  knitr::kable(caption = paste('Citations for the ', current_dataset, ' data set.'))

layer %>% select_if(is.factor) %>%
  pivot_longer(cols = everything(), names_to = 'column_name') %>%
  group_by_all() %>%
  tally() %>%
  knitr::kable(caption = 'Layer-level catagorical variable counts.')

if(nrow(profile) != 0) {
  profile %>% select_if(is.factor) %>%
    pivot_longer(cols = everything(), names_to = 'column_name') %>%
    group_by_all() %>%
    tally() %>%
    knitr::kable(caption = 'Profile-level catagorical variable counts.')
}

ggplot(locations) +
  geom_polygon(data = map_data('world'), aes(x=long, y = lat, group = group)) +
  geom_point(aes(x=`long (dec. deg)`, y = `lat (dec. deg)`), color = 'red', size = 3) +
  theme(axis.title = element_blank())

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) &
                        !ends_with('(dec. deg)'), #pull out the numerics that aren't lat/long
                      values_to = 'measurements', names_to = 'variable')) +
  geom_histogram(aes(x = measurements)) + #make a histrogram
  facet_wrap(~variable,scales = 'free')

ggplot(data = layer %>%
         pivot_longer(cols = where(is.numeric) & #pull out all the numerical colums
                        !starts_with('layer') & #as long as they aren't layer bounds
                        !ends_with('(dec. deg)'), #and aren't lat/lon
                      #create the paired measurement-variable tables
                      values_to = 'measurements', names_to = 'variable') %>%
         pivot_longer(cols = c('layer_top (cm)',
                               'layer_bot (cm)',), #pull out the layers
                      #create a paired depth-variable
                      values_to='depth', names_to = 'layer_variable')) +
  #make nifty line plots that span the layer boundaries
  geom_line(aes(x = depth, y = measurements,
                 group = paste(dataset_name_sub, site_name, profile_name))) +
  facet_wrap(~variable,scales = 'free')

#remove data from the layer that appears in the profile that is not the top/bottom/soc
#select all numerical data, pivot and histogram
#select all factor data and tally

Fixed issues

Below is a list of known issues that were addressed between ISCN 3 and ISCN 3.3 for the University of Michigan Biological Station Forest Accelerated Succession Experiment (UMBS_FASET) data set.

• The UMBS FASET data set is located in the United States and has 405 layer-level observations and 137 profile-level observations of 1 unique latitude-longitude location.
• ISCN SOC computation soil organic carbon calculations were removed and duplicate rows deleted.
• Values in the hzn column of the layer table were reassigned to an NA character from "?", a nonspecific character.
• Values in country (country) column were reassigned to "United States" in the layer and profile tables because it was the known location of the collection state.

Future issues

• Citation practices were initially incompatible with CC-BY license for ISCN3; contacting data providers about modifying release for ISCN4
• Data is unpublished, and references "100 yrs of research." Revisit in ISCN 4. Check and see if data is still being collected at this research site

ISCN/SOCDRaHR2 documentation built on May 26, 2023, 6:44 a.m.