Nothing
Introduction to the isdparser package
In isdparser: Parse 'NOAA' Integrated Surface Data Files
isdparser is an parser for ISD/ISD NOAA files
Code liberated from rnoaa to focus on ISD parsing since it's sorta
complicated. Has minimal dependencies, so you can parse your ISD/ISH
files without needing the deps that rnoaa needs. Will be used by
rnoaa once on CRAN.
Documentation at ftp://ftp.ncdc.noaa.gov/pub/data/noaa/ish-format-document.pdf
Package API:
isd_parse() - parse all lines in a file, with parallel optionisd_parse_line() - parse a single line - you choose which lines to parse
and how to apply the function to your linesisd_transform() - transform ISD data variablesisd_parse_csv() - parse csv format files
isd_parse_csv() parses NOAA ISD csv files, whereas isd_parse() and isd_parse_line()
both handle compressed files where each row of data is a string that needs to be parsed.
isd_parse_csv() is faster than isd_parse() because parsing each line takes some
time - although using isd_parse(parallel = TRUE) option gets closer to the speed of
isd_parse_csv().
Install
Stable from CRAN
install.packages("isdparser")
Dev version
remotes::install_github("ropensci/isdparser")
library("isdparser")
isd_parse_csv: parse a CSV file
Using a csv file included in the package:
path <- system.file('extdata/00702699999.csv', package = "isdparser")
isd_parse_csv(path)
#> # A tibble: 6,843 x 68
#> station date source latitude longitude elevation name
#> <int> <dttm> <int> <dbl> <dbl> <dbl> <chr>
#> 1 7.03e8 2017-02-10 14:04:00 4 0 0 7026 WXPO…
#> 2 7.03e8 2017-02-10 14:14:00 4 0 0 7026 WXPO…
#> 3 7.03e8 2017-02-10 14:19:00 4 0 0 7026 WXPO…
#> 4 7.03e8 2017-02-10 14:24:00 4 0 0 7026 WXPO…
#> 5 7.03e8 2017-02-10 14:29:00 4 0 0 7026 WXPO…
#> 6 7.03e8 2017-02-10 14:34:00 4 0 0 7026 WXPO…
#> 7 7.03e8 2017-02-10 14:39:00 4 0 0 7026 WXPO…
#> 8 7.03e8 2017-02-10 14:44:00 4 0 0 7026 WXPO…
#> 9 7.03e8 2017-02-10 14:49:00 4 0 0 7026 WXPO…
#> 10 7.03e8 2017-02-10 14:54:00 4 0 0 7026 WXPO…
#> # … with 6,833 more rows, and 61 more variables: report_type <chr>,
#> # call_sign <int>, quality_control <chr>, wnd <chr>, cig <chr>, vis <chr>,
#> # tmp <chr>, dew <chr>, slp <chr>, wind_direction <chr>,
#> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>,
#> # wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>, automated_atmospheric_condition_code <chr>,
#> # quality_automated_atmospheric_condition_code <chr>, coverage_code <chr>,
#> # coverage_quality_code <chr>, base_height_dimension <chr>,
#> # base_height_quality_code <chr>, cloud_type_code <chr>,
#> # cloud_type_quality_code <chr>, connective_cloud_attribute <chr>,
#> # vertical_datum_attribute <chr>, base_height_upper_range_attribute <chr>,
#> # base_height_lower_range_attribute <chr>, coverage <chr>,
#> # opaque_coverage <chr>, coverage_quality <chr>, lowest_cover <chr>,
#> # lowest_cover_quality <chr>, low_cloud_genus <chr>,
#> # low_cloud_genus_quality <chr>, lowest_cloud_base_height <chr>,
#> # lowest_cloud_base_height_quality <chr>, mid_cloud_genus <chr>,
#> # mid_cloud_genus_quality <chr>, high_cloud_genus <chr>,
#> # high_cloud_genus_quality <chr>, altimeter_setting_rate <chr>,
#> # altimeter_quality_code <chr>, station_pressure_rate <chr>,
#> # station_pressure_quality_code <chr>, speed_rate <chr>, quality_code <chr>,
#> # rem <chr>, eqd <chr>
Download a file first:
path <- file.path(tempdir(), "00702699999.csv")
x <- "https://www.ncei.noaa.gov/data/global-hourly/access/2017/00702699999.csv"
download.file(x, path)
isd_parse_csv(path)
#> # A tibble: 6,843 x 68
#> station date source latitude longitude elevation name
#> <int> <dttm> <int> <dbl> <dbl> <dbl> <chr>
#> 1 7.03e8 2017-02-10 14:04:00 4 0 0 7026 WXPO…
#> 2 7.03e8 2017-02-10 14:14:00 4 0 0 7026 WXPO…
#> 3 7.03e8 2017-02-10 14:19:00 4 0 0 7026 WXPO…
#> 4 7.03e8 2017-02-10 14:24:00 4 0 0 7026 WXPO…
#> 5 7.03e8 2017-02-10 14:29:00 4 0 0 7026 WXPO…
#> 6 7.03e8 2017-02-10 14:34:00 4 0 0 7026 WXPO…
#> 7 7.03e8 2017-02-10 14:39:00 4 0 0 7026 WXPO…
#> 8 7.03e8 2017-02-10 14:44:00 4 0 0 7026 WXPO…
#> 9 7.03e8 2017-02-10 14:49:00 4 0 0 7026 WXPO…
#> 10 7.03e8 2017-02-10 14:54:00 4 0 0 7026 WXPO…
#> # … with 6,833 more rows, and 61 more variables: report_type <chr>,
#> # call_sign <int>, quality_control <chr>, wnd <chr>, cig <chr>, vis <chr>,
#> # tmp <chr>, dew <chr>, slp <chr>, wind_direction <chr>,
#> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>,
#> # wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>, automated_atmospheric_condition_code <chr>,
#> # quality_automated_atmospheric_condition_code <chr>, coverage_code <chr>,
#> # coverage_quality_code <chr>, base_height_dimension <chr>,
#> # base_height_quality_code <chr>, cloud_type_code <chr>,
#> # cloud_type_quality_code <chr>, connective_cloud_attribute <chr>,
#> # vertical_datum_attribute <chr>, base_height_upper_range_attribute <chr>,
#> # base_height_lower_range_attribute <chr>, coverage <chr>,
#> # opaque_coverage <chr>, coverage_quality <chr>, lowest_cover <chr>,
#> # lowest_cover_quality <chr>, low_cloud_genus <chr>,
#> # low_cloud_genus_quality <chr>, lowest_cloud_base_height <chr>,
#> # lowest_cloud_base_height_quality <chr>, mid_cloud_genus <chr>,
#> # mid_cloud_genus_quality <chr>, high_cloud_genus <chr>,
#> # high_cloud_genus_quality <chr>, altimeter_setting_rate <chr>,
#> # altimeter_quality_code <chr>, station_pressure_rate <chr>,
#> # station_pressure_quality_code <chr>, speed_rate <chr>, quality_code <chr>,
#> # rem <chr>, eqd <chr>
isd_parse_line: parse lines from an ASCII strings file
path <- system.file('extdata/024130-99999-2016.gz', package = "isdparser")
lns <- readLines(path, encoding = "latin1")
isd_parse_line(lns[1])
#> # A tibble: 1 x 38
#> total_chars usaf_station wban_station date time date_flag latitude longitude
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 0054 024130 99999 2016… 0000 4 +60750 +012767
#> # … with 30 more variables: type_code <chr>, elevation <chr>,
#> # call_letter <chr>, quality <chr>, wind_direction <chr>,
#> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>,
#> # wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>,
#> # AW1_present_weather_observation_identifier <chr>,
#> # AW1_automated_atmospheric_condition_code <chr>,
#> # AW1_quality_automated_atmospheric_condition_code <chr>, REM_remarks <chr>,
#> # REM_identifier <chr>, REM_length_quantity <chr>, REM_comment <chr>
Or, give back a list
head(
isd_parse_line(lns[1], as_data_frame = FALSE)
)
#> $total_chars
#> [1] "0054"
#>
#> $usaf_station
#> [1] "024130"
#>
#> $wban_station
#> [1] "99999"
#>
#> $date
#> [1] "20160101"
#>
#> $time
#> [1] "0000"
#>
#> $date_flag
#> [1] "4"
Optionally don't include "Additional" and "Remarks" sections in parsed output.
isd_parse_line(lns[1], additional = FALSE)
#> # A tibble: 1 x 31
#> total_chars usaf_station wban_station date time date_flag latitude longitude
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 0054 024130 99999 2016… 0000 4 +60750 +012767
#> # … with 23 more variables: type_code <chr>, elevation <chr>,
#> # call_letter <chr>, quality <chr>, wind_direction <chr>,
#> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>,
#> # wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>
isd_parse: parse an ASCII strings file
Downloading a new file
path <- file.path(tempdir(), "007026-99999-2017.gz")
y <- "ftp://ftp.ncdc.noaa.gov/pub/data/noaa/2017/007026-99999-2017.gz"
download.file(y, path)
isd_parse(path)
#> # A tibble: 6,843 x 72
#> total_chars usaf_station wban_station date time date_flag latitude
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 0157 007026 99999 2017… 1404 4 +00000
#> 2 0157 007026 99999 2017… 1414 4 +00000
#> 3 0157 007026 99999 2017… 1419 4 +00000
#> 4 0157 007026 99999 2017… 1424 4 +00000
#> 5 0157 007026 99999 2017… 1429 4 +00000
#> 6 0144 007026 99999 2017… 1434 4 +00000
#> 7 0157 007026 99999 2017… 1439 4 +00000
#> 8 0157 007026 99999 2017… 1444 4 +00000
#> 9 0172 007026 99999 2017… 1449 4 +00000
#> 10 0157 007026 99999 2017… 1454 4 +00000
#> # … with 6,833 more rows, and 65 more variables: longitude <chr>,
#> # type_code <chr>, elevation <chr>, call_letter <chr>, quality <chr>,
#> # wind_direction <chr>, wind_direction_quality <chr>, wind_code <chr>,
#> # wind_speed <chr>, wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>, GF1_sky_condition <chr>, GF1_coverage <chr>,
#> # GF1_opaque_coverage <chr>, GF1_coverage_quality <chr>,
#> # GF1_lowest_cover <chr>, GF1_lowest_cover_quality <chr>,
#> # GF1_low_cloud_genus <chr>, GF1_low_cloud_genus_quality <chr>,
#> # GF1_lowest_cloud_base_height <chr>,
#> # GF1_lowest_cloud_base_height_quality <chr>, GF1_mid_cloud_genus <chr>,
#> # GF1_mid_cloud_genus_quality <chr>, GF1_high_cloud_genus <chr>,
#> # GF1_high_cloud_genus_quality <chr>, MA1_atmospheric_pressure <chr>,
#> # MA1_altimeter_setting_rate <chr>, MA1_altimeter_quality_code <chr>,
#> # MA1_station_pressure_rate <chr>, MA1_station_pressure_quality_code <chr>,
#> # REM_remarks <chr>, REM_identifier <chr>, REM_length_quantity <chr>,
#> # REM_comment <chr>, OC1_wind_gust_observation_identifier <chr>,
#> # OC1_speed_rate <chr>, OC1_quality_code <chr>,
#> # GA1_sky_cover_layer_identifier <chr>, GA1_coverage_code <chr>,
#> # GA1_coverage_quality_code <chr>, GA1_base_height_dimension <chr>,
#> # GA1_base_height_quality_code <chr>, GA1_cloud_type_code <chr>,
#> # GA1_cloud_type_quality_code <chr>, GE1_sky_condition <chr>,
#> # GE1_connective_cloud_attribute <chr>, GE1_vertical_datum_attribute <chr>,
#> # GE1_base_height_upper_range_attribute <chr>,
#> # GE1_base_height_lower_range_attribute <chr>,
#> # AW1_present_weather_observation_identifier <chr>,
#> # AW1_automated_atmospheric_condition_code <chr>,
#> # AW1_quality_automated_atmospheric_condition_code <chr>
Parallel
isd_parse(path, parallel = TRUE)
Progress
note: Progress not printed if parallel = TRUE
isd_parse(path, progress = TRUE)
#>
#> |========================================================================================| 100%
#> # A tibble: 2,601 × 42
#> total_chars usaf_station wban_station date time date_flag latitude longitude type_code
#> <dbl> <chr> <chr> <date> <chr> <chr> <dbl> <dbl> <chr>
#> 1 54 024130 99999 2016-01-01 0000 4 60.75 12.767 FM-12
#> 2 54 024130 99999 2016-01-01 0100 4 60.75 12.767 FM-12
#> 3 54 024130 99999 2016-01-01 0200 4 60.75 12.767 FM-12
#> 4 54 024130 99999 2016-01-01 0300 4 60.75 12.767 FM-12
#> 5 54 024130 99999 2016-01-01 0400 4 60.75 12.767 FM-12
#> 6 39 024130 99999 2016-01-01 0500 4 60.75 12.767 FM-12
#> 7 54 024130 99999 2016-01-01 0600 4 60.75 12.767 FM-12
#> 8 39 024130 99999 2016-01-01 0700 4 60.75 12.767 FM-12
#> 9 54 024130 99999 2016-01-01 0800 4 60.75 12.767 FM-12
#> 10 54 024130 99999 2016-01-01 0900 4 60.75 12.767 FM-12
#> # ... with 2,591 more rows, and 33 more variables: elevation <dbl>, call_letter <chr>, quality <chr>,
#> # wind_direction <dbl>, wind_direction_quality <chr>, wind_code <chr>, wind_speed <dbl>,
#> # wind_speed_quality <chr>, ceiling_height <chr>, ceiling_height_quality <chr>,
#> # ceiling_height_determination <chr>, ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>, visibility_code_quality <chr>,
#> # temperature <dbl>, temperature_quality <chr>, temperature_dewpoint <dbl>,
#> # temperature_dewpoint_quality <chr>, air_pressure <dbl>, air_pressure_quality <chr>,
#> # AW1_present_weather_observation_identifier <chr>, AW1_automated_atmospheric_condition_code <chr>,
#> # AW1_quality_automated_atmospheric_condition_code <chr>, N03_original_observation <chr>,
#> # N03_original_value_text <chr>, N03_units_code <chr>, N03_parameter_code <chr>, REM_remarks <chr>,
#> # REM_identifier <chr>, REM_length_quantity <chr>, REM_comment <chr>
Additional data
Optionally don't include "Additional" and "Remarks" sections in parsed output.
isd_parse(path, additional = FALSE)
#> # A tibble: 6,843 x 31
#> total_chars usaf_station wban_station date time date_flag latitude
#> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#> 1 0157 007026 99999 2017… 1404 4 +00000
#> 2 0157 007026 99999 2017… 1414 4 +00000
#> 3 0157 007026 99999 2017… 1419 4 +00000
#> 4 0157 007026 99999 2017… 1424 4 +00000
#> 5 0157 007026 99999 2017… 1429 4 +00000
#> 6 0144 007026 99999 2017… 1434 4 +00000
#> 7 0157 007026 99999 2017… 1439 4 +00000
#> 8 0157 007026 99999 2017… 1444 4 +00000
#> 9 0172 007026 99999 2017… 1449 4 +00000
#> 10 0157 007026 99999 2017… 1454 4 +00000
#> # … with 6,833 more rows, and 24 more variables: longitude <chr>,
#> # type_code <chr>, elevation <chr>, call_letter <chr>, quality <chr>,
#> # wind_direction <chr>, wind_direction_quality <chr>, wind_code <chr>,
#> # wind_speed <chr>, wind_speed_quality <chr>, ceiling_height <chr>,
#> # ceiling_height_quality <chr>, ceiling_height_determination <chr>,
#> # ceiling_height_cavok <chr>, visibility_distance <chr>,
#> # visibility_distance_quality <chr>, visibility_code <chr>,
#> # visibility_code_quality <chr>, temperature <chr>,
#> # temperature_quality <chr>, temperature_dewpoint <chr>,
#> # temperature_dewpoint_quality <chr>, air_pressure <chr>,
#> # air_pressure_quality <chr>
Try the isdparser package in your browser
Any scripts or data that you put into this service are public.
isdparser documentation built on Feb. 18, 2020, 1:09 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
isdparser is an parser for ISD/ISD NOAA files
Code liberated from rnoaa to focus on ISD parsing since it's sorta
complicated. Has minimal dependencies, so you can parse your ISD/ISH
files without needing the deps that rnoaa needs. Will be used by
rnoaa once on CRAN.
Documentation at ftp://ftp.ncdc.noaa.gov/pub/data/noaa/ish-format-document.pdf
Package API:
isd_parse()- parse all lines in a file, with parallel optionisd_parse_line()- parse a single line - you choose which lines to parse and how to apply the function to your linesisd_transform()- transform ISD data variablesisd_parse_csv()- parse csv format files
isd_parse_csv() parses NOAA ISD csv files, whereas isd_parse() and isd_parse_line()
both handle compressed files where each row of data is a string that needs to be parsed.
isd_parse_csv() is faster than isd_parse() because parsing each line takes some
time - although using isd_parse(parallel = TRUE) option gets closer to the speed of
isd_parse_csv().
Install
Stable from CRAN
install.packages("isdparser")
Dev version
remotes::install_github("ropensci/isdparser")
library("isdparser")
isd_parse_csv: parse a CSV file
Using a csv file included in the package:
path <- system.file('extdata/00702699999.csv', package = "isdparser") isd_parse_csv(path) #> # A tibble: 6,843 x 68 #> station date source latitude longitude elevation name #> <int> <dttm> <int> <dbl> <dbl> <dbl> <chr> #> 1 7.03e8 2017-02-10 14:04:00 4 0 0 7026 WXPO… #> 2 7.03e8 2017-02-10 14:14:00 4 0 0 7026 WXPO… #> 3 7.03e8 2017-02-10 14:19:00 4 0 0 7026 WXPO… #> 4 7.03e8 2017-02-10 14:24:00 4 0 0 7026 WXPO… #> 5 7.03e8 2017-02-10 14:29:00 4 0 0 7026 WXPO… #> 6 7.03e8 2017-02-10 14:34:00 4 0 0 7026 WXPO… #> 7 7.03e8 2017-02-10 14:39:00 4 0 0 7026 WXPO… #> 8 7.03e8 2017-02-10 14:44:00 4 0 0 7026 WXPO… #> 9 7.03e8 2017-02-10 14:49:00 4 0 0 7026 WXPO… #> 10 7.03e8 2017-02-10 14:54:00 4 0 0 7026 WXPO… #> # … with 6,833 more rows, and 61 more variables: report_type <chr>, #> # call_sign <int>, quality_control <chr>, wnd <chr>, cig <chr>, vis <chr>, #> # tmp <chr>, dew <chr>, slp <chr>, wind_direction <chr>, #> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>, #> # wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>, automated_atmospheric_condition_code <chr>, #> # quality_automated_atmospheric_condition_code <chr>, coverage_code <chr>, #> # coverage_quality_code <chr>, base_height_dimension <chr>, #> # base_height_quality_code <chr>, cloud_type_code <chr>, #> # cloud_type_quality_code <chr>, connective_cloud_attribute <chr>, #> # vertical_datum_attribute <chr>, base_height_upper_range_attribute <chr>, #> # base_height_lower_range_attribute <chr>, coverage <chr>, #> # opaque_coverage <chr>, coverage_quality <chr>, lowest_cover <chr>, #> # lowest_cover_quality <chr>, low_cloud_genus <chr>, #> # low_cloud_genus_quality <chr>, lowest_cloud_base_height <chr>, #> # lowest_cloud_base_height_quality <chr>, mid_cloud_genus <chr>, #> # mid_cloud_genus_quality <chr>, high_cloud_genus <chr>, #> # high_cloud_genus_quality <chr>, altimeter_setting_rate <chr>, #> # altimeter_quality_code <chr>, station_pressure_rate <chr>, #> # station_pressure_quality_code <chr>, speed_rate <chr>, quality_code <chr>, #> # rem <chr>, eqd <chr>
Download a file first:
path <- file.path(tempdir(), "00702699999.csv") x <- "https://www.ncei.noaa.gov/data/global-hourly/access/2017/00702699999.csv" download.file(x, path) isd_parse_csv(path) #> # A tibble: 6,843 x 68 #> station date source latitude longitude elevation name #> <int> <dttm> <int> <dbl> <dbl> <dbl> <chr> #> 1 7.03e8 2017-02-10 14:04:00 4 0 0 7026 WXPO… #> 2 7.03e8 2017-02-10 14:14:00 4 0 0 7026 WXPO… #> 3 7.03e8 2017-02-10 14:19:00 4 0 0 7026 WXPO… #> 4 7.03e8 2017-02-10 14:24:00 4 0 0 7026 WXPO… #> 5 7.03e8 2017-02-10 14:29:00 4 0 0 7026 WXPO… #> 6 7.03e8 2017-02-10 14:34:00 4 0 0 7026 WXPO… #> 7 7.03e8 2017-02-10 14:39:00 4 0 0 7026 WXPO… #> 8 7.03e8 2017-02-10 14:44:00 4 0 0 7026 WXPO… #> 9 7.03e8 2017-02-10 14:49:00 4 0 0 7026 WXPO… #> 10 7.03e8 2017-02-10 14:54:00 4 0 0 7026 WXPO… #> # … with 6,833 more rows, and 61 more variables: report_type <chr>, #> # call_sign <int>, quality_control <chr>, wnd <chr>, cig <chr>, vis <chr>, #> # tmp <chr>, dew <chr>, slp <chr>, wind_direction <chr>, #> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>, #> # wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>, automated_atmospheric_condition_code <chr>, #> # quality_automated_atmospheric_condition_code <chr>, coverage_code <chr>, #> # coverage_quality_code <chr>, base_height_dimension <chr>, #> # base_height_quality_code <chr>, cloud_type_code <chr>, #> # cloud_type_quality_code <chr>, connective_cloud_attribute <chr>, #> # vertical_datum_attribute <chr>, base_height_upper_range_attribute <chr>, #> # base_height_lower_range_attribute <chr>, coverage <chr>, #> # opaque_coverage <chr>, coverage_quality <chr>, lowest_cover <chr>, #> # lowest_cover_quality <chr>, low_cloud_genus <chr>, #> # low_cloud_genus_quality <chr>, lowest_cloud_base_height <chr>, #> # lowest_cloud_base_height_quality <chr>, mid_cloud_genus <chr>, #> # mid_cloud_genus_quality <chr>, high_cloud_genus <chr>, #> # high_cloud_genus_quality <chr>, altimeter_setting_rate <chr>, #> # altimeter_quality_code <chr>, station_pressure_rate <chr>, #> # station_pressure_quality_code <chr>, speed_rate <chr>, quality_code <chr>, #> # rem <chr>, eqd <chr>
isd_parse_line: parse lines from an ASCII strings file
path <- system.file('extdata/024130-99999-2016.gz', package = "isdparser") lns <- readLines(path, encoding = "latin1") isd_parse_line(lns[1]) #> # A tibble: 1 x 38 #> total_chars usaf_station wban_station date time date_flag latitude longitude #> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> #> 1 0054 024130 99999 2016… 0000 4 +60750 +012767 #> # … with 30 more variables: type_code <chr>, elevation <chr>, #> # call_letter <chr>, quality <chr>, wind_direction <chr>, #> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>, #> # wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>, #> # AW1_present_weather_observation_identifier <chr>, #> # AW1_automated_atmospheric_condition_code <chr>, #> # AW1_quality_automated_atmospheric_condition_code <chr>, REM_remarks <chr>, #> # REM_identifier <chr>, REM_length_quantity <chr>, REM_comment <chr>
Or, give back a list
head( isd_parse_line(lns[1], as_data_frame = FALSE) ) #> $total_chars #> [1] "0054" #> #> $usaf_station #> [1] "024130" #> #> $wban_station #> [1] "99999" #> #> $date #> [1] "20160101" #> #> $time #> [1] "0000" #> #> $date_flag #> [1] "4"
Optionally don't include "Additional" and "Remarks" sections in parsed output.
isd_parse_line(lns[1], additional = FALSE) #> # A tibble: 1 x 31 #> total_chars usaf_station wban_station date time date_flag latitude longitude #> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> #> 1 0054 024130 99999 2016… 0000 4 +60750 +012767 #> # … with 23 more variables: type_code <chr>, elevation <chr>, #> # call_letter <chr>, quality <chr>, wind_direction <chr>, #> # wind_direction_quality <chr>, wind_code <chr>, wind_speed <chr>, #> # wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>
isd_parse: parse an ASCII strings file
Downloading a new file
path <- file.path(tempdir(), "007026-99999-2017.gz") y <- "ftp://ftp.ncdc.noaa.gov/pub/data/noaa/2017/007026-99999-2017.gz" download.file(y, path) isd_parse(path) #> # A tibble: 6,843 x 72 #> total_chars usaf_station wban_station date time date_flag latitude #> <chr> <chr> <chr> <chr> <chr> <chr> <chr> #> 1 0157 007026 99999 2017… 1404 4 +00000 #> 2 0157 007026 99999 2017… 1414 4 +00000 #> 3 0157 007026 99999 2017… 1419 4 +00000 #> 4 0157 007026 99999 2017… 1424 4 +00000 #> 5 0157 007026 99999 2017… 1429 4 +00000 #> 6 0144 007026 99999 2017… 1434 4 +00000 #> 7 0157 007026 99999 2017… 1439 4 +00000 #> 8 0157 007026 99999 2017… 1444 4 +00000 #> 9 0172 007026 99999 2017… 1449 4 +00000 #> 10 0157 007026 99999 2017… 1454 4 +00000 #> # … with 6,833 more rows, and 65 more variables: longitude <chr>, #> # type_code <chr>, elevation <chr>, call_letter <chr>, quality <chr>, #> # wind_direction <chr>, wind_direction_quality <chr>, wind_code <chr>, #> # wind_speed <chr>, wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>, GF1_sky_condition <chr>, GF1_coverage <chr>, #> # GF1_opaque_coverage <chr>, GF1_coverage_quality <chr>, #> # GF1_lowest_cover <chr>, GF1_lowest_cover_quality <chr>, #> # GF1_low_cloud_genus <chr>, GF1_low_cloud_genus_quality <chr>, #> # GF1_lowest_cloud_base_height <chr>, #> # GF1_lowest_cloud_base_height_quality <chr>, GF1_mid_cloud_genus <chr>, #> # GF1_mid_cloud_genus_quality <chr>, GF1_high_cloud_genus <chr>, #> # GF1_high_cloud_genus_quality <chr>, MA1_atmospheric_pressure <chr>, #> # MA1_altimeter_setting_rate <chr>, MA1_altimeter_quality_code <chr>, #> # MA1_station_pressure_rate <chr>, MA1_station_pressure_quality_code <chr>, #> # REM_remarks <chr>, REM_identifier <chr>, REM_length_quantity <chr>, #> # REM_comment <chr>, OC1_wind_gust_observation_identifier <chr>, #> # OC1_speed_rate <chr>, OC1_quality_code <chr>, #> # GA1_sky_cover_layer_identifier <chr>, GA1_coverage_code <chr>, #> # GA1_coverage_quality_code <chr>, GA1_base_height_dimension <chr>, #> # GA1_base_height_quality_code <chr>, GA1_cloud_type_code <chr>, #> # GA1_cloud_type_quality_code <chr>, GE1_sky_condition <chr>, #> # GE1_connective_cloud_attribute <chr>, GE1_vertical_datum_attribute <chr>, #> # GE1_base_height_upper_range_attribute <chr>, #> # GE1_base_height_lower_range_attribute <chr>, #> # AW1_present_weather_observation_identifier <chr>, #> # AW1_automated_atmospheric_condition_code <chr>, #> # AW1_quality_automated_atmospheric_condition_code <chr>
Parallel
isd_parse(path, parallel = TRUE)
Progress
note: Progress not printed if
parallel = TRUE
isd_parse(path, progress = TRUE) #> #> |========================================================================================| 100% #> # A tibble: 2,601 × 42 #> total_chars usaf_station wban_station date time date_flag latitude longitude type_code #> <dbl> <chr> <chr> <date> <chr> <chr> <dbl> <dbl> <chr> #> 1 54 024130 99999 2016-01-01 0000 4 60.75 12.767 FM-12 #> 2 54 024130 99999 2016-01-01 0100 4 60.75 12.767 FM-12 #> 3 54 024130 99999 2016-01-01 0200 4 60.75 12.767 FM-12 #> 4 54 024130 99999 2016-01-01 0300 4 60.75 12.767 FM-12 #> 5 54 024130 99999 2016-01-01 0400 4 60.75 12.767 FM-12 #> 6 39 024130 99999 2016-01-01 0500 4 60.75 12.767 FM-12 #> 7 54 024130 99999 2016-01-01 0600 4 60.75 12.767 FM-12 #> 8 39 024130 99999 2016-01-01 0700 4 60.75 12.767 FM-12 #> 9 54 024130 99999 2016-01-01 0800 4 60.75 12.767 FM-12 #> 10 54 024130 99999 2016-01-01 0900 4 60.75 12.767 FM-12 #> # ... with 2,591 more rows, and 33 more variables: elevation <dbl>, call_letter <chr>, quality <chr>, #> # wind_direction <dbl>, wind_direction_quality <chr>, wind_code <chr>, wind_speed <dbl>, #> # wind_speed_quality <chr>, ceiling_height <chr>, ceiling_height_quality <chr>, #> # ceiling_height_determination <chr>, ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, visibility_code_quality <chr>, #> # temperature <dbl>, temperature_quality <chr>, temperature_dewpoint <dbl>, #> # temperature_dewpoint_quality <chr>, air_pressure <dbl>, air_pressure_quality <chr>, #> # AW1_present_weather_observation_identifier <chr>, AW1_automated_atmospheric_condition_code <chr>, #> # AW1_quality_automated_atmospheric_condition_code <chr>, N03_original_observation <chr>, #> # N03_original_value_text <chr>, N03_units_code <chr>, N03_parameter_code <chr>, REM_remarks <chr>, #> # REM_identifier <chr>, REM_length_quantity <chr>, REM_comment <chr>
Additional data
Optionally don't include "Additional" and "Remarks" sections in parsed output.
isd_parse(path, additional = FALSE) #> # A tibble: 6,843 x 31 #> total_chars usaf_station wban_station date time date_flag latitude #> <chr> <chr> <chr> <chr> <chr> <chr> <chr> #> 1 0157 007026 99999 2017… 1404 4 +00000 #> 2 0157 007026 99999 2017… 1414 4 +00000 #> 3 0157 007026 99999 2017… 1419 4 +00000 #> 4 0157 007026 99999 2017… 1424 4 +00000 #> 5 0157 007026 99999 2017… 1429 4 +00000 #> 6 0144 007026 99999 2017… 1434 4 +00000 #> 7 0157 007026 99999 2017… 1439 4 +00000 #> 8 0157 007026 99999 2017… 1444 4 +00000 #> 9 0172 007026 99999 2017… 1449 4 +00000 #> 10 0157 007026 99999 2017… 1454 4 +00000 #> # … with 6,833 more rows, and 24 more variables: longitude <chr>, #> # type_code <chr>, elevation <chr>, call_letter <chr>, quality <chr>, #> # wind_direction <chr>, wind_direction_quality <chr>, wind_code <chr>, #> # wind_speed <chr>, wind_speed_quality <chr>, ceiling_height <chr>, #> # ceiling_height_quality <chr>, ceiling_height_determination <chr>, #> # ceiling_height_cavok <chr>, visibility_distance <chr>, #> # visibility_distance_quality <chr>, visibility_code <chr>, #> # visibility_code_quality <chr>, temperature <chr>, #> # temperature_quality <chr>, temperature_dewpoint <chr>, #> # temperature_dewpoint_quality <chr>, air_pressure <chr>, #> # air_pressure_quality <chr>
Try the isdparser package in your browser
Any scripts or data that you put into this service are public.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.