README.md
In ptaconet/opendapr: Fast and Efficient Access to MODIS Earth Observation Data
modisfast
Table of contents
• Overview •
Installation •
Get started •
Data collections
available •
Manual testing of the
functionality • Foundational
framework •
Comparison with similar R
packages • Citation •
Future developments •
Contributing •
Acknowledgments
News
modisfast 1.0.0 now available on
CRAN, and associated
software paper published in the
Journal of Open Source Software !
Overview
modisfast is an R package designed for easy and fast
downloads of
MODIS
Land products,
VIIRS
Land products, and GPM (Global
Precipitation Measurement Mission) Earth Observation data.
modisfast uses the abilities offered by the
OPeNDAP framework (Open-source
Project for a Network Data Access Protocol) to download a subset of
Earth Observation data cube, along spatial, temporal or any other data
dimension (depth, …). This way, it reduces downloading time and disk
usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when
your region of interest is only 30 km x 30 km wide and you need 2 bands
! Moreover, modisfast enables parallel downloads of data.
This package is hence particularly suited for retrieving MODIS or VIIRS
data over long time series and over areas, rather than short
time series and points.
Importantly, the robust, sustainable, and cost-free foundational
framework of modisfast, both for the data
provider (NASA) and the software (R, OPeNDAP, the tidyverse and GDAL
suite of packages and software), guarantees the long-term reliability
and open-source nature of the package.
By enabling to download subsets of data cubes, modisfast facilites the
access to Earth science data for R users in places where internet
connection is slow or expensive and promotes digital sobriety for our
research work.
Installation
You can install the released version of modisfast from
CRAN with :
install.packages("modisfast")
or the development version (to get a bug fix or to use a feature from
the development version) with :
if(!require(devtools)){install.packages("devtools")}
devtools::install_github("ptaconet/modisfast")
Get Started
Accessing and opening MODIS data with modisfast is a simple 3-steps
workflow. This example shows how to download and import a one-year-long
monthly time series of MODIS Normalized Difference Vegetation Index
(NDVI) at 1 km spatial resolution over the whole country of Madagascar.
1/ First, define the variables of interest (ROI, time frame,
collection, and bands) :
# Load the packages
library(modisfast)
library(sf)
library(terra)
# ROI and time range of interest
roi <- st_as_sf(data.frame(id = "madagascar", geom = "POLYGON((41.95 -11.37,51.26 -11.37,51.26 -26.17,41.95 -26.17,41.95 -11.37))"), wkt = "geom", crs = 4326) # a ROI of interest, format sf polygon
time_range <- as.Date(c("2023-01-01", "2023-12-31")) # a time range of interest
# MODIS collections and variables (bands) of interest
collection <- "MOD13A3.061" # run mf_list_collections() for an exhaustive list of collections available
variables <- c("_1_km_monthly_NDVI") # run mf_list_variables("MOD13A3.061") for an exhaustive list of variables available for the collection "MOD13A3.061"
2/ Then, get the URL of the data and download them :
## Login to Earthdata servers with your EOSDIS credentials.
# To create an account (free) go to : https://urs.earthdata.nasa.gov/.
log <- mf_login(credentials = c("username", "password")) # set your own EOSDIS username and password
## Get the URLs of the data
urls <- mf_get_url(
collection = collection,
variables = variables,
roi = roi,
time_range = time_range
)
## Download the data. By default the data is downloaded in a temporary directory, but you can specify a folder
res_dl <- mf_download_data(urls, parallel = TRUE)
3/ And finally, import the data in R as a terra::SpatRaster object
using the function mf_import_data() ( :warning: see
here
why you should use this function, instead of the original
terra::rast(), in the context of modisfast) :
r <- mf_import_data(
path = dirname(res_dl$destfile[1]),
collection = collection,
proj_epsg = 4326
)
terra::plot(r, col = rev(terrain.colors(20)))
Time series of monthly 1-km MODIS NDVI
over Madagascar for the year 2023, retrieved with
modisfast
et voilà !
Want more examples ? modisfast provides three long-form documentations
and examples to learn more about the package :
- a “Get started”
article
describing the core features of the package;
- a “Get data on several regions or periods of interest simultaneously”
article
detailing advanced functionalities of
modisfast (for multi-time
frame or multi-regions data access);
- a “Full use case”
article
showcasing an example of use of the package in a scientific context
(here: landscape epidemiology).
Collections available in modisfast
Currently modisfast supports download of 77 data collections,
extracted from the following meta-collections :
- MODIS land
products
made available by the NASA / USGS LP DAAC
( :arrow_right: source OPeNDAP
server) ;
- VIIRS land
products
made available by the NASA / USGS LP DAAC
( :arrow_right: source OPeNDAP
server)
- Global Precipitation Measurement
(GPM) made available by the NASA / JAXA GES
DISC ( :arrow_right: source OPeNDAP
server).
Details of each product available for download are provided in the
tables below or through the function mf_list_collections().
Albedo data collections (click to expand)
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MCD43A1.061
MODIS
Albedo
MODIS/Terra and Aqua BRDF/Albedo Model Parameters Daily L3 Global 500 m
SIN Grid
500 m
Daily
2000-02-24 to present
MCD43A2.061
MODIS
Albedo
MODIS/Terra and Aqua BRDF/Albedo Quality Daily L3 Global 500 m SIN Grid
500 m
Daily
2000-02-24 to present
MCD43A3.061
MODIS
Albedo
MODIS/Terra and Aqua Albedo Daily L3 Global 500 m SIN Grid
500 m
Daily
2000-02-24 to present
VNP43MA1.001
VIIRS
Albedo
VIIRS/NPP BRDF/Albedo Model Parameters Daily L3 Global 1 km SIN
1000 m
Daily
2012-01-17 to present
VNP43MA2.001
VIIRS
Albedo
VIIRS/NPP BRDF/Albedo Quality Daily L3 Global 1 km SIN Grid
1000 m
Daily
2012-01-17 to present
VNP43MA3.001
VIIRS
Albedo
VIIRS/NPP Albedo Daily L3 Global 1 km SIN Grid
1000 m
Daily
2012-01-17 to present
VNP43IA2.001
VIIRS
Albedo
VIIRS/NPP BRDF/Albedo Quality Daily L3 Global 500 m SIN Grid
500 m
Daily
2012-01-17 to present
VNP43IA3.001
VIIRS
Albedo
VIIRS/NPP Albedo Daily L3 Global 500 m SIN Grid
500 m
Daily
2012-01-17 to present
Burned areas data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MCD64A1.061
MODIS
Burned areas
MODIS/Terra+Aqua Burned Area Monthly L3 Global 500 m SIN Grid
500 m
30 day
2000-11-01 to present
Evapotranspiration data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MOD16A2.061
MODIS
Evapotranspiration
MODIS/Terra Net Evapotranspiration 8-Day L4 Global 500m SIN Grid v061
500 m
8 day
2021-01-01 to present
MYD16A2.061
MODIS
Evapotranspiration
MODIS/Aqua Net Evapotranspiration 8-Day L4 Global 500m SIN Grid v061
500 m
8 day
2021-01-01 to present
MOD16A2GF.061
MODIS
Evapotranspiration
MODIS/Terra Net Evapotranspiration Gap-Filled 8-Day L4 Global 500 m SIN
Grid
500 m
8 day
2000-01-01 to present
MOD16A3GF.061
MODIS
Evapotranspiration
MODIS/Terra Net Evapotranspiration Gap-Filled Yearly L4 Global 500 m SIN
Grid
500 m
365 day
2000-02-18 to present
MYD16A2GF.061
MODIS
Evapotranspiration
MODIS/Aqua Net Evapotranspiration Gap-Filled 8-Day L4 Global 500 m SIN
Grid
500 m
8 day
2002-01-01 to present
MYD16A3GF.061
MODIS
Evapotranspiration
MODIS/Aqua Net Evapotranspiration Gap-Filled Yearly L4 Global 500 m SIN
Grid
500 m
365 day
2002-07-04 to present
Land cover data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MCD12Q1.061
MODIS
Land cover
MODIS/Terra+Aqua Land Cover Type Yearly L3 Global 500 m SIN Grid
500 m
365 day
2001-01-01 to present
Land surface temperature data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MOD11A1.061
MODIS
Land surface temperature
MODIS/Terra Land Surface Temperature/Emissivity Daily L3 Global 1km SIN
Grid v061
1000 m
Daily
2000-02-24 to present
MYD11A1.061
MODIS
Land surface temperature
MODIS/Aqua Land Surface Temperature/Emissivity Daily L3 Global 1km SIN
Grid v061
1000 m
Daily
2002-07-05 to present
MYD11A2.061
MODIS
Land surface temperature
MODIS/Aqua Land Surface Temperature/Emissivity 8-Day L3 Global 1 km SIN
Grid v061
1000 m
8 day
2002-07-04 to present
MOD11A2.061
MODIS
Land surface temperature
MODIS/Terra Land Surface Temperature/Emissivity 8-Day L3 Global 1 km SIN
Grid v061
1000 m
8 day
2000-02-18 to present
MOD11B3.061
MODIS
Land surface temperature
MODIS/Terra Land Surface Temperature/Emissivity Monthly L3 Global 6 km
SIN Grid
6000 m
30 day
2000-02-01 to present
MYD11B3.061
MODIS
Land surface temperature
MODIS/Aqua Land Surface Temperature/Emissivity Monthly L3 Global 6 km
SIN Grid
6000 m
30 day
2002-07-01 to present
MOD11B2.061
MODIS
Land surface temperature
MODIS/Terra Land Surface Temperature/Emissivity 8-Day L3 Global 6 km SIN
Grid
6000 m
8 day
2000-02-18 to present
MYD11B2.061
MODIS
Land surface temperature
MODIS/Aqua Land Surface Temperature/Emissivity 8-Day L3 Global 6 km SIN
Grid
6000 m
8 day
2002-07-04 to present
VNP21A1D.001
VIIRS
Land surface temperature
VIIRS/NPP Land Surface Temperature and Emissivity Daily L3 Global 1 km
SIN Grid Day
1000 m
Daily
2012-01-19 to present
VNP21A1N.001
VIIRS
Land surface temperature
VIIRS/NPP Land Surface Temperature and Emissivity Daily L3 Global 1 km
SIN Grid Night
1000 m
Daily
2012-01-19 to present
VNP21A2.001
VIIRS
Land surface temperature
VIIRS/NPP Land Surface Temperature and Emissivity 8-Day L3 Global 1 km
SIN Grid
1000 m
8 day
2012-01-25 to present
Leaf area index data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MYD15A2H.061
MODIS
Leaf area index
MODIS/Aqua Leaf Area Index/FPAR 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2002-07-04 to present
MCD15A2H.061
MODIS
Leaf area index
MODIS/Terra+Aqua Leaf Area Index/FPAR 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2002-07-04 to present
MCD15A3H.061
MODIS
Leaf area index
MODIS/Terra+Aqua Leaf Area Index/FPAR 4-Day L4 Global 500 m SIN Grid
500 m
4 day
2002-07-04 to present
MOD15A2H.061
MODIS
Leaf area index
MODIS/Terra Leaf Area Index/FPAR 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2000-02-18 to present
VNP15A2H.001
VIIRS
Leaf area index
VIIRS/NPP Leaf Area Index/FPAR 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2012-01-17 to present
Ocean Reflectance data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MODOCGA.061
MODIS
Ocean Reflectance
MODIS/Terra Ocean Reflectance Daily L2G-Lite Global 1 km SIN Grid
1000 m
Daily
2000-02-24 to present
MYDOCGA.061
MODIS
Ocean Reflectance
MODIS/Aqua Ocean Reflectance Daily L2G-Lite Global 1 km SIN Grid
1000 m
Daily
2002-07-04 to present
Primary Productivity data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MOD17A2H.061
MODIS
Primary Productivity
MODIS/Aqua Gross Primary Productivity 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2021-01-01 to present
MYD17A2H.061
MODIS
Primary Productivity
MODIS/Terra Gross Primary Productivity 8-Day L4 Global 500 m SIN Grid
500 m
8 day
2021-01-01 to present
MOD17A2HGF.061
MODIS
Primary Productivity
MODIS/Terra Gross Primary Productivity Gap-Filled 8-Day L4 Global 500 m
SIN Grid
500 m
8 day
2000-01-01 to present
MOD17A3HGF.061
MODIS
Primary Productivity
MODIS/Terra Net Primary Production Gap-Filled Yearly L4 Global 500 m SIN
Grid
500 m
365 day
2000-02-18 to present
MYD17A2HGF.061
MODIS
Primary Productivity
MODIS/Aqua Gross Primary Productivity Gap-Filled 8-Day L4 Global 500 m
SIN Grid
500 m
8 day
2002-01-01 to present
MYD17A3HGF.061
MODIS
Primary Productivity
MODIS/Aqua Net Primary Production Gap-Filled Yearly L4 Global 500 m SIN
Grid
500 m
365 day
2002-07-04 to present
Rainfall data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
GPM_3IMERGDE.06
GPM
Rainfall
GPM IMERG Early Precipitation L3 1 day 0.1 degree x 0.1 degree V06
10000 m
Daily
2000-06-01 to present
GPM_3IMERGDF.06
GPM
Rainfall
GPM IMERG Final Precipitation L3 1 day 0.1 degree x 0.1 degree V06
10000 m
Daily
2000-06-01 to present
GPM_3IMERGDF.07
GPM
Rainfall
GPM IMERG Final Precipitation L3 1 day 0.1 degree x 0.1 degree V07
10000 m
Daily
2000-06-01 to present
GPM_3IMERGDL.06
GPM
Rainfall
GPM IMERG Late Precipitation L3 1 day 0.1 degree x 0.1 degree V06
10000 m
Daily
2000-06-01 to present
GPM_3IMERGHH.06
GPM
Rainfall
GPM IMERG Final Precipitation L3 Half Hourly 0.1 degree x 0.1 degree V06
10000 m
30 minute
2000-06-01 to present
GPM_3IMERGHH.07
GPM
Rainfall
GPM IMERG Final Precipitation L3 Half Hourly 0.1 degree x 0.1 degree V07
10000 m
30 minute
2000-06-01 to present
GPM_3IMERGHHE.06
GPM
Rainfall
GPM IMERG Early Precipitation L3 Half Hourly 0.1 degree x 0.1 degree V06
10000 m
30 minute
2000-06-01 to present
GPM_3IMERGHHL.06
GPM
Rainfall
GPM IMERG Late Precipitation L3 Half Hourly 0.1 degree x 0.1 degree V06
10000 m
30 minute
2000-06-01 to present
GPM_3IMERGM.06
GPM
Rainfall
GPM IMERG Final Precipitation L3 1 month 0.1 degree x 0.1 degree V06
10000 m
1 month
2000-06-01 to present
GPM_3IMERGM.07
GPM
Rainfall
GPM IMERG Final Precipitation L3 1 month 0.1 degree x 0.1 degree V07
10000 m
1 month
2000-06-01 to present
Surface reflectance data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MOD09GA.061
MODIS
Surface reflectance
MODIS/Terra Surface Reflectance Daily L2G Global 1 km and 500 m SIN Grid
1000 m
Daily
2000-02-24 to present
MYD09GA.061
MODIS
Surface reflectance
MODIS/Aqua Surface Reflectance Daily L2G Global 1 km and 500 m SIN Grid
1000 m
Daily
2002-07-04 to present
MOD09GQ.061
MODIS
Surface reflectance
MODIS/Terra Surface Reflectance Daily L2G Global 250 m SIN Grid
250 m
Daily
2000-02-24 to present
MOD09Q1.061
MODIS
Surface reflectance
MODIS/Terra Surface Reflectance 8-Day L3 Global 250 m SIN Grid
250 m
8 day
2000-02-18 to present
MYD09GQ.061
MODIS
Surface reflectance
MODIS/Aqua Surface Reflectance Daily L2G Global 250 m SIN Grid
250 m
Daily
2002-07-04 to present
MYD09Q1.061
MODIS
Surface reflectance
MODIS/Aqua Surface Reflectance 8-Day L3 Global 250 m SIN Grid
250 m
8 day
2002-07-04 to present
MCD43A4.061
MODIS
Surface reflectance
MODIS/Terra and Aqua Nadir BRDF-Adjusted Reflectance Daily L3 Global 500
m SIN Grid
500 m
Daily
2000-02-24 to present
MOD09A1.061
MODIS
Surface reflectance
MODIS/Terra Surface Reflectance 8-Day L3 Global 500 m SIN Grid
500 m
8 day
2000-02-18 to present
MYD09A1.061
MODIS
Surface reflectance
MODIS/Aqua Surface Reflectance 8-Day L3 Global 500 m SIN Grid
500 m
8 day
2002-07-04 to present
VNP09A1.001
VIIRS
Surface reflectance
VIIRS/NPP Surface Reflectance 8-Day L3 Global 1 km SIN Grid
1000 m
8 day
2012-01-17 to present
VNP43MA4.001
VIIRS
Surface reflectance
VIIRS/NPP Nadir BRDF-Adjusted Reflectance Daily L3 Global 1 km SIN
1000 m
Daily
2012-01-17 to present
VNP43IA4.001
VIIRS
Surface reflectance
VIIRS/NPP Nadir BRDF-Adjusted Reflectance Daily L3 Global 500 m SIN Grid
500 m
Daily
2012-01-17 to present
VNP09H1.001
VIIRS
Surface reflectance
VIIRS/NPP Surface Reflectance 8-Day L3 Global 500 m SIN Grid
500 m
8 day
2012-01-17 to present
Thermal Anomalies/Fire data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
VNP14A1.001
VIIRS
Thermal Anomalies/Fire
VIIRS/NPP Thermal Anomalies/Fire Daily L3 Global 1 km SIN Grid
1000 m
Daily
2012-01-19 to present
Thermal Bands data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MODTBGA.061
MODIS
Thermal Bands
MODIS/Terra Thermal Bands Daily L2G-Lite Global 1 km SIN Grid
1000 m
Daily
2000-02-24 to present
MYDTBGA.061
MODIS
Thermal Bands
MODIS/Aqua Thermal Bands Daily L2G-Lite Global 1 km SIN Grid
1000 m
Daily
2002-07-04 to present
Vegetation indices data collections
Collection
Source
Type
Name
Spatial resolution
Temporal resolution
Temporal extent
MOD13A2.061
MODIS
Vegetation indices
MODIS/Terra Vegetation Indices 16-Day L3 Global 1 km SIN Grid
1000 m
16 day
2000-02-18 to present
MOD13A3.061
MODIS
Vegetation indices
MODIS/Terra Vegetation Indices Monthly L3 Global 1 km SIN Grid
1000 m
30 day
2000-02-01 to present
MYD13A2.061
MODIS
Vegetation indices
MODIS/Aqua Vegetation Indices 16-Day L3 Global 1 km SIN Grid
1000 m
16 day
2002-07-04 to present
MYD13A3.061
MODIS
Vegetation indices
MODIS/Aqua Vegetation Indices Monthly L3 Global 1 km SIN Grid
1000 m
30 day
2002-07-01 to present
MOD13Q1.061
MODIS
Vegetation indices
MODIS/Terra Vegetation Indices 16-Day L3 Global 250m SIN Grid v061
250 m
16 day
2000-02-18 to present
MYD13Q1.061
MODIS
Vegetation indices
MODIS/Aqua Vegetation Indices 16-Day L3 Global 250m SIN Grid v061
250 m
16 day
2002-07-04 to present
MOD13A1.061
MODIS
Vegetation indices
MODIS/Terra Vegetation Indices 16-Day L3 Global 500 m SIN Grid
500 m
16 day
2000-02-18 to present
MYD13A1.061
MODIS
Vegetation indices
MODIS/Aqua Vegetation Indices 16-Day L3 Global 500 m SIN Grid
500 m
16 day
2002-07-04 to present
VNP13A2.001
VIIRS
Vegetation indices
VIIRS/NPP Vegetation Indices 16-Day L3 Global 1 km SIN Grid
1000 m
16 day
2012-01-17 to present
VNP13A3.001
VIIRS
Vegetation indices
VIIRS/NPP Vegetation Indices Monthly L3 Global 1 km SIN Grid
1000 m
30 day
2012-01-01 to present
VNP13A1.001
VIIRS
Vegetation indices
VIIRS/NPP Vegetation Indices 16-Day L3 Global 500 m SIN Grid
500 m
16 day
2012-01-17 to present
Manual testing of the functionality
Since most modisfast functions depend on EarthData credentials,
automated tests are disabled. However, after installation, users can
manually test the package’s functionality by running these lines of code
:
# replace "username" and "password" with your own EOSDIS (Earthdata) credentials
earthdata_un <- "username"
earthdata_pw <- "password"
devtools::test("modisfast")
Foundational framework
Technically, modisfast is a programmatic interface (R wrapper) to
several NASA OPeNDAP servers. OPeNDAP is the
acronym for Open-source Project for a Network Data Access Protocol and
designates both the software, the access protocol, and the corporation
that develops them. The OPeNDAP is designed to simplify access to
structured and high-volume data, such as satellite products, over the
Web. It is a collaborative effort involving multiple institutions and
companies, with open-source code, free software, and adherence to the
Open Geospatial Consortium (OGC) standards. It
is widely used by NASA, which partly finances it.
A key feature of OPeNDAP is its capability to apply filters at the data
download process, ensuring that only the necessary data is retrieved.
These filters, specified within a URL, can be spatial, temporal, or
dimensional. Although powerful, OPeNDAP URLs are not trivial to build.
modisfast facilitates this process by constructing the URL based on
the spatial, temporal, and dimensional filters provided by the user in
the function mf_get_url().
Let’s take an example to understand.
The following URL :arrow_down:
https://opendap.cr.usgs.gov/opendap/hyrax/MOD11A2.061/h17v08.ncml.nc4?MODIS_Grid_8Day_1km_LST_eos_cf_projection,LST_Day_1km[775:793][55:140][512:560],LST_Night_1km[775:793][55:140][512:560],QC_Day[775:793][55:140][512:560],QC_Night[775:793][55:140][512:560],time[775:793],YDim[55:140],XDim[512:560]
is a link to download the following subset of MOD11A2.061 data in netCDF
:
- bands LST_Day_1km, LST_Night_1km, QC_Day, QC_Night ;
- each available date between the 2017-01-01 and the 2017-06-01 ;
- within the following bounding box (lon/lat): -5.41 8.84, -5.82 9.54.
The indices within the [] refer to values encoding for the spatial and
temporal filters.
These OPeNDAP URLs are not trivial to build. modisfast converts the
spatial, temporal and dimensional filters (R objects) provided by the
user through the function mf_get_url() into the appropriate OPeNDAP
URL(s). Subsequently, the function mf_download_data() allows for
downloading the data using the
httr and parallel
packages.
Comparison with similar R packages
There are other R packages available for accessing MODIS data. Below is
a comparison of modisfast with other packages available for downloading
chunks of MODIS or VIIRS data :
| Package | Data | Available on CRAN | Utilizes open standards for data access protocols | Spatial subsetting* | Dimensional subsetting* | Maximum area size allowed for download | Speed** |
|:-------------------------------------------------------------:|:----------------------------------:|:------------------:|:-------------------------------------------------:|:--------------------:|:------------------------:|:--------------------------------------:|:------------------:|
| modisfast | MODIS, VIIRS, GPM | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | unlimited | :white_check_mark: |
| appeears | MODIS, VIIRS, and many others | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | unlimited | variable |
| MODISTools | MODIS, VIIRS | :white_check_mark: | :x: | :white_check_mark: | :white_check_mark: | 200 km x 200 km | :white_check_mark: |
| rgee | MODIS, VIIRS, GPM, and many others | :white_check_mark: | :x: | :white_check_mark: | :white_check_mark: | unlimited | not tested |
| MODIStsp | MODIS | :x: | | :x: | :white_check_mark: | unlimited | NA |
| MODIS | MODIS | :x: | :x: | :x: | :x: | NA | NA |
* at the downloading phase
** Take a look at the article “Comparison of performance with other
similar R
packages”
to get an overview of how modisfast compares to these packages in
terms of data access time.
Citation
This package is licensed under a GNU General Public License v3.0 or
later license.
We thank in advance people that use modisfast for citing it in their
work / publication(s). For this, please use the following citation :
Taconet et al., (2024). modisfast: An R package for fast and efficient
access to MODIS, VIIRS and GPM Earth Observation data. Journal of Open
Source Software, 9(103), 7343, https://doi.org/10.21105/joss.07343
Future developments
Future developments of the package may include access to additional data
collections from other OPeNDAP servers, and support for a variety of
data formats as they become available from data providers through their
OPeNDAP servers. Furthermore, the creation of an RShiny application on
top of the package is being considered, as a means of further
simplifying data access for users with limited coding skills.
Contributing
All types of contributions are encouraged and valued. For more
information, check out our Contributor
Guidelines.
Please note that the modisfast project is released with a Contributor
Code of
Conduct.
By contributing to this project, you agree to abide by its terms.
Acknowledgments
We thank NASA and its partners for making all their Earth science data
freely available, and implementing open data access protocols such as
OPeNDAP. modisfast heavily builds on top of the OPeNDAP, so we thank
the non-profit OPeNDAP, Inc. for
developing the eponym tool in an open and collaborative way.
We also thank the contributors that have tested the package, reviewed
the documentation and brought valuable feedbacks to improve the package
: Florian de Boissieu, Julien
Taconet.
This work has been developed over the course of several research
projects (REACT 1, REACT 2, ANORHYTHM and DIV-YOO) funded by Expertise
France, the French National Research Agency (ANR), and the French
National Research Institute for Sustainable Development (IRD).
ptaconet/opendapr documentation built on Nov. 20, 2024, 10:04 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
modisfast
Table of contents
• Overview • Installation • Get started • Data collections available • Manual testing of the functionality • Foundational framework • Comparison with similar R packages • Citation • Future developments • Contributing • Acknowledgments
News
modisfast 1.0.0 now available on
CRAN, and associated
software paper published in the
Journal of Open Source Software !
Overview
modisfast is an R package designed for easy and fast
downloads of
MODIS
Land products,
VIIRS
Land products, and GPM (Global
Precipitation Measurement Mission) Earth Observation data.
modisfast uses the abilities offered by the
OPeNDAP framework (Open-source
Project for a Network Data Access Protocol) to download a subset of
Earth Observation data cube, along spatial, temporal or any other data
dimension (depth, …). This way, it reduces downloading time and disk
usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when
your region of interest is only 30 km x 30 km wide and you need 2 bands
! Moreover, modisfast enables parallel downloads of data.
This package is hence particularly suited for retrieving MODIS or VIIRS data over long time series and over areas, rather than short time series and points.
Importantly, the robust, sustainable, and cost-free foundational
framework of modisfast, both for the data
provider (NASA) and the software (R, OPeNDAP, the tidyverse and GDAL
suite of packages and software), guarantees the long-term reliability
and open-source nature of the package.
By enabling to download subsets of data cubes, modisfast facilites the
access to Earth science data for R users in places where internet
connection is slow or expensive and promotes digital sobriety for our
research work.
Installation
You can install the released version of modisfast from
CRAN with :
install.packages("modisfast")
or the development version (to get a bug fix or to use a feature from the development version) with :
if(!require(devtools)){install.packages("devtools")}
devtools::install_github("ptaconet/modisfast")
Get Started
Accessing and opening MODIS data with modisfast is a simple 3-steps
workflow. This example shows how to download and import a one-year-long
monthly time series of MODIS Normalized Difference Vegetation Index
(NDVI) at 1 km spatial resolution over the whole country of Madagascar.
1/ First, define the variables of interest (ROI, time frame, collection, and bands) :
# Load the packages
library(modisfast)
library(sf)
library(terra)
# ROI and time range of interest
roi <- st_as_sf(data.frame(id = "madagascar", geom = "POLYGON((41.95 -11.37,51.26 -11.37,51.26 -26.17,41.95 -26.17,41.95 -11.37))"), wkt = "geom", crs = 4326) # a ROI of interest, format sf polygon
time_range <- as.Date(c("2023-01-01", "2023-12-31")) # a time range of interest
# MODIS collections and variables (bands) of interest
collection <- "MOD13A3.061" # run mf_list_collections() for an exhaustive list of collections available
variables <- c("_1_km_monthly_NDVI") # run mf_list_variables("MOD13A3.061") for an exhaustive list of variables available for the collection "MOD13A3.061"
2/ Then, get the URL of the data and download them :
## Login to Earthdata servers with your EOSDIS credentials.
# To create an account (free) go to : https://urs.earthdata.nasa.gov/.
log <- mf_login(credentials = c("username", "password")) # set your own EOSDIS username and password
## Get the URLs of the data
urls <- mf_get_url(
collection = collection,
variables = variables,
roi = roi,
time_range = time_range
)
## Download the data. By default the data is downloaded in a temporary directory, but you can specify a folder
res_dl <- mf_download_data(urls, parallel = TRUE)
3/ And finally, import the data in R as a terra::SpatRaster object
using the function mf_import_data() ( :warning: see
here
why you should use this function, instead of the original
terra::rast(), in the context of modisfast) :
r <- mf_import_data(
path = dirname(res_dl$destfile[1]),
collection = collection,
proj_epsg = 4326
)
terra::plot(r, col = rev(terrain.colors(20)))
modisfast
et voilà !
Want more examples ? modisfast provides three long-form documentations
and examples to learn more about the package :
- a “Get started” article describing the core features of the package;
- a “Get data on several regions or periods of interest simultaneously”
article
detailing advanced functionalities of
modisfast(for multi-time frame or multi-regions data access); - a “Full use case” article showcasing an example of use of the package in a scientific context (here: landscape epidemiology).
Collections available in modisfast
Currently modisfast supports download of 77 data collections,
extracted from the following meta-collections :
- MODIS land products made available by the NASA / USGS LP DAAC ( :arrow_right: source OPeNDAP server) ;
- VIIRS land products made available by the NASA / USGS LP DAAC ( :arrow_right: source OPeNDAP server)
- Global Precipitation Measurement (GPM) made available by the NASA / JAXA GES DISC ( :arrow_right: source OPeNDAP server).
Details of each product available for download are provided in the
tables below or through the function mf_list_collections().
Manual testing of the functionality
Since most modisfast functions depend on EarthData credentials,
automated tests are disabled. However, after installation, users can
manually test the package’s functionality by running these lines of code
:
# replace "username" and "password" with your own EOSDIS (Earthdata) credentials
earthdata_un <- "username"
earthdata_pw <- "password"
devtools::test("modisfast")
Foundational framework
Technically, modisfast is a programmatic interface (R wrapper) to
several NASA OPeNDAP servers. OPeNDAP is the
acronym for Open-source Project for a Network Data Access Protocol and
designates both the software, the access protocol, and the corporation
that develops them. The OPeNDAP is designed to simplify access to
structured and high-volume data, such as satellite products, over the
Web. It is a collaborative effort involving multiple institutions and
companies, with open-source code, free software, and adherence to the
Open Geospatial Consortium (OGC) standards. It
is widely used by NASA, which partly finances it.
A key feature of OPeNDAP is its capability to apply filters at the data
download process, ensuring that only the necessary data is retrieved.
These filters, specified within a URL, can be spatial, temporal, or
dimensional. Although powerful, OPeNDAP URLs are not trivial to build.
modisfast facilitates this process by constructing the URL based on
the spatial, temporal, and dimensional filters provided by the user in
the function mf_get_url().
Let’s take an example to understand.
The following URL :arrow_down:
https://opendap.cr.usgs.gov/opendap/hyrax/MOD11A2.061/h17v08.ncml.nc4?MODIS_Grid_8Day_1km_LST_eos_cf_projection,LST_Day_1km[775:793][55:140][512:560],LST_Night_1km[775:793][55:140][512:560],QC_Day[775:793][55:140][512:560],QC_Night[775:793][55:140][512:560],time[775:793],YDim[55:140],XDim[512:560]
is a link to download the following subset of MOD11A2.061 data in netCDF :
- bands LST_Day_1km, LST_Night_1km, QC_Day, QC_Night ;
- each available date between the 2017-01-01 and the 2017-06-01 ;
- within the following bounding box (lon/lat): -5.41 8.84, -5.82 9.54.
The indices within the [] refer to values encoding for the spatial and
temporal filters.
These OPeNDAP URLs are not trivial to build. modisfast converts the
spatial, temporal and dimensional filters (R objects) provided by the
user through the function mf_get_url() into the appropriate OPeNDAP
URL(s). Subsequently, the function mf_download_data() allows for
downloading the data using the
httr and parallel
packages.
Comparison with similar R packages
There are other R packages available for accessing MODIS data. Below is a comparison of modisfast with other packages available for downloading chunks of MODIS or VIIRS data :
| Package | Data | Available on CRAN | Utilizes open standards for data access protocols | Spatial subsetting* | Dimensional subsetting* | Maximum area size allowed for download | Speed** |
|:-------------------------------------------------------------:|:----------------------------------:|:------------------:|:-------------------------------------------------:|:--------------------:|:------------------------:|:--------------------------------------:|:------------------:|
| modisfast | MODIS, VIIRS, GPM | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | unlimited | :white_check_mark: |
| appeears | MODIS, VIIRS, and many others | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | unlimited | variable |
| MODISTools | MODIS, VIIRS | :white_check_mark: | :x: | :white_check_mark: | :white_check_mark: | 200 km x 200 km | :white_check_mark: |
| rgee | MODIS, VIIRS, GPM, and many others | :white_check_mark: | :x: | :white_check_mark: | :white_check_mark: | unlimited | not tested |
| MODIStsp | MODIS | :x: | | :x: | :white_check_mark: | unlimited | NA |
| MODIS | MODIS | :x: | :x: | :x: | :x: | NA | NA |
* at the downloading phase
** Take a look at the article “Comparison of performance with other
similar R
packages”
to get an overview of how modisfast compares to these packages in
terms of data access time.
Citation
This package is licensed under a GNU General Public License v3.0 or later license.
We thank in advance people that use modisfast for citing it in their
work / publication(s). For this, please use the following citation :
Taconet et al., (2024). modisfast: An R package for fast and efficient access to MODIS, VIIRS and GPM Earth Observation data. Journal of Open Source Software, 9(103), 7343, https://doi.org/10.21105/joss.07343
Future developments
Future developments of the package may include access to additional data collections from other OPeNDAP servers, and support for a variety of data formats as they become available from data providers through their OPeNDAP servers. Furthermore, the creation of an RShiny application on top of the package is being considered, as a means of further simplifying data access for users with limited coding skills.
Contributing
All types of contributions are encouraged and valued. For more information, check out our Contributor Guidelines.
Please note that the modisfast project is released with a Contributor
Code of
Conduct.
By contributing to this project, you agree to abide by its terms.
Acknowledgments
We thank NASA and its partners for making all their Earth science data
freely available, and implementing open data access protocols such as
OPeNDAP. modisfast heavily builds on top of the OPeNDAP, so we thank
the non-profit OPeNDAP, Inc. for
developing the eponym tool in an open and collaborative way.
We also thank the contributors that have tested the package, reviewed the documentation and brought valuable feedbacks to improve the package : Florian de Boissieu, Julien Taconet.
This work has been developed over the course of several research projects (REACT 1, REACT 2, ANORHYTHM and DIV-YOO) funded by Expertise France, the French National Research Agency (ANR), and the French National Research Institute for Sustainable Development (IRD).
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.