frost_removeDuplicates: remove duplicate records from Frost API data
In metno/gibson: get in-situ observations

frost_removeDuplicates

R Documentation

remove duplicate records from Frost API data

Description

Remove duplicates records from the output of gibson_frost, that is the KDVH data retrieved through the Frost API frost.met.no.

Duplicates are defined as one or more records refererring to the same sourceId(station:sensor), observation_timestamp and having the same observed value and quality flag value. This situation might occur because the same data are retrieved by Frost from different KDVH tables.

Usage

frost_removeDuplicates(oldElementCode=NULL,
                       frost_data=NULL)

Arguments

oldElementCodes abbreviations are listed at https://frost.met.no/elementtable. Not all abbreviations have been implemented (see frost_translate_oldElementCodes).

frost_data

the data frame frost_data returned by gibson_frost

Details

Observations with time resolution smaller than 1 day does not have duplicates.

Diurnal data might have duplicates.

Value

A numeric vector with the position of the NON-duplicates in frost_data.

Author(s)

Cristian Lussana

Examples


# load libraries
library(gibson)
# get the data
frost<-gibson_frost(client_id=auth,oldElementCodes=c("RR"), sources="ALL",start_date="2018-03-02T08:00",stop_date="2018-03-04T13:00",format="%Y-%m-%dT%H:%M",countries="NO",stationholders="MET.NO",url.show=T)

res<-frost_removeDuplicates(oldElementCode="RR",frost_data=frost$frost_data)

#> length(res)
#[1] 687
#> length(frost$frost_data$sourceId)
#[1] 1297

# create a data frame without duplicates
frost_data_noDuplicates<-data.frame(frost$frost_data$elementId[ixNoDup],frost$frost_data$sourceId[ixNoDup],frost$frost_data$referenceTime[ixNoDup],frost$frost_data$value[ixNoDup],frost$frost_data$qcode[ixNoDup],frost$frost_data$timeOffset[ixNoDup],frost$frost_data$timeResolution[ixNoDup],frost$frost_data$level[ixNoDup],frost$frost_data$levelType[ixNoDup],frost$frost_data$oldElementCodes[ixNoDup],stringsAsFactors=F)
names(frost_data_noDuplicates)<-c("elementId","sourceId","referenceTime","value","qcode","timeOffset","timeResolution","level","levelType","oldElementCodes")

# write files
write.table(frost$frost_data,file="data_original")
write.table(frost_data_noDuplicates,file="data_noDuplicates")

metno/gibson documentation built on Feb. 12, 2024, 7:25 a.m.