Hydro-Data-Source-Data-Source-Bin: RomDataSource

In HARPgroup/hydro-tools: Data processing and accessors for VAHydro

RomDataSource

Description

RomDataSource is an object that contains tables and methods of model data objects. It serves as a global object store that references to all objects of a given domain and pointers to remote data source if applicable. It offers the flexibility of synching (via POST or INSERT/UPDATE) to a remote data source OR allows users to update local data bases or sources. It does not permit a mix of these data streams, keeping the RomDataSource object consistent.

Details

Provides integrated, queryable universe of features, observations and meta data

Value

reference class of type openmi.om.base.

Public fields

site

URL of some RESTful repository

json_obj_url

URL for retrieving full objects

connection_type

rest or odbc

connection

rest or odbc

rest_uname

username to connect to RESTful repository

dbname

DATABASE TO USE IN odbC CONNECTION

debug

show debug info for queries

timeline

for default time series data

tsvalues

table of time series data

propvalues

table of object properties (can be contained by objects)

features

table of physical features

var_defs

table of variable definitions

admin_features

table of adminreg features

prop_json_cache

list of json objects retrieved, keyed by ID

ts_cache

list of ts objects instantiated

feature_cache

list of feature objects instantiated

prop_cache

list of prop objects instantiated

var_cache

list of var objects instantiated

Methods

Public methods

• RomDataSource$new()

• RomDataSource$reset()

• RomDataSource$get_token()

• RomDataSource$get_vardef()

• RomDataSource$get_prop()

• RomDataSource$get_ts()

• RomDataSource$auth_read()

• RomDataSource$set_ts()

• RomDataSource$set_prop()

• RomDataSource$insure_cols()

• RomDataSource$set_vardef()

• RomDataSource$store_features()

• RomDataSource$set_feature()

• RomDataSource$get()

• RomDataSource$post()

• RomDataSource$delete()

• RomDataSource$get_json_prop()

• RomDataSource$get_nested_export()

• RomDataSource$clone()

---

Method new()

Initialize a RomDataSource, returning a RomDataSource R6 object that is now populated with site, username, connection type, database name, and any connection string details if using ODBC. This object will also have numerous methods that are described in ?RomDataSource

Usage

RomDataSource$new(
  site,
  rest_uname = NULL,
  connection_type = "rest",
  dbname = NULL
)

Arguments

site

URL of some RESTful repository or the host of the target database. At DEQ, this is defined in the default config files.

rest_uname

username to connect to RESTful repository or database. At DEQ, this should be defined within local config files

connection_type

String, either 'rest' or 'odbc' depending on target database

dbname

Used only when connection_type = 'odbc'. This is the database name of the database that supports the ODBC connection. At DEQ, this should generally target the active dbase, not alpha, although both are supported

Returns

Instance of RomDataSource, now with populated site, rest_uname, connection_type, and dbname data

---

Method reset()

Clear entries and reset tables stored in RomDataSource. Used to clean out features, properties, tsvalues, or variable definitions

Usage

RomDataSource$reset(table)

Arguments

table

Which table(s) should be cleared? . Defaults to 'all', but will also accept a vector that contains any or all of: c('props', 'features', 'tsvalues', 'var_defs')

Returns

nothing, but will clear requested data tables stored on this object #'@description Set the connection to either the database via ODBC or to a REST service (through the act of obtaining a token). OFten done in DEQ config.R files.

---

Method get_token()

Usage

RomDataSource$get_token(rest_pw = NULL, odbc_port = 5431)

Arguments

rest_pw

Password to REST/odbc service requested. Will prompt user for entry if not provided

odbc_port

If using odbc, which port should be used? Will prompt user for entry if not provided. Defaults to DEQ port approved by OIS

Returns

Nothing, but sets internal private token. Private fields are not visible via RomDataSource$token notation and are hidden for security

---

Method get_vardef()

Queries first local variable definitions and if necessary dh_variabledefinition for a varkey, setting the varid within RomDataSource instance

Usage

RomDataSource$get_vardef(varkey, force_refresh = FALSE, debug = FALSE)

Arguments

varkey

Variable key as defined in dh_variabledefinition as varkey. See Hydrotools readme for more information, but represents an abbreviated variable name to define a property or timeseries.

force_refresh

Should the remote database be checked for new info?

debug

Show relevant debugging info

Returns

Nothing, but sets variable definitions on RomDataSource Queries first local properties and if needed or requested dh_property for the properties specified in the user config file. This method creates a WHERE clause from the config file for an SQL query of the user input for entity_type provided by user. Will only return the first found property.

---

Method get_prop()

Usage

RomDataSource$get_prop(
  config,
  return_type = "data.frame",
  force_refresh = FALSE,
  obj = FALSE
)

Arguments

config

A list of values that includes query constructors. Each entry in this list and its corresponding value will be added to the WHERE clause of a query that queries the table specified in entity_type with the exception of "limit", which will only be used to generate the LIMIT clause for the query. May include a primary key ID, in which case it will be all that is used in the WHERE clause. Relevant data for list may include propname, propvalue, hydrocode, or other fields shown in the Hydrotools Readme e.g. list(entity_type, featureid, pid, varid, tstime, tsendtime). See Readme for more options

return_type

'data.frame' or 'list'

force_refresh

Should the remote data source be queried (TRUE) or just local properties stored on the RomDataSource instance

obj

(optional) object class calling this routine, can supply a base query via sql_select_from field as would otherwise be returned via hydrotools:::fn_guess_sql()

Returns

Data frame of the first property returned from query of properties based on data provided in config

---

Method get_ts()

Usage

RomDataSource$get_ts(
  config,
  return_type = "data.frame",
  force_refresh = FALSE,
  obj = FALSE
)

Arguments

config

A list of values that includes query constructors. Each entry in this list and its corresponding value will be added to the WHERE clause of a query that queries the table specified in entity_type with the exception of "limit", which will only be used to generate the LIMIT clause for the query. May include a primary key ID, in which case it will be all that is used in the WHERE clause. Relevant data for list may include propname, propvalue, hydrocode, or other fields shown in the Hydrotools Readme e.g. list(entity_type, featureid, tid = NULL, varid = NULL, tstime = NULL, tsendtime = NULL, tscode = NULL, tlid = NULL) timeline ID (not yet used). See Readme for more options

return_type

'data.frame' or 'list', but will default to data.frame

force_refresh

Should the remote data source be queried (TRUE) or just local properties stored on the RomDataSource instance

obj

(optional) object class calling this routine, can supply a base query via sql_select_from field as would otherwise be returned via hydrotools:::fn_guess_sql()

Returns

Data frame of the first TS returned from query of dh_timeseries based on data provided in config

---

Method auth_read()

Used to read views from a REST service.

Usage

RomDataSource$auth_read(
  uri,
  content_type = "text/csv",
  delim = ",",
  enc = "xml"
)

Arguments

uri

remote address to retrieve data

content_type

http content-type

delim

delimiter

enc

encoding

Returns

result of web request

---

Method set_ts()

Usage

RomDataSource$set_ts(ts)

Arguments

ts

= list(entity_type, featureid, tid = NULL, varid = NULL, tstime = NULL, tsendtime = NULL, tscode = NULL, tlid = NULL) timeline ID (not yet used)

Returns

nothing sets internal private token

---

Method set_prop()

Usage

RomDataSource$set_prop(prop)

Arguments

prop

= list(entity_type, featureid, pid = NULL, varid = NULL, tstime = NULL, tsendtime = NULL, tscode = NULL, tlid = NULL) timeline ID (not yet used)

Returns

nothing seprop internal private token

---

Method insure_cols()

Usage

RomDataSource$insure_cols(src_df, dest_df)

Arguments

src_df

= df to verify/insure

dest_df

= df template to supply valid names

Returns

local df index?

---

Method set_vardef()

Usage

RomDataSource$set_vardef(vardef)

Arguments

vardef

= list(varid, varkey, varname, varunits, varcode,...)

Returns

local df index?

---

Method store_features()

Usage

RomDataSource$store_features(features)

Arguments

features

= list(entity_type, featureid, pid = NULL, varid = NULL, tstime = NULL, tsendtime = NULL, tscode = NULL, tlid = NULL) timeline ID (not yet used)

Returns

nothing seprop internal private token

---

Method set_feature()

Usage

RomDataSource$set_feature(feature)

Arguments

feature

= list(entity_type, featureid, pid = NULL, varid = NULL, tstime = NULL, tsendtime = NULL, tscode = NULL, tlid = NULL) timeline ID (not yet used)

Returns

nothing seprop internal private token

---

Method get()

Usage

RomDataSource$get(entity_type, pk, config, obj = FALSE)

Arguments

entity_type

Most often dh_feature or dh_properties. Indicates which table to retrieve data from. This will be the target table for the query constructed from user inputs input list

pk

Primary key column name, often hydroid or pid. See Readme for additional information or contact OWSPA Data Coordinatory if you do not know the appropriate primary key for the table of interest. If referenced in config, will be the only item in the WHERE clause of the user query

config

A list of potential values to be added to an SQL query that targets entity_type. This list will be iterated through and added to a WHERE clause to ensure user query operates. If a primary key id is provided named the same as the user input pk, the WHERE clause will only contain a statement on this key id. "limit" may also be in this list to limit query results

obj

(optional) object class calling this routine, can supply a base query via sql_select_from field as would otherwise be returned via hydrotools:::fn_guess_sql()

Returns

Results of the query of entity_type constructed from the config supplied by user. Typically a data frame.

---

Method post()

Usage

RomDataSource$post(entity_type, pk, config)

Arguments

entity_type

= dh_feature, dh_properties, ...

pk

= primary key column name, e.g. hydroid, pid, ...

config

= contents of record to post in list(pid, propname, propvalue, ...)

Returns

local df index?

---

Method delete()

Usage

RomDataSource$delete(entity_type, pk, config, obj = FALSE)

Arguments

entity_type

= dh_feature, dh_properties, ...

pk

= primary key column name, e.g. hydroid, pid, ...

config

= contents of record to post in list(pid, propname, propvalue, ...)

obj

= (optional) object class calling this routine, can supply extra info

Returns

local df index?

---

Method get_json_prop()

Usage

RomDataSource$get_json_prop(pid)

Arguments

pid

= object pid

Returns

unserialized json as list, with object stored in ds$prop_json_cache

---

Method get_nested_export()

Usage

RomDataSource$get_nested_export(ds, featureid, props, depth = 0)

Arguments

ds

= satasource object, kept for posterity, as this may not always live here

featureid

= object pid

props

= container for stashing

depth

= depth limit for nesting (rarely used)

Returns

unserialized json as list, with object stored in ds$prop_json_cache

---

Method clone()

The objects of this class are cloneable with this method.

Usage

RomDataSource$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

See Also

NA

Examples

##Get new datasource via odbc
#ds <- RomDataSource$new(site,
#                         rest_uname = odbc_uname,
#                         connection_type = 'odbc',
#                         dbname = databaseName)
# ds$get_token(rest_pw = odbc_pw)
##Pointer to external db
# ds$connection

HARPgroup/hydro-tools documentation built on July 4, 2025, 11:05 a.m.