motus.org

Copse

R Documentation

Create a copse, a persistent structure for holding R objects with a tree structure.

Description

This function is a factory for an S3 class ("Copse") that uses an SQLite database to store R objects of class ("Twig") which are related as one or more trees. Each Twig holds a named set of plain-old-data R objects stored in a JSON field, and zero or more user-specified fixed fields, stored as SQLite columns. Twig get/set semantics use $ and $[[]]; i.e. they work like environments. Within an R process, Twigs are copied by reference so that there's really only one version of any Twig. Changes to Twigs are recorded atomically to the SQLite database, so that other processes also have access to them. An atomic get/modify/set operation can be performed on a Twig by using the with() method.

Usage

Copse(db, table, ...)

Arguments

db

path to sqlite database with the copse table. This will be created if it doesn't exist.

table

name of sqlite table in the database. This will be created if it doesn't exist in db, with this schema:

   CREATE TABLE <table> (
      id    INTEGER UNIQUE PRIMARY KEY,
      pid   INTEGER REFERENCES <table> (id), -- ID of parent twig, if any
      stump INTEGER REFERENCES <table> (id), -- ID of ultimate ancestor, if any
      ctime FLOAT(53),                       -- twig creation time, unix timestamp
      mtime FLOAT(53),                       -- twig modification time, unix timestamp
      ...
      data  JSON                             -- JSON-serialized object data
   )

where ... represents additional user-specified columns present in all twigs.

The ... columns can be considered the structurally constant part of a Copse, while the data column holds the structurally variable part.

Details

The Copse implements concept of "job" for this package. Job state is stored persistently, and a job should be resumable if the server dies due to power outage, bugs, or fixable errors in the data submitted as a job.

The JSON fields are not indexed, so for fast lookup of simple datatypes, the latter should be provided in the ... at Copse creation time. The JSON fields allow structured datatypes, and allow new records to include data fields not specified when the Copse was created. So if new types of Jobs are implemented as the motusServer package evolves, we don't need to modify the existing jobs database.

The copse manages these functions:

create the database table
get Twigs from the DB
put Twigs to the DB when they change in R
record timestamps for Twig creation and modification

Value

This function creates an object of class "Copse". It has these S3 methods:

newTwig(Copse, ..., .parent=NULL): new twig with the named items in (...), and with parent Twig .parent
Copse[[TwigID]]: twig(s) with given ID(s) or NULL
Copse[query, sort]: twig(s) satisfying query, in order by sort criterion
child(Copse, Twig, n): nth child of given twig, or NULL
children(Copse, Twig): Twigs which are children of Twig
childrenWhere(Copse, Twig, expr): Twigs which are children of Twig and satisfy the expression
numChildren(Copse, Twig): number of children of Twig
parent(Copse, Twig): all Twigs which are parents to given Twig(s)
stump(Copse, Twig): Twigs which are ultimate ancestors to given Twig(s)
query(Copse, query): run an arbitrary sql query on the Copse
setParent(Copse, Twig1, Twig2): set parent of Twig1 to be Twig2
with(Copse, expr): evaluate expr within a transaction on the Copse database; this provides atomic get-modify-set semantics across processes accessing the same Copse. If an error occurs while evaluating expr, the transaction is rolled back.

Internally a Copse uses these symbols:

sql: safeSQL connection object to DB
db: path to db
table: name of table in DB

Twigs are S3 objects of class "Twig" with these S3 methods:

$(Twig, name): return value of name in Twig, with name an unquoted symbol
$<-(Twig, name, value): set value of name in Twig, with name an unquoted symbol
[(Twig, index): return a subset of Twigs, allowing the usual ways of indexing an integer vector
[[(Twig, name): return value of name in Twig, with name a quoted character scalar
[[<-(Twig, name, value): set value of name in Twig, with name a quoted character scalar
names(Twig): list names in Twig
as.list(Twig): list of named items in Twig and their values
parent(Twig): get parent Twig of Twig, or NULL if it has none
stump(Twig): ultimate ancestor of Twig, or NULL if none
parent<-(Twig, Twig): set parent of Twig to Twig with ID TwigID (can pass a Twig instead):
child(Twig, n): get nth child of Twig, or NULL if it doesn't exist
children(Twig): get list of IDs of children of Twig
progeny(Twig): Twigs which are progeny of Twig
childrenWhere(Twig, expr): list of child Twigs for which given expr is TRUE. The expression is applied against the data field of each row. The identifier "." stands for the item's top level, so the third element of a numeric vector called 'blam' would be represented as '.$blam[3]' in expr
numChildren(Twig): get number of children of Twig
copse(Twig): get Copse object that owns Twig
mtime(Twig): twig creation time, as unix timestamp
ctime(Twig): twig modification time, as unix timestamp
blob(Twig): twig data JSON-serialized
setData(Twig, names, values, clearOld=FALSE): set named data items for twig; if clearOld is TRUE, delete all existing data first. As a shortcut, if values is missing, treat names as a named list, rather than a char vector of names. Uses a single DB query to set all items.
delete(Twig): called only from garbage collection; reduces the use count of the real twig, dropping it from its Copse's map when the count reaches zero

Internally, a Twig is a numeric vector of class "Twig" with attribute Copse being the Twig's Copse.

Author(s)

John Brzustowski jbrzusto@REMOVE_THIS_PART_fastmail.fm

Examples


hats = Copse("/home/john/inventory.sqlite", "hats")
b = newTwig(hats, name="bowler", size=22, colour="black")
h = hats[id < 10 || .$size > 20]  ## query can involve id, pid, mtime, ctime, fixed columns, or data variables selected using .$...$...[]...
h[[1]]

MotusWTS/motusServer documentation built on Aug. 8, 2024, 10:23 p.m.