inst/doc/upstream_api.md

The motusClient R package upstream API

The motusClient R package maintains your copy of a tag detection database. The database is built from data provided by a server, typically at motus.org This document describes the API calls required by the motusClient package; i.e. what requests must a server respond to if it is to work with this package.

Quick Links to API Entries by Topic

Changelog

API summary

Request

Reply

Examples are given for each call using the command-line client curl with quoting appropriate for the Bash shell. These examples return the raw bzip2-compressed data. To view the response, redirect the output of curl into a file and use 7zip to decompress it (for example), or add | bunzip2 -cd to the end of the command in Bash.

The server is at https://sgdata.motus.org and the URL prefix is "/data/custom/".

API calls

api info

api_info ()

authenticate user

authenticate_user (user, password)

  - user: username
  - password: password (in cleartext)

  e.g.
  curl --data-urlencode json='{"user":"someone","password":"bigsecret"}' https://sgdata.motus.org/data/custom/authenticate_user

or

Notes

  1. The authToken returned by this API must be included in most other API calls.

  2. Authorization is by project: if a user has permission for a project, then that user can see:

  3. all detections from receivers deployed by that project

  4. all detections of tags deployed by that project

If an API call does not find any data for which the user is authorized, it will return a json object of the usual structure, except that arrays will have length zero. This represents an R data.frame with the correct column names but zero rows.

The API doesn't currently provide a way to tell whether there are additional data which would be returned for a given call if the user had authorization for more projects.

"Ownership" of detections follows these assumptions:

These assumptions allow for simpler, more efficient database queries.

deviceID for receiver

deviceID_for_receiver (serno, authToken)

   - serno: array of string; receiver serial number(s)
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"serno":"SG-1234BBBK5678","authToken":"XXX"}' https://sgdata.motus.org/data/custom/deviceID_for_receiver

receivers for project

receivers_for_project (projectID, authToken)

   - projectID: integer; project ID
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/receivers_for_project

batches for tag project

batches_for_tag_project (projectID, batchID, includeTesting, authToken)

   - projectID: integer; project ID
   - batchID: integer; largest batchID we already have for this project
   - includeTesting: (optional) boolean; default: false.  If true, include those batches marked
     `testing` on the server.  This parameter is ignored for non-administrator users.
     **Do not use** this parameter unless you are prepared to correct your database to
     remove such batches!
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":0, "authToken":"XXX"}' https://sgdata.motus.org/data/custom/batches_for_tag_project

Paging for this query is achieved by using the largest returned value of batchID as batchID on subsequent calls. When there are no further batches, the API returns an empty list.

batches for receiver

batches_for_receiver (deviceID, batchID, includeTesting, authToken)

   - deviceID: integer; motus device ID, e.g. as returned by receivers_for_project
   - batchID: integer; largest batchID we already have for this project
   - includeTesting: (optional) boolean; default: false.  If true, include those batches marked
     `testing` on the server.  This parameter is ignored for non-administrator users.
     **Do not use** this parameter unless you are prepared to correct your database to
     remove such batches!
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":0, "authToken":"XXX"}' https://sgdata.motus.org/data/custom/batches_for_receiver

Paging for this query is achieved by using the largest returned value of batchID as batchID on subsequent calls. When there are no further batches, the API returns an empty list.

batches for all

batches_for_all (batchID, includeTesting, authToken) - administrative users only

   - batchID: integer; largest batchID we already have
   - includeTesting: (optional) boolean; default: false.  If true, include those batches marked
     `testing` on the server.  This parameter is ignored for non-administrator users.
     **Do not use** this parameter unless you are prepared to correct your database to
     remove such batches!
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"batchID":0, "authToken":"XXX"}' https://sgdata.motus.org/data/custom/batches_for_all

Paging for this query is achieved by using the largest returned value of batchID as batchID on subsequent calls. When there are no further batches, the API returns an empty list.

This call is intended only for users who are building a database of all detections. Currently, that means only administrative users.

runs for tag project

runs_for_tag_project (projectID, batchID, runID, authToken)

   - projectID: integer; project ID
   - batchID: integer; batch ID
   - runID: double; largest run ID we *already* have from this batch and tag project
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":111,"runID":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/runs_for_tag_project

Paging for this query is achieved by using the last returned value of runID as runID on subsequent calls. When there are no further runs, the API returns an empty list.

If a run's done field is zero, then later processing of data from the same receiver by the server might extend the run. In that case the same run might be returned by later calls to the same API, but with a different batchID; and the run's tsEnd, len, or done fields could change if the run were extended or known to have ended.

runs for receiver

runs_for_receiver (batchID, runID, authToken)

   - batchID: integer; batch ID
   - runID: double; largest runID we *already* have from this batch
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":111,"runID":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/runs_for_receiver

Paging for this query is achieved by using the last returned value of runID as runID on subsequent calls. When there are no further runs, the API returns an empty list.

For regular users, this only returns runs if the user has permission for the project which owns the receiver deployment covering this batch.

For admin users, all runs are returned, regardless of batch ownership (or lack thereof).

If a run's done field is zero, then later processing of data from the same receiver by the server might extend the run. In that case the same run might be returned by later calls to the same API, but with a different batchID; and the run's tsEnd, len, or done fields could change if the run were extended or known to have ended.

hits for tag project

hits_for_tag_project (projectID, batchID, hitID, authToken)

   - projectID: integer; project ID
   - batchID: integer; batchID
   - hitID: double; largest hitID we *already* have from this batch
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":111,"hitID":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/hits_for_tag_project

Paging for this query is achieved by using the last returned value of hitID as hitID on subsequent calls. When there are no further hits, the API returns an empty list.

hits for receiver

hits_for_receiver (batchID, hitID, authToken)

   - batchID: integer; batchID
   - hitID: double; largest hitID we *already* have from this batch
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"batchID":111,"hitID":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/hits_for_receiver

Paging for this query is achieved by using the last returned value of hitID as hitID on subsequent calls. When there are no further hits, the API returns an empty list.

For regular users, this only returns hits if the user has permission for the project which owns the receiver deployment covering this batch.

For admin users, all hits are returned, regardless of batch ownership (or lack thereof).

gps for tag project

gps_for_tag_project (projectID, batchID, ts, authToken)

   - projectID: integer; project ID of tags
   - batchID: integer; batchID where tags from projectID were detected
   - ts: largest gps timestamp we *already* have for this batch
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":111,"ts":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/gps_for_tag_project

Paging for this query is achieved by using the last returned value of ts as ts on subsequent calls. When there are no further GPS fixes, the API returns an empty list.

gps for receiver

gps_for_receiver (batchID, ts, authToken)

   - batchID: integer; batchID
   - ts: largest gps timestamp we *already* have for this batch
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"batchID":111,"ts":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/gps_for_receiver

Paging for this query is achieved by using the last returned value of ts as ts on subsequent calls. When there are no further GPS fixes, the API returns an empty list.

For regular users, this only returns gps records if the user has permission for the project which owns the receiver deployment covering this batch.

For admin users, all gps records are returned, regardless of batch ownership (or lack thereof).

metadata for tags

metadata for tags (motusTagIDs, authToken)

   - motusTagIDs: integer array; motus tag IDs; tag metadata will
     only be returned for tag deployments whose project has indicated
     their metadata are public, or tags deployments by one of the
     projects the user has permissions to.
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"motusTagIDs":[12345,12346,12347],"authToken":"XXX"}' https://sgdata.motus.org/data/custom/metadata_for_tags

metadata for receivers

metadata for receivers (deviceIDs, authToken)

   - deviceID: integer; device ID; receiver metadata will only be
     returned for receivers whose project has indicated their
     metadata are public, or receivers in one of the projects the
     user has permissions to.
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"deviceIDs":[123,124,125],"authToken":"XXX"}' https://sgdata.motus.org/data/custom/metadata_for_receivers

tags for ambiguities

tags for ambiguities (ambigIDs, authToken)

   - ambigIDs: integer; tag ambiguity IDs; this is an array of negative
     integers, each representing 2 to 6 tags for which detections are
     indistinguishable over some period of time; i.e. a detection of
     the given ambigID could represent any of the motus tagIDs.  (6 is
     an implementation limit, not a conceptual one.)
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"ambigIDs":[-3,-4,-5],"authToken":"XXX"}' https://sgdata.motus.org/data/custom/tags_for_ambiguities

size of update for tag project

size_of_update_for_tag_project (projectID, batchID, authToken)

   - projectID: integer; project ID
   - batchID: integer; ID of largest batch client already has
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"projectID":123,"batchID":15538,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/size_of_update_for_tag_project

size of update for receiver

size_of_update_for_receiver (deviceID, batchID, authToken)

   - deviceID: integer; motus device ID
   - batchID: integer; ID of largest batch client already has
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"deviceID":221,"batchID":15538,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/size_of_update_for_receiver

For regular users, this only counts items where the user has permission for the project which owns the receiver deployment covering the batch.

For admin users, all items are counted, regardless of batch ownership (or lack thereof).

project ambiguities for tag project

project_ambiguities_for_tag_project (projectID)

   - projectID: integer; projectID

  e.g.
  curl --data-urlencode json='{"projectID":123,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/project_ambiguities_for_tag_project

pulse counts for receiver

pulse_counts_for_receiver (batchID, ant, hourBin, authToken)

   - batchID: integer; batchID
   - ant: integer; antenna number
   - hourBin: double; the hourBin is defined as floor(timestamp / 3600), where timestamp is the usual
     "seconds since 1 Jan 1970 GMT" unix timestamp.
   - authToken: authorization token returned by authenticate_user

  e.g.
  curl --data-urlencode json='{"batchID":111,"ant": 0,"hourBin":0,"authToken":"XXX"}' https://sgdata.motus.org/data/custom/pulse_counts_for_receiver

Paging for this query is achieved by using the last returned values of ant and hourBin as ant and hourBin on subsequent calls. When there are no further pulse counts, the API returns an empty list.

Note that this API returns pulses sorted by hourBin within antenna for each batch.

For regular users, this only returns pulse counts if the user has permission for the project which owns the receiver deployment covering this batch.

For admin users, all pulse counts are returned, regardless of batch ownership (or lack thereof).

Changelog

2018-02-16: change types of runID and hitID to double to accomodate more than 2 billion of each. We are already at 30% of 2 billion for hits, and given the reprocessing contract stipulates we never re-use hitID, an integer value doesn't afford much room for growth.

2017-11-30: added the optional includeTesting boolean parameter to batch_for_* entries. This parameter defaults to false. If true and the user is an administrator, then records for batches marked as testing on the server are returned as if they were normal batches. Otherwise, testing batches are not returned. This is intended to help debug server-side code by allowing batches of data to be processed and their results made available "under the radar". This parameter should never be set to true when requesting data for end users: any data associated with testing batches can be deleted from the sgdata.motus.org server at any time, and should be considered invalid.

It is guaranteed that batchIDs, runIDs, and hitIDs from testing batches will not be re-used for production batches.

This change does not require any change on the client side unless the client wishes to obtain testing batches.



jbrzusto/motusClient documentation built on May 30, 2019, 4:33 p.m.