neuronlistfh objects consist of a list of neuron objects
along with an optional attached dataframe containing information about the
neurons. In contrast to
neuronlist objects the neurons are not
present in memory but are instead dynamically loaded from disk as required.
neuronlistfh objects also inherit from
therefore any appropriate methods e.g.
plot3d.neuronlist can also be
neuronlistfh constructs a neuronlistfh object from a
keyfilemap. End users will
not typically use this function to make a
They will usually read them using
read.neuronlistfh and sometimes
create them by using
as.neuronlistfh on a
is.neuronlistfh test if an object is a neuronlistfh
as.neuronlistfh generic function to convert an object to
as.neuronlistfh.neuronlist converts a regular neuronlist
to one backed by a filehash object with an on disk representation
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Optional dataframe, where each row describes one neuron
A named character vector in which the elements are filenames on disk (managed by the filehash object) and the names are the keys used in R to refer to the neuron objects. Note that the keyfilemap defines the order of objects in the neuronlist and will be used to reorder the dataframe if necessary.
A logical indicating whether to add a hashed environment for rapid object lookup by name or an integer or an integer definining a threhsold number of objects when this will happen (see Implementation details).
Object to test
Object to convert
Additional arguments for methods, eventually passed to
The path to the underlying
The url pointing to a remote repository containing files for each neuron.
Whether to write objects to disk. Missing implies that
existing objects will not be overwritten. Default
neuronlistfh object which is a character
neuronlistfh, neuronlist and attributes
db, df. See
neuronlistfh objects are a hybrid between
neuronlist objects that organise data and metadata for
collections of neurons and a backing
filehash object. Instead of
keeping objects in memory, they are always loaded from disk.
Although this sounds like it might be slow, for nearly all practical
purposes (e.g. plotting neurons) the time to read the neuron from disk is
small compared with the time to plot the neuron; the OS will cache repeated
reads of the same file. The benefits in memory and startup time (<1s vs
100s for our 16,000 neuron database) are vital for collections of 1000s of
neurons e.g. for dynamic report generation using knitr or for users with
<8Gb RAM or running 32 bit R.
neuronlistfh objects include:
attr("keyfilemap") A named character vector that determines the
ordering of objects in the neuronlist and translates keys in R to filenames
on disk. For objects created by
as.neuronlistfh the filenames will
be the md5 hash of the object as calculated using
design means that the same key can be used to refer to multiple distinct
objects on disk. Objects are effecitvely versioned by their contents. So if
an updated neuronlistfh object is posted to a website and then fetched by a
user it will result in the automated download of any updated objects to
which it refers.
attr("db") The backing database - typically of class
filehashRDS. This manages the loading of objects from disk.
attr(x,"df") The data.frame of metadata which can be used to select
and plot neurons. See
neuronlist for examples.
attr(x,"hashmap") (Optional) a hashed environment which can be used for rapid lookup using key names (rather than numeric/logical indices). There is a space potential to pay for this redundant lookup method, but it is normally worth while given that the dataframe object is typically considerably larger. To give some numbers, the additional environment might occupy ~ 1 time from 0.5 ms to 1us. Having located the object, on my machine it can take as little as 0.1ms to load from disk, so these savings are relevant.
Presently only backing objects which extend the
filehash class are
supported (although in theory other backing objects could be added). These
filehash RDS2 (experimental)
We have also implemented a simple remote access protocol (currently only
RDS format). This allows a neuronlistfh object to be read
from a url and downloaded to a local path. Subsequent attempts to access
neurons stored in this list will result in automated download of the
requested neuron to the local cache.
An alternative backend, the experimental
RDS2 format is supported
(available at https://github.com/jefferis/filehash). This is likely
to be the most effective for large (5,000-500,000) collections of neurons,
especially when using network filesystems (nfs, afp) which are typically
very slow at listing large directories.
Note that objects are stored in a filehash, which by definition does not
have any ordering of its elements. However neuronlist objects (like lists)
do have an ordering. Therefore the names of a neuronlistfh object are not
necessarily the same as the result of calling
names() on the
underlying filehash object.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
## Not run: kcnl=read.neuronlistfh('http://jefferislab.org/si/nblast/flycircuit/kcs20.rds', 'path/to/my/project/folder') # this will automatically download the neurons from the web the first time # it is run plot3d(kcnl) ## End(Not run) ## Not run: # create neuronlistfh object backed by filehash with one file per neuron # by convention we create a subfolder called data in which the objects live kcs20fh=as.neuronlistfh(kcs20, dbdir='/path/to/my/kcdb/data') plot3d(subset(kcs20fh,type=='gamma')) # ... and, again by convention, save the neuronlisfh object next to filehash # backing database write.neuronlistfh(kcs20fh, file='/path/to/my/kcdb/kcdb.rds') # in a new session read.neuronlistfh("/path/to/my/kcdb/kcdb.rds") plot3d(subset(kcs20fh, type=='gamma')) ## End(Not run)
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.