library(formatR) knitr::opts_chunk$set(echo = FALSE)
The current implementation for the
@sensitivity slot in a
PharmacoSet has some
Firstly, it does not natively support dose-response experiments with multiple drugs and/or cancer cell lines. As a result we have not been able to include this data into a PharmacoSet thus far.
Secondly, drug combination data has the potential to scale to high dimensionality. As a result we need an object that is highly performant to ensure computations on such data can be completed in a timely manner.
The current use case is supporting drug and cell-line combinations in
PharmacoGx, but we wanted to create something flexible enough to fit
other use cases. As such, the current class makes no mention of drugs or cell-lines,
nor anything specifically related to Bioinformatics or Computation Biology. Rather, we tried to design a general purpose data structure which could support
high dimensional data for any use case.
Our design takes the best aspects
MultiAssayExperiment class and implements
them using the
data.table package, which provides an R API to a rich set of
tools for high performance data processing implemented in C.
We have borrowed directly from the
assays slot names.
We also implemented the
SummarizedExperiment accessor generics for the
There are, however, some important differences which make this object more flexible when dealing with high dimensional data.
SummarizedExperiment, there are three distinct
classes of columns in
The first is the
colKey, these are implemented internally to keep mappings between each assay and the associated samples or drugs; these will not be returned by the accessors by default. The second is the
colIDs, these hold all of the
information necessary to uniquely identify a row or column and are used to
colKey. Finally, there are the
columns, which store any additional data about samples or drugs not required
to uniquely identify a row in either table.
Within the assays the
colKey are combined to form a primary key
for each assay row. This is required because each assay is stored in 'long'
format, instead of wide format as in the assay matrices within a
SummarizedExperiment. Thanks to the fast implementation of binary search
data.table package, assay tables can scale up to tens or even
hundreds of millions of rows while still being relatively performant.
Also worth noting is the cardinality between
colData for a given
assay within the assays list. As indicated by the lower connection between these
tables and an assay, for each row or column key there may be zero or more rows in
the assay table. Conversely for each row in the assay there may be zero or one key
rowData. When combined, the
colKey for a given
row in an assay become a composite key which maps that to
he current implementation of the
buildLongTable function is able to assemble
LongTable object from two sources. The first is a single large table with
all assays, row and column data contained within it. This is the structure of the Merck drug combination data that has been used to test the data structure thus far.
filePath <- '../data/merckLongTable.csv' merckDT <- fread(filePath, na.strings=c('NULL')) colnames(merckDT)
We can see that all the data related to the treatment response experiment is contained within this table.
To build a
LongTable object from this file:
rowDataCols <- list( c(cell_line1="cell_line", BatchID="BatchID")) colDataCols <- list( c(drug1='drugA_name', drug2='drugB_name', drug1dose='drugA Conc (uM)', drug2dose='drugB Conc (uM)'), c(comboName='combination_name')) assayCols <- list(viability=paste0('viability', seq_len(4)), viability_summary=c('mu/muMax', 'X/X0')) longTable <- buildLongTable(from=filePath, rowDataCols, colDataCols, assayCols)
This function will also work if directly passed a
longTable1 <- buildLongTable(from=merckDT, rowDataCols, colDataCols, assayCols) paste0('All equal? ', all.equal(longTable, longTable1))
The second option for building a
LongTable is to pass it a list of different
assays with a shared set of row and column identifiers. We haven't had a chance to testing this functionality with real data yet, but do have a toy example.
assayList <- assays(longTable, withDimnames=TRUE, metadata=TRUE, key=FALSE) assayList$new_viability <- assayList$viability # Add a fake additional assay assayCols$new_viability <- assayCols$viability # Add column names for fake assay longTable2 <- buildLongTable(from=assayList, lapply(rowDataCols, names), lapply(colDataCols, names), assayCols)
We can see that a new assay has been added to the
LongTable object when passed
a list of assay tables containing the required row and column IDs. Additionally, any row or
column IDs not already in rowData or colData will be appended to these slots automatically!
As mentioned previously, a
LongTable has both list and table like behaviours.
For table like operations, a given
LongTable can be thought of as a
colKey rectangular object.
data.frame like sub-setting for this
object, the constructor makes pseudo row and column names, which are the ID columns
for each row of
colData pasted together with a ':'.
We see that the rownames for the Merck
LongTable are the cell line name
pasted to the batch id.
For the column names, a similar pattern is followed by combining the colID columns in the form 'drug1:drug2:drug1dose:drug2dose'.
We can subset a
LongTable using the same row and column name syntax as
row <- rownames(longTable) columns <- colnames(longTable)[1:2] longTable[row, columns]
However, unlike a
matrix this subsetting also accepts partial
row and column names as well as regex queries.
head(rowData(longTable), 3) head(colData(longTable), 3)
For example, if we want to get all instance where '5-FU' is the drug:
This has matched all colnames where 5-FU was in either drug1 or drug2. If we only want to match drug1, we have several options:
all.equal(longTable[, '5-FU:*:*:*'], longTable[, '^5-FU'])
In addition to regex queries, a
LongTable object supports arbitrarily complex
subset queries using the
data.table API. To access this API, you will need to use the
. function, which allows you
to pass raw R expressions to be evaluated inside the
For example if I want to subset to rows where the cell line is VCAP and columns where drug1 is Temozolomide and drug2 is either Lapatinib or Bortezomib:
longTable[.(cell_line1 == 'CAOV3'), # row query .(drug1 == 'Temozolomide' & drug2 %in% c('Lapatinib', 'Bortezomib'))] # column query
We can also invert matches or subset on other columns in
subLongTable <- longTable[.(BatchID != 2), .(drug1 == 'Temozolomide' & drug2 != 'Lapatinib')]
To show that it works as expected:
print(paste0('BatchID: ', paste0(unique(rowData(subLongTable)$BatchID), collapse=', '))) print(paste0('drug2: ', paste0(unique(colData(subLongTable)$drug2), collapse=', ')))
head(rowData(longTable, key=TRUE), 3)
head(colData(longTable, key=TRUE), 3)
assays <- assays(longTable) assays[]
assays <- assays(longTable, withDimnames=TRUE) colnames(assays[])
assays <- assays(longTable, withDimnames=TRUE, metadata=TRUE) colnames(assays[])
Using these names we can access specific assays within a
colnames(assay(longTable, 'viability')) assay(longTable, 'viability')
colnames(assay(longTable, 'viability', withDimnames=TRUE)) assay(longTable, 'viability', withDimnames=TRUE)
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.