In msuefishlab/eodplotter: Plot EOD events from tdms files
EODPlotter User Manual
Source Code
https://github.com/msuefishlab/eodplotter
Installation
EODPlotter is designed as an R package, to be installed using a package
manager. EODPlotter is a command line alternative to the TDMSViewer
We will install EODPlotter from github. Open RStudio and run
install.packages('devtools')
This will install the devtools package which has the install_github function. Then run
devtools::install_github('msuefishlab/eodplotter')
After running this it should install the EODViewer code and its
dependencies automatically.
To install a specific version, e.g. the version mentioned in this guide, you
can also run
install_github('msuefishlab/eodplotter@0.0.4')
Using EODPlotter as a command line tool
Configure your command line
Add this line to your .bash_profile
export PATH=$PATH:`Rscript -e 'cat(paste0(.libPaths(),"/eodplotter/scripts",collapse=":"))'`:~/bin
Process a TDMS file
First use peak_finder to find all EODs and outputs a new file, e.g. file.peaks.csv, which is named based on the input file (if you analyzed fish.tdms, it would output fish.peaks.csv)
peak_finder -f file.tdms -v
Then try using the tdmsplot command, it plots the raw signal as a graph, use to diagnose signal problems and view where EODs were called
tdmsplot -f file.tdms -p file.tdms.peaks.csv
Finally analyze the signal and the peaks to produce output files
eodplot -f file.tdms -p file.peaks.csv
After running these commands you will have a number of files
- file.signal.png
- file.average.landmarks.png
- file.average.png
- file.all.png
- file.stats.csv
- file.landmarks.csv
Where file is the prefix of your TDMS file that you analyzed e.g. if it was fish123.tdms, then you would have fish123.stats.csv.
The file.stats.csv file contains useful information on duration and amplitude, and also number of EODs measured. Other files can be used for diagnosing issues with the analysis
Process all files in parallel
This script will run the EOD analysis over all tdms files in a directory
parallel "peak_finder -f {}; tdmsplot -f {} -p {/}.peaks.csv; eodplot -f {} -p {/}.peaks.csv" ::: /path/to/files/*.tdms
Combine all stats.csv
A helper script called combine_stats is added so after processing many TDMS files, simply run
combine_stats
This will output a combined stats file
Summarize EOD results from different experiments
If you have different experiments with different timepoints and you want to compare them all then this script is an example of something you could try
This will plot the amplitude from multiple experiments where it assumes some particular file naming conventions e.g.
- 1_MOCntrl1_Baseline_2017_06_29_07_34_08.tdms
- 2_MOCntrl1_30_2017_06_29_08_21_23.tdms
- 3_MOCntrl1_60_2017_06_29_09_00_21.tdms
Generic pattern <number>_MO<experimentname>_minutes_timestamp.tdms
library(stringr)
library(ggplot2)
suppressMessages(library(lubridate))
combined=read.csv('combined.data.csv',stringsAsFactors=F)
l = str_split(substr(combined$name,0,str_locate(combined$name,'2017')[,1]-2),'_')
combined$trial = as.numeric(sapply(l,function(r) r[1]))
combined$group = tolower(sapply(l,function(r) r[2]))
m = sapply(l, function(r) r[3])
m[m == 'Baseline'] = 0
combined$timepoint = as.numeric(m)
combined$date = ymd_hms(str_replace(substring(combined$name, str_locate(combined$name,'2017')[,1]), '.tdms.stats.csv',''))
png('combined.fixed.png', width=1000,height=800)
ggplot(combined, aes(timepoint, amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Peak-to-peak amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
pdf('combined.fixed.pdf', width=11,height=8)
ggplot(combined, aes(timepoint, amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Peak-to-peak amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
ret = do.call(rbind, lapply(unique(combined$group), function(group) {
r = combined[combined$group == group, ]
r$relative_amplitude = r$amplitude / r[r$timepoint == 0, ]$amplitude
r
}))
png('baseline.fixed.png', width=1000,height=800)
ggplot(ret, aes(timepoint, relative_amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude (relative to baseline)') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Relative amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
pdf('baseline.fixed.pdf', width=11,height=8)
ggplot(ret, aes(timepoint, relative_amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude (relative to baseline)') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Relative am
Example output
Using EODPlotter as an R library
Load the library
library(eodplotter)
Plot the raw signal of the data using plotTdms
plotTdms('../tests/testthat/file.tdms')
Find peaks
p = peakFinder('../tests/testthat/file.tdms')
print(head(p))
Look at where the peaks were called
plotTdms('../tests/testthat/file.tdms', peaks = p)
Get a matrix of EOD signal
m = getEODMatrix('../tests/testthat/file.tdms', p, prebaseline=T)
print(head(m))
Plot the matrix of EOD signals
plotAverage(m)
Plot all the signals
plotTotal(m)
Plot the matrix of EOD signals
landmarks = findLandmarks(m)
print(landmarks)
Plot the landmarks
plotLandmarks(m, landmarks)
Get stats about the landmarks
getStats(p, landmarks)
Command line reference
tdmsplot
tdmsplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/tdmsplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-v VERBOSE, --verbose=VERBOSE
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-s START, --start=START
Start time
-e END, --end=END
End time
Outputs a file.signal.png file representing the raw signal values of the EODs
peak_finder
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/peak_finder [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-v, --verbose
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-n NUMBER, --number=NUMBER
Peak finder threshold, sigma or voltage threshold
-d DIRECTION, --direction=DIRECTION
Direction, enter either positive/negative/none
-s START, --start=START
Start time
-e END, --end=END
End time
-r REMOVE, --remove=REMOVE
Amount to remove from start/end of recording
Outputs a file.peaks.png file representing the timepoints where EODs were detected in the TDMS file
eodplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/eodplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-p PEAK, --peak=PEAK
Peak file input
-v, --verbose
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-w WINDOW, --window=WINDOW
Window size around peak value
-b, --prebaseline
Pre-normalization baseline subtract
-d, --postbaseline
Post-normalization baseline subtract
-n, --normalize
Normalize waveform between 0-1 peak to peak height
-a ALPHA, --alpha=ALPHA
Alpha channel for individual peaks
Outputs a bunch of files completing the analysis
- file.all.png - a visualization of all EOD signal traces
- file.average.png - the average EOD waveform
- file.average.landmarks.png - the average EOD waveform with landmarks plotted
- file.landmarks.csv - the data on the landmarks detected
- file.stats.csv - the data on duration, amplitude, and number of EODs detected
multieodplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/multieodplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS/Peak file table
-c CHANNEL, --channel=CHANNEL
Channel name
-w WINDOW, --window=WINDOW
Window size around peak value
-v, --verbose
Print verbose output
-b, --prebaseline
Pre-normalization baseline subtract
-d, --postbaseline
Post-normalization baseline subtract
-n, --normalize
Normalize waveform between 0-1 peak to peak height
-a ALPHA, --alpha=ALPHA
Transparency factor aka alpha channel for plotting EODs
-p, --pdf
Use PDF output
-A, --all
Plot all traces
Conclusion
This demo shows you how to use the eodplotter as an R library. Note that there is the alternative usage of the library using the command line as well, which can be more streamlined
msuefishlab/eodplotter documentation built on May 29, 2019, 4:52 p.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
EODPlotter User Manual
Source Code
https://github.com/msuefishlab/eodplotter
Installation
EODPlotter is designed as an R package, to be installed using a package manager. EODPlotter is a command line alternative to the TDMSViewer
We will install EODPlotter from github. Open RStudio and run
install.packages('devtools')
This will install the devtools package which has the install_github function. Then run
devtools::install_github('msuefishlab/eodplotter')
After running this it should install the EODViewer code and its dependencies automatically.
To install a specific version, e.g. the version mentioned in this guide, you can also run
install_github('msuefishlab/eodplotter@0.0.4')
Using EODPlotter as a command line tool
Configure your command line
Add this line to your .bash_profile
export PATH=$PATH:`Rscript -e 'cat(paste0(.libPaths(),"/eodplotter/scripts",collapse=":"))'`:~/bin
Process a TDMS file
First use peak_finder to find all EODs and outputs a new file, e.g. file.peaks.csv, which is named based on the input file (if you analyzed fish.tdms, it would output fish.peaks.csv)
peak_finder -f file.tdms -v
Then try using the tdmsplot command, it plots the raw signal as a graph, use to diagnose signal problems and view where EODs were called
tdmsplot -f file.tdms -p file.tdms.peaks.csv
Finally analyze the signal and the peaks to produce output files
eodplot -f file.tdms -p file.peaks.csv
After running these commands you will have a number of files
- file.signal.png
- file.average.landmarks.png
- file.average.png
- file.all.png
- file.stats.csv
- file.landmarks.csv
Where file is the prefix of your TDMS file that you analyzed e.g. if it was fish123.tdms, then you would have fish123.stats.csv.
The file.stats.csv file contains useful information on duration and amplitude, and also number of EODs measured. Other files can be used for diagnosing issues with the analysis
Process all files in parallel
This script will run the EOD analysis over all tdms files in a directory
parallel "peak_finder -f {}; tdmsplot -f {} -p {/}.peaks.csv; eodplot -f {} -p {/}.peaks.csv" ::: /path/to/files/*.tdms
Combine all stats.csv
A helper script called combine_stats is added so after processing many TDMS files, simply run
combine_stats
This will output a combined stats file
Summarize EOD results from different experiments
If you have different experiments with different timepoints and you want to compare them all then this script is an example of something you could try
This will plot the amplitude from multiple experiments where it assumes some particular file naming conventions e.g.
- 1_MOCntrl1_Baseline_2017_06_29_07_34_08.tdms
- 2_MOCntrl1_30_2017_06_29_08_21_23.tdms
- 3_MOCntrl1_60_2017_06_29_09_00_21.tdms
Generic pattern <number>_MO<experimentname>_minutes_timestamp.tdms
library(stringr)
library(ggplot2)
suppressMessages(library(lubridate))
combined=read.csv('combined.data.csv',stringsAsFactors=F)
l = str_split(substr(combined$name,0,str_locate(combined$name,'2017')[,1]-2),'_')
combined$trial = as.numeric(sapply(l,function(r) r[1]))
combined$group = tolower(sapply(l,function(r) r[2]))
m = sapply(l, function(r) r[3])
m[m == 'Baseline'] = 0
combined$timepoint = as.numeric(m)
combined$date = ymd_hms(str_replace(substring(combined$name, str_locate(combined$name,'2017')[,1]), '.tdms.stats.csv',''))
png('combined.fixed.png', width=1000,height=800)
ggplot(combined, aes(timepoint, amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Peak-to-peak amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
pdf('combined.fixed.pdf', width=11,height=8)
ggplot(combined, aes(timepoint, amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Peak-to-peak amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
ret = do.call(rbind, lapply(unique(combined$group), function(group) {
r = combined[combined$group == group, ]
r$relative_amplitude = r$amplitude / r[r$timepoint == 0, ]$amplitude
r
}))
png('baseline.fixed.png', width=1000,height=800)
ggplot(ret, aes(timepoint, relative_amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude (relative to baseline)') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Relative amplitude") +scale_colour_brewer(palette="Set2")
invisible(dev.off())
pdf('baseline.fixed.pdf', width=11,height=8)
ggplot(ret, aes(timepoint, relative_amplitude, color = group)) + geom_line() + ggtitle('MO EOD amplitude (relative to baseline)') + scale_x_continuous(name="Timepoint") + scale_y_continuous(name="Relative am
Example output
Using EODPlotter as an R library
Load the library
library(eodplotter)
Plot the raw signal of the data using plotTdms
plotTdms('../tests/testthat/file.tdms')
Find peaks
p = peakFinder('../tests/testthat/file.tdms') print(head(p))
Look at where the peaks were called
plotTdms('../tests/testthat/file.tdms', peaks = p)
Get a matrix of EOD signal
m = getEODMatrix('../tests/testthat/file.tdms', p, prebaseline=T) print(head(m))
Plot the matrix of EOD signals
plotAverage(m)
Plot all the signals
plotTotal(m)
Plot the matrix of EOD signals
landmarks = findLandmarks(m) print(landmarks)
Plot the landmarks
plotLandmarks(m, landmarks)
Get stats about the landmarks
getStats(p, landmarks)
Command line reference
tdmsplot
tdmsplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/tdmsplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-v VERBOSE, --verbose=VERBOSE
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-s START, --start=START
Start time
-e END, --end=END
End time
Outputs a file.signal.png file representing the raw signal values of the EODs
peak_finder
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/peak_finder [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-v, --verbose
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-n NUMBER, --number=NUMBER
Peak finder threshold, sigma or voltage threshold
-d DIRECTION, --direction=DIRECTION
Direction, enter either positive/negative/none
-s START, --start=START
Start time
-e END, --end=END
End time
-r REMOVE, --remove=REMOVE
Amount to remove from start/end of recording
Outputs a file.peaks.png file representing the timepoints where EODs were detected in the TDMS file
eodplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/eodplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS file input
-p PEAK, --peak=PEAK
Peak file input
-v, --verbose
Print verbose output
-c CHANNEL, --channel=CHANNEL
Channel name
-w WINDOW, --window=WINDOW
Window size around peak value
-b, --prebaseline
Pre-normalization baseline subtract
-d, --postbaseline
Post-normalization baseline subtract
-n, --normalize
Normalize waveform between 0-1 peak to peak height
-a ALPHA, --alpha=ALPHA
Alpha channel for individual peaks
Outputs a bunch of files completing the analysis
- file.all.png - a visualization of all EOD signal traces
- file.average.png - the average EOD waveform
- file.average.landmarks.png - the average EOD waveform with landmarks plotted
- file.landmarks.csv - the data on the landmarks detected
- file.stats.csv - the data on duration, amplitude, and number of EODs detected
multieodplot
Usage: /Users/cdiesh/src/github.com/msuefishlab/eodplotter/inst/scripts/multieodplot [options]
Options:
-h, --help
Show this help message and exit
-f FILE, --file=FILE
TDMS/Peak file table
-c CHANNEL, --channel=CHANNEL
Channel name
-w WINDOW, --window=WINDOW
Window size around peak value
-v, --verbose
Print verbose output
-b, --prebaseline
Pre-normalization baseline subtract
-d, --postbaseline
Post-normalization baseline subtract
-n, --normalize
Normalize waveform between 0-1 peak to peak height
-a ALPHA, --alpha=ALPHA
Transparency factor aka alpha channel for plotting EODs
-p, --pdf
Use PDF output
-A, --all
Plot all traces
Conclusion
This demo shows you how to use the eodplotter as an R library. Note that there is the alternative usage of the library using the command line as well, which can be more streamlined
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.