SummarizeMultiRuns: Assess/evaluate multiple summarized runs on one dataset by...

In WuyangFF95/SynSigEval: Evaluate Results of Mutational Signature Analysis Software on Synthetic Spectra Created by Package SynSigGen

Assess/evaluate multiple summarized runs on one dataset by one computational approach.

Description

Summarize results from each computational approach in resultPath/run.names (generated by running a computational approach), combine them into resultPath.

Usage

SummarizeMultiRuns(datasetName, toolName, resultPath, run.names)

Arguments

datasetName	Name of the dataset. (e.g. "S.0.1.Rsq.0.1"). Usually, it is has the same name as basename(top.dir).
toolName	Name of computational approach. (e.g. "SigProExtractor")
resultPath	Path expected to have multiple result folders each named as run.names (e.g. "seed.1"). The example resultPath is S.0.1.Rsq.0.1/sp.sp/ExtrAttr/hdp.results/ in old folder structure, or 3a.Original_output_K_unspecified/hdp/S.0.1.Rsq.0.1 in new folder structure.
run.names	A character vector records the list of directories which are under resultPath and contain results of computational approach, and a summary folder generated by SummarizeSigOneExtrAttrSubdir.

Details

Also writes multiple files into folder resultPath.

Value

A list contain values of measures measures in multiple runs:

• $averCosSim Average cosine similarity. Only similarities between TP sigs and extracted sigs most similar to them.

• $truePos True Positives(TP): Reference signatures which are active in the spectra, and extracted.

• $falseNeg False Negatives(FN): Reference signatures not extracted.

• $falsePos False Positives(FP): Signatures wrongly extracted, not resembling any reference signatures.

• $TPR True positive rate (TPR, Sensitivity): TP / (TP + FN)

• $PPV Positive predictive value (PPV, Precision): TP / (FP + TP)

• $cosSim Cosine similarity between each of the reference signatures, and its most similar extracted signature.

• $AggManhattanDist (if exposures of signatures were inferred) Scaled Manhattan distance between ground-truth and inferred exposures to each of the reference signatures.

This list also contains mean and sd, and other statistics of these measures in

• $fivenum - summary generated by fivenum - columns of this table refer to Tukey's five number summary for each extraction measure across all runs:

  • min - minimum

  • lower-hinge - first quartile. Serve as the lower-hinge of the box-whisker plot.

  • median - median of measure across all runs.

  • upper-hinge - third quartile. Serve as the upper-hinge of the box-whisker plot.

  • max - maximum

• $fivenumMD - Tukey's five number summary for aggregately-scaled Manhattan distance.

• $meanSD - mean and standard deviation for extraction measures.

• $meanSDMD - mean and standard deviation for aggregately-scaled Manhattan distance.

WuyangFF95/SynSigEval documentation built on Sept. 18, 2022, 11:41 a.m.