experiments.bench: Noise sensitivity benchmark
In paubellot/netbenchmark: Benchmarking of several gene network inference methods

Description Usage Arguments Details Value Author(s) See Also Examples

For a given vector of character of the names of wrapper functions that compute a network inference methods, experiments.bench performs a number of experiments sensitivity test. It makes use of five different big gene datasets subsampling them to generate different datasets.num of the network with different number of experiments.

    experiments.bench(methods = "all.fast", datasources.names = "all",
        experiments = c(20, 50, 150), eval = "AUPR",
        no.topedges = 20, datasets.num = 3, local.noise = 20,
        global.noise = 0, noiseType = "normal", sym = TRUE,
        seed = NULL, verbose= TRUE)

`methods`	A vector of characters containing the names of network inference algorithms wrappers to be compared (default: "all.fast").
`datasources.names`	A vector of characters containing the names of network datasets to be included in the benchmark (default: "all").
`experiments`	A vector to set the number of experiments to test the methods (default=c(20,50,150)).
`eval`	The name of the evaluation metric among the following ones: "no.truepos", "AUROC" or "AUPR" (default : "AUPR") - see `evaluate`.
`no.topedges`	Float specifying the percentage number of links to be considered in the evaluation (default: 20).
`datasets.num`	Number of repetitions in the noise evaluation, for each method and each dataset and each noise intensity (default: 3).
`local.noise`	Integer specifying the desired percentage of local noise to be added at each of the subsampled datasets (default: 20) - see `datasource.subsample`.
`global.noise`	Integer specifying the desired percentage of global noise to be added at each of the subsampled datasets (default: 20) - see `datasource.subsample`.
`noiseType`	Character specifying the type of the noise to be added: "normal" or "lognormal" (default: "normal") - see `datasource.subsample`.
`sym`	Logical specifying if the evaluation is symmetric (default: TRUE) - see `evaluate`.
`seed`	A single value, interpreted as an integer to specify seeds, useful for creating simulations that can be reproduced (default: `NULL`) - see `set.seed`.
`verbose`	Logical specifying if the code should provide a log about what the function is doing (default: TRUE).

The argument methods accepts "all.fast" and "all" (case insensitive) as a parameters:

"all.fast" performs network inference with "aracne", "c3net", "clr", "GeneNet", "mutual ranking", "mrnet"
"all" performs network inference with "aracne", "c3net", "clr", "GeneNet", "Genie3", "mutual ranking", "mrnet", "mrnetb"

It evaluates the first no.topedges % of the possible links inferred by each algorithm at each dataset.

Two different types of noises are added independently:

"Local": the standard deviation of the noise is different for each variable. local.noise specifies the percentage for each variable ( \pm 20 \% ).
"Global": the standard deviation of the noise is the same for the whole dataset. global.noise specifies the percentage of the mean standard deviation of all the variables ( \pm 20 \% ).

The distribution of noise is set with noiseType, it is possible to choose between "normal" (rnorm) and "lognormal" (rlnorm). The argument noiseType can be a single character, this specifies the same distribution for both "Local" and "Global" noise, it also can be a vector of characters with two elements, the former specifies the distribution of "Local" noise and the later the distribution of "Global" noise.

experiments.bench returns a list with three elements:

A data.frame which is the result table containing the number of true positives as an evaluation measure. It evaluates each algorithm specified at methods at each one of the specified datasources.names with different noise intensities.
A data.frame which is the corresponding pvalue table of the corresponding statistical test for each one of the datasets.num between the best algorithm and the others.
The seed of the random number generators that allows the replication of the results.

Pau Bellot and Patrick Meyer

netbenchmark, noise.bench