Description Usage Arguments Details Value References Examples
View source: R/bayesian_sequential_tests.R
This function implements a sequential approach of the Bayesian version of the sign test to compare the performance of machine learning algorithms to one another. Sample size is not fixed in advance, data are evaluated as they are collected. Further sampling is stopped in accordance with a pre-defined stopping rule.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
problem |
('character') |
baseline |
('character') |
algorithm |
('character') |
measure |
('character') |
compare |
('character') |
s |
('double') |
z_0 |
('double') |
rope |
('double') |
weights |
('any') |
mc_samples |
('double') |
max_repls |
('double') |
prob |
('double') |
min_repls |
('double') |
... |
(any) |
The basis for this test has first been implemented in rNPBST. For testing over multiple datasets, donĀ“t specify the problem set argument in the function.
('list')
A list containing the following components:
measure
('character')
A string with the name of the
measure column.
method
('character')
A string with the name of the
method.
baseline
('character')
A string with the name of the
first algorithm. Value in 'algorithm' column.
data_frame
('list')
A list containing the following
components:
algorithm
('character')
Second algorithm. Value in
'algorithm' column. If not defined, the baseline is tested against
all algorithms in the data frame.
left
('double')
Left probability.
rope
('double')
Rope probability.
right
('double')
Right probability.
repls
('double')
Number of evaluated replications.
probabilities
('character')
Decisions based on posterior
probabilities and threshold probability.
https://github.com/JacintoCC/rNPBST
1 2 3 |
results <- seq_b_sign_test(df = test_benchmark_small,
baseline = "algo_1", algorithm = "algo_2", max_repls = 10)
|
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.