In vqf/mtest: Direct statistical test to compare multinomial trials

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

The m-test can be used to analyze contingency tables. The main assumptions are:

• The table contains the results of several experiments.
• Each experiment consists of multiple trials which can can yield two or more exclusive outcomes.
• The number of trials in each experiment is fixed, either chosen beforehand or simply not chosen by the researcher. The number of times each outcome occurs is not fixed, any combination is possible.
• The probability of a given outcome in any trial of each experiment is always the same, regardless of previous outcomes (i. e., they are Bernouilli trials). This happens for instance when we repeatedly toss a coin.
• We have no prior information on the probability of each outcome in each experiment.

Under these assumptions, the m-test calculates the p-value under the null hypothesis that the probability of each outcome is the same in every experiment.

Example 1 - binomial test

A researcher sends 17 tissue slides for hystological examination. Nine of those slides correspond to wild-type subjects (experiment 1, in this case a control). The other eight slides come from subjects with a certain phenotype (experiment 2). The pathologist (who is blinded to those groups) examines a complex phenotype called altered, which the researcher hypothesizes is affected by the experimental group. Therefore, there are two outcomes ("altered" and "not altered"). The null hypothesis is that the probability of a given sample to be altered is the same in both experimental groups. However, we do not know any of those probabilities.We set a p=0.05 threshold to reject the null hypothesis. The results:

| Outcome | Experiment1 | Experiment2 | |--------------------:|:-----------:|:-----------:| | Altered | 2 | 6 | | Not altered | 7 | 2 |

The m-test allows the calculation of the p-value of this result. The input can be a matrix like the one depicted in the table. You can also provide each experiment as a list of vectors with the description of each experiment. under the null hypothesis:

library(mtest)
m.test(list(c(2, 7), c(6, 2)))

Therefore, we reject the null hypothesis. According to this result, the probability of a sample to be altered is different in the group with a phenotype and in the control group.

Example 2 - one-sided binomial test

Only for a 2x2 contingency table, we can also test the hypothesis that one probability is higher than the other with the os.m.test function. In this example, we will call the outcomes success (outcome 1) and failure (outcome 2). The null hypothesis states that the probability of success in the first experiment is higher than the probability of success in the second experiment. Therefore, we must place the experiment where we hypothesize the highest probability of success as the first experiment.

This example is taken from Prof. Timothy Hanson's course Stat 205: Elementary Statistics for the Biological and Life Sciences (link):

• Migraine headache patients took part in a double-blind clinical trial to assess experimental surgery.
• 75 patients were randomly assigned to real surgery on migraine trigger sites (n1 = 49) or sham surgery (n2 = 26) in which an incision was made but nothing else.
• The surgeons hoped that patients would experience \a substantial reduction in migraine headaches," which we will label as success.

The null hypothesis is that the probability of success is higher in the placebo group. Setting a limit p-val of 0.05,

| Migraine reduction? | Surgery | Placebo | |-----------------------:|:-----------:|:-----------:| | Success | 41 | 15 | | No Success | 8 | 11 |

os.m.test(list(c(15, 11), c(41, 8)))

Therefore, we can reject the null hypothesis. Notice that this test does not take into account the possibility that the probability is the same. You may also want to run a two-sided test to see if you can reject that hypothesis. With the same p-val limit of 0.05,

m.test(list(c(15, 11), c(41, 8)))

And we conclude that the probability of success in the surgery group is higher than in the placebo group.

Example 3 - multinomial test

We can also analyze larger contingency tables, although the computational power required can be limiting.

We have four urns with colored balls. We can draw from each urn, but we do not know the proportion of colors in it. There are three kinds of balls (outcomes): red, green and blue. Now we want to know if the probability to draw a given color in every urn is the same. This would mean that the proportion of balls of each color is the same in all urns. We set that as the null hypothesis and establish a threshold p-value of 0.05 to reject it. To use the m-test in this setting, there are at least two considerations:

• The probability of a given outcome (say red) must be the same in every draw from the same urn. However, when we draw a red ball, the number of red balls in the urn is lower than before. This means that in the next draw the probability to extract a red ball is lower. Therefore, for the m-test to be applicable, the ball drawn must be returned to the urn before the next extraction. This is called drawing with replacement.
• The last assumption of the m-test says that we know nothing about the probability of any outcome. In this case, we know that this probability is discrete, as it can only take rational values. This means that the test is not exact. The result will approach the real p-value as the total number of balls in the urn increases.

Suppose we get these results after 9 draws in Urns1-3 and 6 draws in Urn4:

| Outcome | Urn1 | Urn2 | Urn3 | Urn4 | |--------------------:|:----:|:----:|:----:|:----:| | Red | 3 | 4 | 1 | 3 | | Green | 3 | 5 | 2 | 2 | | Blue | 3 | 0 | 6 | 1 |

m.test(list(c(3, 3, 3), c(4, 5, 0), c(1, 2, 6), c(3, 2, 1)))

Therefore, we reject the null hypothesis. However, the test does not tell us the cause for this rejection. There might be an excess of red balls in Urn1 or perhaps the proportion of balls in all four urns is completely different.

vqf/mtest documentation built on Dec. 23, 2021, 4:11 p.m.