In ABS-dev/DiagTestKit: Functions used in evaluating sensitivity and specificity at CVB Statistics

knitr::opts_chunk$set(echo = TRUE, tidy = FALSE)
options(width = 50)
library(DiagTestKit)
data("ExampleData")

Introduction

This vignette includes several examples for estimating the performance characteristics of an experimental kit testing samples from one or more populations with the experimental test and one or more reference tests. The examples in this vignette begin with a data frame containing the counts for all possible testing combinations for all populations. For a more basic example that includes obtaining the data frame of counts necessary to use the estimateSnSp function, please see the Getting Started Vignette. The notation used here is consistent with that found in CVB STATWI0002.

The simplest case illustrated here is estimating the sensitivity and specificity of a 2--state experimental test using the test results of the experimental kit and those obtained from a 2--state fallible reference test on samples tested from a single population. The time necessary to perform all optimizations was slightly over a minute. The optimization takes significantly longer when estimating the parameters for a 3--state experimental test because the optimization is over 4 parameters for a 3--state experimental test, rather than over 2 parameters for a 2--state experimental test. The most complex example here is a 3--state experimental test, three 2--state fallible reference tests and samples tested from 3 populations. The time necessary to perform all optimizations was slightly less than 12 minutes using a Windows 10 platform with an Intel i7 processor. Code in this vignette was run using version r packageVersion("DiagTestKit") of the DiagTestKit package.

Example 1 --- One, 2--state reference test, One population, 2--state experimental test

In this example, samples randomly selected from a single population are tested by one 2--state fallible reference test and a 2--state experimental test. Here, $S_1$ = 2, $S_2$ = 2 and $K$ = 4. There are 4 unique sets of testing combinations. Further, the probabilities associated with a suspect test result are 0 for both the experimental test method and the reference test method. Specifically, $\phi_1$ = $\psi_1$ = $\phi_2$ = $\psi_2$ = 0. This example uses named data frames for the input variables Sn.ref and Sp.ref.

ex1 <- estimateSnSp(dat = data1,
          Sn.ref = data.frame(ref = c(0.90, 0)),
          Sp.ref = data.frame(ref = c(0.99, 0)),
          prev.pop = c(A = 0.80),
          control = estimateSnSpControl(seed = 64725,
                                        rep.iter = FALSE))
unique(ex1$detailOut$Converge)
unique(ex1$detailOut$Message)
ex1

Changing the confidence level

The confidence level can be updated without requiring repeating the simulations.

ex1_update <- updateAlpha(ex1, newAlpha = 0.01)
ex1_update

Example 2 --- One, 2--state reference test, Three populations, 2--state experimental test

In this example, samples randomly selected from 3 populations are tested by one 2--state fallible reference test and a 2--state experimental test. Here, $S_1$ = 2, $S_2$ = 2 and $K$ = 4. Again there are 4 unique sets of testing combinations, but a total of 12 counts, one for each unique test combination in each of the three populations. The probabilities associated with a suspect test result are 0 for both the experimental test method and the reference test method. Specifically, $\psi_1$ = $\phi_1$ = $\psi_2$ = $\phi_2$ = 0. This example uses named vectors for the input variables Sn.ref and Sp.ref because the reference test only had 2 states.

ex2 <- estimateSnSp(dat = data2,
         Sn.ref = c(ref_result = 0.90),
         Sp.ref = c(ref_result = 0.94),
         prev.pop = c(A = 0.92, B = 0.20, C = 0.50),
         control = estimateSnSpControl(seed = 4902342,
                                       rep.iter = FALSE))
unique(ex2$detailOut$Converge)
unique(ex2$detailOut$Message)
ex2

Example 3 --- One, 3--state reference test, One population, 2--state experimental test

Samples randomly selected from a single population are tested by one 3--state fallible reference test and a 2--state experimental test. Here, $S_1$ = 2, $S_2$ = 3 and $K$ = 6. There are 6 unique sets of testing combinations. The probabilities associated with a suspect test result for the experimental test are zero (i.e. $\psi_1$ = $\phi_1$ = 0). Because the reference test has 3 states, the input must include values of $\delta_2 = \frac{\psi_2}{1 - \pi_2}$ and $\gamma_2 = \frac{\phi_2}{1 - \theta_2}$.

ex3 <- estimateSnSp(dat = data3,
        Sn.ref = data.frame(ref = c(0.99, 0.75)),
        Sp.ref = data.frame(ref = c(0.85, 0.67)),
        prev.pop = c(A = 0.92),
        control = estimateSnSpControl(seed = 896421,
                                      rep.iter = FALSE))

unique(ex3$detailOut$Converge)
unique(ex3$detailOut$Message)
ex3

Example 4 --- One, 3--state reference test, Three populations, 2--state experimental test

In this example, samples randomly selected from three populations are tested by one fallible 3--state reference test and a 2--state experimental test. There are 6 unique sets of testing combinations ($S_1$ = 2, $S_2$ = 3 and $K$ = 6) and a total of 18 counts, one for each unique test combination in each population. The probabilities associated with a suspect test result for the experimental test are zero (i.e. $\psi_1$ = $\phi_1$ = 0).

ex4 <- estimateSnSp(dat = data4,
        Sn.ref = data.frame(ref = c(0.95, 0.55)),
        Sp.ref = data.frame(ref = c(0.93, 0.48)),
        prev.pop = c(A = 0.97, B = 0.25, C = 0.68),
        control = estimateSnSpControl(seed = 6589732,
                                    rep.iter = FALSE))

unique(ex4$detailOut$Converge)
unique(ex4$detailOut$Message)
ex4

Example 5 --- One, 2--state reference test, Two populations, 3--state experimental test

In this example, samples randomly selected from two populations are tested by a 3--state experimental test and one 2--state reference test. There are 6 unique testing combinations ($S_1$ = 3, $S_2$ = 2 and $K$ = 6) and a total of 12 counts as samples were obtained from 2 populations. The probabilities associated with a suspect test result for the experimental test ($\psi_1$ and $\phi_1$) are included in the output (but the optimization occurs in terms of $\delta_1$ and $\gamma_1$). The function provides a message to inform the user that the experimental test has 3 states and the optimization will be more time consuming.

ex5 <- estimateSnSp(dat = data5,
         Sn.ref = c(Ref1 = 0.90),
         Sp.ref = c(Ref1 = 0.99),
         prev.pop = c(A = 0.80, B = 0.90),
         control = estimateSnSpControl(seed = 1249856,
                                       rep.iter = FALSE))

unique(ex5$detailOut$Converge)
unique(ex5$detailOut$Message)
ex5

Example 6 --- Two, 2--state reference tests, One population, 2--state experimental

In this example, samples randomly selected from a single population are tested with two 2--state reference tests and a 2--state experimental test. Here, $S_1$ = 2, $S_2$ = 2, $S_3$ = 2 and $K$ = 8. There are 8 unique sets of testing combinations. The probabilities associated with a suspect test result for the experimental test are zero (i.e. $\psi_1$ = $\phi_1$ = 0).

ex6 <- estimateSnSp(dat = data6,
          Sn.ref = c(Ref1_result = 0.95, ref2 = 0.91),
          Sp.ref = c(Ref1_result = 0.85, ref2 = 0.98),
          prev.pop = c(A = 0.82),
          control = estimateSnSpControl(seed = 2948217,
                                        rep.iter = FALSE))

unique(ex6$detailOut$Converge)
unique(ex6$detailOut$Message)
ex6

Example 7 --- Two, 3--state reference tests, Two populations, 2--state experimental

In this example, samples selected from 2 populations were tested by two 3--state reference tests and a 2--state experimental test. Here, $S_1$ = 2, $S_2$ = 3, $S_3$ = 3 and $K$ = 18. There are 18 unique sets of testing combinations and a total of 36 counts, one for each unique test combination in each of the 2 populations. The probabilities associated with a suspect test result for the experimental test are zero (i.e. $\psi_1$ = $\phi_1$ = 0).

ex7 <- estimateSnSp(dat = data7,
          Sn.ref = data.frame(ref1 = c(0.88, 0.75), ref2 = c(0.90, 0.55)),
          Sp.ref = data.frame(ref1 = c(0.97, 0.6), ref2 = c(0.95, 0.5)),
          prev.pop = c(A = 0.87, B = 0.35),
          control = estimateSnSpControl(seed = 1937457,
                                        rep.iter = FALSE))

unique(ex7$detailOut$Converge)
unique(ex7$detailOut$Message)
ex7

Example 8 --- Three, 2--state reference tests, Three populations, 3--state experimental

In this example, samples randomly selected from 3 populations were tested by three 2--state reference tests and a 3--state experimental test. Here, $S_1$ = 3, $S_2$ = 2, $S_3$ = 2, $S_4$ = 2 and $K$ = 24. There are 24 unique sets of testing combinations and a total of 72 counts, one for each unique test combination in each of the 3 populations. The probabilities associated with a suspect test result for the experimental test ($\psi_1$ and $\phi_1$) are included in the output. The function provides a message to inform the user that the experimental test has 3 states and the optimization will be more time consuming.

ex8 <- estimateSnSp(dat = data8,
       Sn.ref = c(ref1_result = 0.92, ref2_result = 0.88, ref3_result = 0.85),
       Sp.ref = c(ref1_result = 0.86, ref2_result = 0.90, ref3_result = 0.92),
       prev.pop = c(A = 0.95, B = 0.62, C = 0.18),
       control = estimateSnSpControl(seed = 865213,
                                     rep.iter = FALSE))

unique(ex8$detailOut$Converge)
unique(ex8$detailOut$Message)
ex8

ABS-dev/DiagTestKit documentation built on Sept. 23, 2024, 9:37 a.m.