Nothing

```
library(shiny)
# Define UI for application that draws a histogram
ui <- fluidPage(sidebarLayout(
sidebarPanel(
sliderInput(
"pretestprob",
"Pre-test probability",
min = 0.01,
max = 1,
value = .50,
step=.01,
width='100%'
),
div(style = "float: left; width: 48%", # this number can be any percentage you need
sliderInput(
"FNR",
"False Negative Rate: Percentage of true patients with test scores below the intersection (1 - Se):",
min = 1,
max = 50,
value = 15,
width='100%'
)
),
div(style = "float: right; width: 48%", # this number can be any percentage you need
sliderInput(
"FPR",
"False Positive Rate: Percentage of true non-patients with test scores above the intersection (1 - Sp):",
min = 1,
max = 50,
value = 15,
width = '100%'
)
),
textOutput("overlap"),
p(
"The point of intersection is equal to the optimal dichotomous threshold
at which the sum of Sensitivity and Specificity (Se + Sp) is maximized,
and the sum of errors (FNR + FPR) is minimized. With the sliders
the percentages of dichotomous errors and the quality of the test are manipulated."
),
checkboxInput(
"combineSliders",
"Combine the FNR slider with the FPR slider ",
TRUE
),
div(style = "float: left; width: 48%", # this number can be any percentage you need
sliderInput(
"negpost.crit",
"Criterion for negative post-test probability:",
min = 0.01,
max = .20,
value = .10,
step=.01,
width='100%'
)
),
div(style = "float: right; width: 48%", # this number can be any percentage you need
sliderInput(
"pospost.crit",
"Criterion for positive post-test probability:",
min = .80,
max = .99,
value = .90,
step=.01,
width = '100%'
)
),
radioButtons(
"outputTable",
label = h5(
"Output Table:"
),
choices = list(
"Sensitivity and Specificity table" = 1,
"Predictive Values table" = 2
),
inline = T,
selected = 1
)
),
# Show a plot of the generated distribution
mainPanel(
plotOutput("distPlot", height='400px'),
tableOutput('table'),
p("* Using the intersection as dichotomous cutpoint."),
h3(
"Medical Decision Methods: Grey Zone Determination for Binormal
Distributed Test Scores"
),
tabsetPanel(
tabPanel(
"Introduction",
h2('Determination of a Grey zone'),
p(
"This Shiny App provides a hands-on demonstration of
the", span("Grey Zone method for bi-normal distributions", style =
"color:blue"), ", used for trichotomizing medical test results. In this
simulation, tests are simulated with a given optimal dichotomous
False Negatives Rate (1 - Sensitivity) and False Positive Rate (1 -
Specificity), allowing to compare the trichotomized results with the
optimal dichotomous results. Tests are simulated with higher
test scores for indicating the presence of the disease."
),
p(
"The Grey zone method defines a range of test scores with an
insufficient post-test probability to distinguishes patients with
and without the targeted disease (Grey zone), and determines which
test scores offer a higher or lower post-test probability than the
pre-set criteria that the targeted disease is present. Only the test
scores outside the Grey zone are used for a positive or negative
classification. The patients with test scores inside the Grey zone
can be retested or further developments can be awaited. These three
zones offer therefore a trichotomization of the test scores. This
Shiny App shows advantages and disadvantages of this evaluation
method of test scores. It is intended to help you to decide for or
against the use of this method for your medical test."
),
p(
"The Grey zone method is developed for epidemiological purposes and
determines two thresholds, one for distinguishing patients whose
test score have a higher positive post-test probability than the
preset value, and one for distinguishing patients whose test score
has a lower negative post-test probability than the preset value.
The Grey Zone is the interval of test scores that are inconclusive
and offer insufficient grounds to distinguish patients with or without
the targeted disease."
),
h4('Pre-test probability and populations'),
p(
'In epidemiological studies, the question is often how you can
measure disease and health outcomes in a population. Using a test
that indicates the presence or absence of a disease, a relevant
question is that of the probability
that a group of persons selected from the general population have
the disease or not, given their test scores. In that case, a suitable pre-test probability is
the prevalence of the disease in the population.'
),
p(
'When a clinical population is involved, for instance a group of
patients who are referred to the Neurology departments of hospitals,
the prevalence of a disease (for instance Mild Cognitive
Impairment), may differ strongly from hospital to
hospital and may differ over time. Clearly, the prevalence of the disease in this
pre-selected clinical population is higher than in the general
population. Choosing a suitable pre-test probability may be
challenging. A pre-test probability of .5 can be suitable,
reflecting uncertainty about the true prevalence in that clinical
population.'
),
p(
'In clinical diagnostic situations, a single test score is obtained and
the question is whether the patient is selected from the
sub-population with or from the sub-population without the disease. A
pre-test probability of .5 is most often suitable, reflecting uncertainty
about the presence or absence of the disease.'
),
h4('Other Tabs'),
tags$ul(
tags$li(
"Hands-on. Look at the Hands-on tab for some examples and their discussion."
),
tags$li(
"Control panel. Use this tab for explanation of the use of the controls."
),
tags$li("Background. Explanation of the method."),
tags$li(
"Conclusion and References. My conclusions (but please draw your own)
and References."
),
),
), # tabPanel
tabPanel(
"Hands-on",
h2(
"Hands-on demonstration of the bi-normal Grey zone method"
),
p(
# "Set the criterion for the post-test probability to .1 and .9 (start
# values) and check the check box to combine the two sliders. With the
# sliders you manipulate the overlap between the two distributions and
# the strength of the simulated test. A lower overlap directly defines
# a stronger test. For convenience, the", span("AUC statistic", style
# = "color:blue"), "is also presented, which is a commonly used
# estimate of the strength of the test."
),
tags$ul(
tags$li(
"1. Move the sliders to values larger than 15%. Observe that the
Grey zone grows when the test has more overlap and becomes weaker.
The Grey Interval has a desirable, low post-test probability of .5
(see Predictive Values Table), which shows that these test scores
badly distinguish patients from non-patients. Please note that the
Grey Zone grows larger and larger when the test has too much
overlap and is therefore of low quality. When the sliders hit 37,
the size of the Grey Zone is 100% and covers all scores. The Grey
Zone method is not to be used for weak tests."
),
tags$li(
'2. Make the test even stronger by reducing the percentages
further. When moving the FNR slider to values below 10, the two
borders are switched (black to the right and red to the left) and
the Grey Zone has a negative size. These stronger tests have a
better optimal dichotomous result and a Grey zone with a negative
size'), tags$em('reduces'), ('the result to the criterion. Clearly, such a
Grey Zone should not be used. Use a higher criterion or be
satisfied with the dichotomous results.'
),
tags$li(
"3. Make the test stronger by reducing the overlap between the two
distributions of the simulated test scores to two times 10%. With
the criterion of .9 for the positive post probability, the left
and right border of the Grey Zone coincide with the intersection
and the Grey Zone does not exist (has a size of 0). The optimal
dichotomous results are equal. For such a strong test, a higher
criterion is needed."
),
tags$li(
"4. Uncheck the check box for combining the two sliders. Keep the
pre-test probability of .5 and the criteria for the post-test
probabilities .1 and .9. Now we can generate instances of
distributions with unequal variance. Set the False Negative Rate
(FNR) to 15% and the False Positive Rate to 25%. The variance of
the distribution of true patients is now reduced to .73 and is
lower than the variance of the non-patients of 1. Please note the
existence of two thresholds for the positive post-test
probability. The smallest Grey Zone is used (red vertical line)
and the secondary threshold (dashed red line) is ignored. The
existence of two thresholds indicate problematic test scores where
higher scores are more indicative of absence than presence of the
disease, lowering the positive post-test probability.", 'Also,
note that the right border walks away from the intersection and,
consequently, the Grey Zone includes a part of the distribution
where the probability of presence of the disease is considerably
larger than absence of the disease: the post-test probability of
the Grey Zone is .66 (Predictive Values table). This may be
considered as not desirable and you may (if possible) want to make
the Grey Zone smaller. You can do this by selecting a lower value
for the positive post-test criterion (less variance, less
restrictive) or a lower value for the negative post-test criterion
(more variance, more restrictive) to obtain a smaller Grey Zone.
Change for instance the value for the positive post-test criterion
to .84. In that case, the post-test predictive value of the Grey
Zone is .56 (Predictive Values table), which may be considered as
more desirable, because less patients with the disease are
excluded, while in this smaller Grey Zone there is only an about
equal chance to be selected from either distribution.'
),
tags$li(
"5. Move the FPR slider to 47. Now, the positive post-test
probability takes a deep dive and does not reach the criterion
anymore. The desired Grey Zone is not available. Again, you have to
set a lower criterion here."
),
tags$li(
"6. Move the FPR slider back to 15 and the post-test probabilities
on .1 and .9. Move pre-test probability slider slowly to the left.
This makes the distribution of patients with the disease present
gradually smaller than the distribution of patients with the
disease absent. The optimal dichotomous results for Sp and Se
remains the same: both .85. Also, AUC decreases very slowly.
Please observe that the Grey Zone walks to the left. Given a test
with given Sensitivity and Specificity, changing the pre-test
probability also changes the relative variances of the
distributions. (Reversely, a test may not have the same Sensitivity
and Specificity when the relative size of the two
distributions change. Not demonstrated here.)",
p("Set the outcomes
table to Predictive Values. Set the pre-test probability to 39%.
Please observe that the lower border of the Grey zone is about equal
to the intersection, while the right border is above the mean of the
red distribution of patients with the disease present. A relatively
large part of the patients with the disease fall in the Grey Zone.
Please note that post-test probability for the lower, negative
non-Grey interval is .10 and for the upper, positive non-Grey
interval is .9, exactly as requested by setting the criterion
values. The Grey zone interval itself has a high post-test
probability of .73, which may be undesired.")
),
tags$li(
"7. Now, set the False Negative Rate to 34%. With a
pretest-probability of 39 and a FPR of 15, two distributions with
equal variance are obtained. Now, the Grey Zone is about equally
distributed around the intersection, with a desirable low
post-test probability of .49 for the Grey zone."
),
),
p(
"In this simulation, tests are simulated with a given optimal
dichotomous False Negatives Rate (1 - Sensitivity) and False
Positive Rate (1 - Specificity), allowing to compare the
trichotomized results with the optimal dichotomous results. In that
case, there are two ways to create distributions with unequal
variance: by choosing different values for FPR and FNR, or by
setting the pre-probability to a value other than .5, creating a
relatively smaller or larger distribution of patients with the
disease. A point of attention is whether the Grey zone is about
equally distributed around the intersection, which provides the best
results. When Grey Zone is not around the intersection or does not
contain the intersection, the post-test predictive value of the test
scores within the Grey Zone may be undesirable high. One might lower
or increase the criterion for either the negative post probability
criterion or the positive post probability criterion, to obtain
better results where the post-test probability is low for the Grey
Zone. It is undesirable to not use test scores that have a good
post-test predictability."
)
), # tabPanel
tabPanel(
"Control panel",
h2('Using the dash board'),
h4('Setting the pre-test probability'),
p(
'The grey panel offers a dash board where the user can simulate many
different tests with bi-normal distributions. The true presence or
absence of the disease is assumed to be known and is determined with
superior means, called a "gold standard". The two bi-normal
distributions form a mixture of patients without (black) and with
(red) the targeted disease. The pre-test probability is the
probability of selecting a patient with the disease. When a patient
is selected from the general population, the pre-test probability
equals the prevalence of the disease. In a typical clinical
diagnostic situation, the test is selected to test for a disease
which presence is probable, given the symptoms of the patient. The
clinical population is then a population of all patients with the
given symptoms, who may have or may not have the disease for which
will be tested. Often, such a clinical population is not well
described and knowledge of the true presence of the disease, given
the symptoms is lacking. Therefore, a pre-test probability of .5 may
be chosen, an equal chance that the patient is chosen from either of
the two sub-populations. One could say that this reflects the
pre-test uncertainty whether the patient, given the symptoms, has
the illness or not.'),
h4('Setting the False Negative Rate (FNR) and False Positive Rate (FPR'),
p(
'The tests differ in quality by their overlap of the scores for the
two groups. The overlap is chosen with two sliders: The upper slider
sets the percentage of patients with test scores above the point of
intersection (blue line in the left graph), indicating the amount of
False Negatives when the optimal dichotomous cut-point would be
used. The lower slider sets the percentage of patients which are
known to not have the disease and who have test scores below the
point of intersection, indicating the amount of False Negatives for
the dichotomous optimal cut-point. This allows us to compare the
results of the trichotomization method with the optimal dichotomous
method. The FNR equals 1 - Sensitivity and the FPR equals 1 -
Specificity. The two values specify the weighted overlap between the
distributions, and the weight is set by the pre-test probability.',
p(
"The total overlap between the two distributions defines directly
the strength of the simulated test, weighted by the pre-test
probability. The overlap is the percentage of all patients' test
scores that are part of both the two distributions. The smaller the
overlap, the stronger the test. In this way, a large amount of
bi-normal tests of varying strengths can be simulated. The strength
of the test is directly determined by the overlap of the
distributions of test scores: a test is stronger when the overlap is
smaller. "
),
),
h4('Combining the FNR slider with the FPR slider'),
p(
'A check box allows the combinations of the two sliders. When the two
sliders are combined, the variance remains equal for the two
distributions. Unchecking makes it possible to use the two sliders
separately, allowing the two distributions to have different
variance.'
),
h4('Criteria for the negative and positive post-test probability'),
p(
'In this application to demonstrate the working of the Grey zone
method, the criteria for the negative and positive post-test
probability can be set. The post-test probability is the probability
of the target disorder after a diagnostic test result is known. The
criterion for the negative post-test probability can be set to
values between .05 and 0.2, where a lower value indicates a more
restrictive value. The negative post test probability is the
probability of the target disorder for patients who receive a
negative decision based on the test result. The criterion for the
positive post-test probability can be set to values between .8 and
0.95, where a higher value indicates a more restrictive value. The
positive post test probability is the probability of the target
disorder for patients who receive a positive decision based on the
test result.'
),
h4('Setting Output table'),
p('The output can be set to a Sensitivity and Specificity table, or to
a table with Predictive Values (including the post-test
probability).')
), # tabPanel
tabPanel("Output Tables",
h2('Sensitivity and Specificity table'),
p(
'The first row provides the Se and Sp for the dichotomized results.
The results are optimized by using the intersection as dichotomous
threshold, which maximizes the sum of Se + Sp and minimizes the sum
of FNR + FPR. The dichotomization uses all scores. The calculations
of Se and Sp are as usual and are not dependent on the pre-test probability.'
),
p(
'The second row provides the Sensitivity and Specificity solely for
the test scores within the Grey Zone, with the intersection as
cut-point. This can only be calculated when the point of intersection
falls within the Grey Zone. Desirable values for both Se and Sp of
the Grey zone are close to .5, indicating that the test is neither
Sensitive nor Specific when using these scores. These Se and Sp are
dependent on the pre-test probability, as the Grey zone may contain
a larger part from one distribution and a smaller part of the other.
Also, the intersection may divide the Grey Zone in unequal parts.'
),
p(
'The third row provides the Sensitivity and Specificity solely for
the test scores outside the Grey Zone. This can only be calculated
when the point of intersection falls within the Grey Zone. Desirable
values for both Se and Sp of the Grey zone are values higher than the
dichotomous results, indicating that the test is more Sensitive and/or
Specific when using these scores. These Se and Sp are dependent on
the pre-test probability, as the Grey zone may contain a larger part
from one distribution and a smaller part of the other.'
),
h2('Predictive Values table'),
p(
'Predictive values can be calculated for every interval of test
scores. The post-test predictive probabilities are always equal to
the Positive Predictive Values (PPV). The Standardized Positive
Predictive Value is equal to the PPV, only when the pre-test
probability is .5. The calculated results for the post-test
predictive probability of the lower negative non-grey interval of
test scores is always equal to the set criterion for the negative
post-test probability. Similarly is the equality between the
post-test predictive probability of the upper positive non-grey
interval of test scores and the set criterion for the positive
post-test probability. The post-test probabilities of the lower
and upper non-Grey Zone are always equal to the pre-set criteria for
the negative and positive post-test probability. Desirable values are
better than the dichotomous positive post-test probability (and 1 -
this value for the set negative criterion value).'
),
p(
'The first row shows the predictive values for the dichotomized
results in the usual way with the intersection as threshold. Here,
the post predictive value is the positive post-test predictive value;
the negative post-test predictive value is 1 - the positive post-test
predictive value (not shown). Please note that the values of the
Standardized Negative Predictive Value (SNPV) and of the Standardized
Positive Predictive Value (SPPV) are independent of the Pre-test
probability. All other values are dependent on the Pre-test
probability.'
),
p(
'The second row shows the predictive values for solely the test-scores
within the Grey Zone. Please note the SNPV and SPPV are not independent
on the pre-test probability as the position of the Grey Zone may concern
a relative larger part of one of the distributions.'
),
p(
'The third row shows the predictive values for solely the test-scores of
the negative non-grey interval below the Grey Zone. Please note the SNPV
and SPPV are not independent on the pre-test probability. '
),
p(
'The fourth row shows the predictive values for solely the test-scores
of the positive non-grey interval above the Grey Zone. Please note the
SNPV and SPPV are not independent on the pre-test probability. '
),
), # tabPanel
tabPanel(
"Background",
h2("Background Information"),
p(
'This method is proposed by Coste et al. (2003, 2006). This method
uses all possible test scores as dichotomous thresholds to
calculate Se, Sp, positive and negative likelihood ratios and
post-test probabilities. The post-test probabilities are calculated for
the accumulated densities of the test scores and indicate the
levels of seriousness of the disease for test scores above and
below all possible dichotomous thresholds.'),
p(
'The criterion is a required degree of closeness of post-test
probabilities to 1 or 0. The default criterion values are .05 and
.95 for respectively a negative and positive classification, which
may be sufficient for use by clinicians or Public Health
professionals for a first classification whether a target disease
may be present or not (Coste et al., 2003). Assuming that a test
distinguishes patients from non-patients, the pre-test probability
sets the lower limit of the possible positive post-test
probabilities and the upper limit of the possible negative post-tets
probabilities. The upper limit of the possible positive post-test
probabilities and the lower limit of the negative post-test
probabilities are dependent on the test quality. Therefore, possible
criteria for the post-test probabilities are limited by the pre-test
probability and the quality of the test.'
),
p(
'A test, intended for confirming the presence or absence of a
disease, can be evaluated using two groups: a group of true patients
who have the disease and a group of patients who truly do not have
the disease (shortly called non-patients). The plot shows the two
simulated densities of the the test scores of the two groups. Higher
test scores indicate the presence of the disease. The ', span( 'Grey
zone method', style = "color:blue"), 'determines the borders of a
Grey zone for these test scores. It is assumed here that the test
scores are normally distributed. This Grey Zone indicates a range of
test scores that have sufficient post test probabilities to
distinguish the two groups of patients. '
),
h2('Test score Densities'),
p(
"The distributions shown in the plot show the densities of the
obtained simulated test scores, which are assumed to be normal
distributed. The test scores of the non-patients are standard normal
distributed, with mean of 0 and standard deviation of 1: N(0, 1).
The distribution of the true patients can vary widely. For a given
test score, the difference of the two densities indicates from which
of the two groups of patients the test score is most likely drawn.
When the sliders are combined, both simulated distributions have a
standard deviation of 1 and only the means differs. The application
starts with a distribution of test scores with a mean of 2.07 and a
standard deviation of 1 (N(2.07, 1))."
),
h2('Using Densities'),
p(
'In the Grey Zone method, post-test probabilities are used for the
determination whether a testcore (or a small range of test scores)
is sampled from the group of patients or from the group of
non-patients. The Grey zone method is intended for classification of
patients, of who it is suspected that they may have the targeted
disease. At the start, the pretest probability is set to .50. In
that case, before testing it is considered equally likely that
the patient may or may not have the disease.'
),
h2("Trichotomization versus dichotomization"),
p(
"The classic techniques for evaluating tests for medical decision
make use of dichotomization of the test scores. Implicitly, all
test scores are considered as equally valid, and all test scores are
used for a positive or negative classification. The R package
UncertainInterval offers several trichotomization methods, that
identify test scores that are insufficiently valid and are better
not used for classification. The idea behind these trichotomization
methods is that patients are better served when these invalid test
scores are not used for a positive or negative classification. These
patients are better off with retesting, if possible with a better
test, or awaiting further developments. In some medical fields the
uncertainty of a diagnostic outcome is followed up by techniques
such as watchful waiting and active surveillance. These more
cautious approaches are especially relevant when possible treatments
can have serious side effects."
),
), # tabPanel
tabPanel(
"Conclusion and References",
h2('In conclusion'),
p(
'The largest advantage of using trichotomization methods is
the identification of patients with uncertain, inconclusive test
scores. This allows for following a more cautious path for these
patients.'),
p(
'The simulations show that the Grey Zone method for bi-normal data may
offer a better positive or negative classification of patients, with a
sufficient post-test probability, when solely the test scores outside
the Grey Zone are used. To reach this goal, criteria for the post-test
probability have to be chosen, relative to the quality of the test.
The method works best when the variances of the distributions of both
the sub-populations of patients are about equal. When the variance is
unequal, a part of scores may be placed in the Grey Zone, that in fact
offer a reasonable distinction between the two sub-populations and therefore a
relatively high pre-test probability of presence of the disease in
the Grey Zone. In that case, the criterion for the sub-population with
the higher variance has to be lowered.'
),
p(
'Assuming that a test distinguishes patients from non-patients, the
pre-test probability sets the lower limit of the possible positive
post-test probabilities and the upper limit of the possible negative
post-tets probabilities. The upper limit of the possible positive
post-test probabilities and the lower limit of the negative
post-test probabilities are dependent on the test quality. Therefore,
possible criteria for the post-test probabilities are limited by the
pre-test probability and the quality of the test.'
),
p(
'The main disadvantage of this method is that different criteria
have to be applied, dependent on the the pre-test probability and
the quality of the test. Important for the qualities of the test are
the variances of the test scores in the two sub-populations of
patients with and without the targeted disease. Setting suitable
criteria is therefore especially a problem when clinical populations
are involved for which the distributions are not exactly known.'
),
h2('References'),
p(
'Coste, J., Jourdain, P., & Pouchot, J. (2006). A gray zone assigned
to inconclusive results of quantitative diagnostic tests:
application to the use of brain natriuretic peptide for diagnosis of
heart failure in acute dyspneic patients. Clinical Chemistry,
52(12), 2229-2235.'
),
p(
'Coste, J., & Pouchot, J. (2003). A grey zone for quantitative
diagnostic and screening tests. International Journal of
Epidemiology, 32(2), 304-313.'
),
p(
'Landsheer, J. A. (2020). Impact of the Prevalence of Cognitive
Impairment on the Accuracy of the Montreal Cognitive Assessment:
The Advantage of Using two MoCA Thresholds to Identify Error-prone
Test Scores. Alzheimer Disease & Associated Disorders.',
tags$a(href = "https://dx.doi.org/10.1097/WAD.0000000000000365",
"https://dx.doi.org/10.1097/WAD.0000000000000365")
),
p(
'Landsheer J. A. (2020). CRAN - Package UncertainInterval 0.6.0.',
tags$a(
href = " https://CRAN.R-project.org/package=UncertainInterval",
" https://CRAN.R-project.org/package=UncertainInterval"
)
),
p(
'Youden, W. J. (1950). Index for rating diagnostic tests.
Cancer, 3(1), 32–35.',
tags$a(
href = "https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3",
"https://doi.org/10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3"
)
),
) # tabPanel
) # tabsetpanel
) # mainpanel
) # sidebarLayout
) # fluidpage
```

**Any scripts or data that you put into this service are public.**

Embedding an R snippet on your website

Add the following code to your website.

For more information on customizing the embed code, read Embedding Snippets.