quality.threshold.uncertain: Function for the description of the qualities of the...

In UncertainInterval: Uncertain Interval Methods for Three-Way Cut-Point Determination in Test Results

Description

This function can be used only for trichotomization (double thresholds or cut-points) methods. In the case of the Uncertain Interval trichotomization method, it provides descriptive statistics for the test scores within the Uncertain Interval. For the TG-ROC trichotomization method it provides the descriptive statistics for TG-ROC's Intermediate Range.

Usage

quality.threshold.uncertain(
  ref,
  test,
  threshold,
  threshold.upper,
  intersection = NULL,
  model = c("kernel", "binormal", "ordinal"),
  tests = FALSE,
  direction = c("auto", "<", ">")
)

Arguments

ref	The reference standard. A column in a data frame or a vector indicating the classification by the reference test. The reference standard must be coded either as 0 (absence of the condition) or 1 (presence of the condition)
test	The index test or test under evaluation. A column in a dataset or vector indicating the test results in a continuous scale.
threshold	The lower decision threshold of a trichotomization method.
threshold.upper	The upper decision threshold of a trichotomization method. Required.
intersection	(default = NULL). When NULL, the intersection is calculated with get.intersection, which uses the kernel density method to obtain the intersection. When another value is assigned to this parameter, this value is used instead.
model	(default = 'kernel'). The model used defines the intersection. Default the kernel densities are used with adjust = 1, for ordinal models adjust = 2 is used. For bi-normal models the bi-normal estimate of the intersection is used. The model defines the intersection, which defines the output of this function.
tests	(default = FALSE). When TRUE the results of chi-square tests and t-tests are included in the results.
direction	Default = "auto". Direction when comparing controls with cases. When the controls have lower values than the cases (direction = "<"). When "auto", mean comparison is used to determine the direction.

Details

The Uncertain Interval is generally defined as an interval below and above the intersection, where the densities of the two distributions of patients with and without the targeted impairment are about equal. The various functions for the estimation of the uncertain interval use a sensitivity and specificity below a desired value (default .55). This function uses the intersection (the optimal dichotomous threshold) to divide the uncertain interval and provides in this way the indices for the uncertain interval when the optimal threshold would have been applied.

The patients that have test scores within the Uncertain Interval are prone to be incorrectly classified on the basis of their test result. The results within the Uncertain Interval differ only slightly for patients with and without the targeted condition. Patients with slightly lower or higher test scores too often have the opposite status. They receive the classification result 'Uncertain'; it is better to apply additional tests or to await further developments.

As the test scores have about equal densities, it may be expected that Chi-square tests are not significant, provided that the count of individuals within the Uncertain Interval is not too large. Most often, the t-tests are also not significant, but as the power of the t-test is considerably larger than the power of the Chi-square test, this is less often the case. It is recommended to look at the difference of the means of the two sub-samples and to visually inspect the inter-mixedness of the densities of the test scores.

When applying the method to the results of a logistic regression, one should be aware of possible problems concerning the determination of the intersection. Somewhere in the middle, logistic predictions can have a range where the distributions have similar densities or have multiple intersections near to each other. Often, this problem can be approached effectively by using the linear predictions instead of the logistic predictions. The linear predictions offer often a far more clear point of intersection. The solution can then be applied to the prediction values using the inverse logit of the intersection and the two cut-points. The logistic predictions and the linear predictions have the same rank ordering.

NOTE: Other trichotomization methods such as TG.ROC have no defined position for its Intermediate Range. For TG.ROC usage of the point where Sensitivity=Specificity seems a reasonable choice.

Value

A list of

direction

Shows whether controls (0) are expected to have higher or lower scores than patients (1).

intersection

The value used as estimate of the intersection (that is, the optimal threshold).

table

The confusion table of UI.class x ref for the Uncertain Interval where the scores are expected to be inconclusive. The point of intersection is used as a dichotomous cut-point within the uncertain interval (UI). UI.class is the classification of the UI scores divided by the point of intersection, 0 (UI scores < point of intersection and 1 (UI scores >= point of intersection. Both the reference standard (ref) and the classification based on the test scores (UI.class) have categories 0 and 1. Table cell 0, 0 shows the True Negatives (TN), cell 0, 1 shows the False Negatives (FN), cell 1, 0 shows the False Positives (FP), and cell 1, 1 shows the True Positives (TP).

cut

The values of the thresholds.

X2

When tests is TRUE, the table with the outcomes of three Chi-square tests of the confusion table is shown:

• TN.FP: Chi-square test of the comparison of TN versus FP.

• FN.TP: Chi-square test of the comparison of FN versus TP.

• overall: Chi-square test of all four cells of the table.

t.test

When tests is TRUE, a table is shown with t-test results for the comparison of the means. Within the Uncertain Interval, the test scores are compared of individuals without the targeted condition (ref = 0) and individuals with the targeted condition (ref = 1).

indices

A named vector, with the following statistics for the test-scores within the Uncertain Interval, using the point of intersection (optimal threshold) as dichotomous cut-point within the uncertain interval.

• Proportion.True: Proportion of classified patients with the targeted condition (TP+FN)/(TN+FP+FN+TP). Equal to the sample prevalence when all patients are classified.

• UI.CCR: Correct Classification Rate or Accuracy (TP+TN)/(TN+FP+FN+TP)

• UI.balance: balance between correct and incorrect classified (TP+TN)/(FP+FN)

• UI.Sp: Specificity TN/(TN+FN)

• UI.Se: Sensitivity TP/(TP+FN)

• UI.NPV: Negative Predictive Value TN/(TN+FN)

• UI.PPV: Positive Predictive Value TP/(TN+FN)

• UI.SNPV: Standardized Negative Predictive Value

• UI.SPPV: Standardized Positive Predictive Value

• LR-: Negative Likelihood Ratio P(-|D+))/(P(-|D-)) The probability of a person with the condition receiving a negative classification / probability of a person without the condition receiving a negative classification.

• LR+: Positive Likelihood Ratio (P(+|D+))/(P(+|D-)) The probability of a person with the condition receiving a positive classification / probability of a person without the condition receiving a positive classification.

• UI.C: Concordance or C-Statistic or AUC: The probability that a random chosen patient with the condition is correctly ranked higher than a randomly chosen patient without the condition. Equal to AUC, with for the uncertain interval an expected outcome smaller than .60. (Not equal to a partial AUC.)

See Also

UncertainInterval for an explanatory glossary of the different statistics used within this package.

Examples

# A simple test model
ref=c(rep(0,500), rep(1,500))
test=c(rnorm(500,0,1), rnorm(500,1,sd=1))
ua = ui.nonpar(ref, test)
quality.threshold.uncertain(ref, test, ua[1], ua[2])

UncertainInterval documentation built on March 3, 2021, 1:10 a.m.