In jimvine/confinterpret: Descriptive Interpretations of Confidence Intervals
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.width=6.5, fig.height=6.5
)
library("confinterpret")
#' Arrange an interpretation_set object into a table, for printing etc.
tabulate_interpretation_set <- function(interpretation_set, name) {
int_md <- as.data.frame(sapply(interpretation_set$interpretations,
"[[", "interpretation_md"))
rownames(int_md) <- LETTERS[1 : nrow(int_md)]
colnames(int_md) <- name
int_md
}
table_nums <- captioner::captioner(prefix = "Table")
figure_nums <- captioner::captioner(prefix = "Figure")
As a framework for interpreting range-based results, one of the stengths of the confinterpret package is that it is extensible. The extensibility is primarily provided in the form of being able to systematically define new interpretation_set objects. The following example is provided to demonstrate how users can make use of this extensibility.
Motivating example: 'Practical superiority'
The concept of a minimum practical difference is used in some forms of analyses, such as typically forming the basis for establishing the non-inferiority margin in non-inferiority tests. (See ?interpret_noninferiority for further details.) In this example we demonstrate the process for establishing a new interpretation_set using this concept in the case where we wish to conduct a superiority test, but to insist on superiority above some threshold before declaring the intervention to be meaingfully superior.
Although this is not a typical approach, and hence not provided by default within the package, it is plausible as an option that a user may wish to conduct. A result that is of statistical significance --- which is equivalent to having a confidence interval that is entirely superior --- may not equate to a practically significant result. A narrow confidence interval that is all superior --- for example 0.1% to 0.3% improvement --- would pass a standard superiority test but might not be practically superior if decision-makers are only interested in improvements of more than, say, 1%. The example is, however, provided to illustrate the process of extending the package, rather than to particularly recommend this design of interpretations.
The process of establishing a new interpretation_set object is helpful because it forces the user to consider, and formally specify, the interpretation that will apply in all eventualities. This is particularly useful if conducted before analysis commences, as it gets the analysts to establish in advance how they will describe the findings if any particular outcome is observed. As one part of a strategy to reduce researcher degrees of freedom [@simmons_false-positive_2011] it can contribute to the enhancement of the credibility of findings and ultimately improve the quality of the published literature.
Whilst an interpretation_set object can be built from scratch in one step with a call to the interpretation_set() function, the following process can make it easier to build the object in stages:
- First, build the object with only identifying labels, not meaningful interpretations.
- Then plot the object to see the range of interpretations that need to be provided, based on the boundaries.
- Draft the interpretations and re-build the object with these in place.
- Re-plot and print the object to check your interpretations are correct.
Building an interpretation_set with identifying labels
To build an interpretation_set object you will first need to consider the boundaries that specify the type of analysis you are undertaking. Having established the boundaries you will be able to identify how many interpretations you need to specify, as follows.
The number of boundaries, $n_b$ specified in an interpretation_set can be thought of as establishing a number of regions, $n_r$, within which the lower and upper confidence limits can sit. This is given by:
$$n_r = n_b + 1$$
since each boundary can be thought of as having an associated region above it (running up to the next boundary, or in the case of the top boundary, running to infinity), and in addition there is a region for below the bottom boundary. The number of regions in turn determines the number of interpretations. The valid combinations are those where the upper confidence limit is in a region greater than or equal to the region of the lower confidence limit. Consequently, the number of valid combinations is given by:
$$sum(1 : n_r)$$
which equals:
$$n_r (n_r + 1) / 2$$
An interpretation needs to be provided for each of these combinations. For likely (i.e., small) numbers of boundaries, the numbers of regions and interpretations are as shown in r table_nums(name = "numbers_b_r_i", display = "cite").
number_boundaries <- 1 : 5
number_regions <- number_boundaries + 1L
number_interpretations <- as.integer(number_regions * (number_regions + 1L) / 2)
numbers <- data.frame(number_boundaries,
number_regions,
number_interpretations,
row.names = NULL)
colnames(numbers) <- c("Number of boundaries, $n_b$",
"Number of regions, $n_r$",
"Number of interpretations")
knitr::kable(numbers,
caption = table_nums(name = "numbers_b_r_i",
caption = "Number of regions and interpretations for each number of boundaries."))
Since the interpretation_set() function calls a validator function, if you make an error in setting up the correct number of interpretations it will be caught and an error reported.
The order in which interpretations should be provided is well-defined. The order is based on first specifying all the cases where the lower confidence limit is in the bottom region, and each of the regions for the upper confidence limit (again, starting from the bottom and increasing to the top); next come all of the cases where the lower confidence limit is in the second-from-bottom region (in this case the valid regions for the upper confidence limit will start at the second-from-bottom and go up to the top region); and so on. So for a 2 region (1 boundary) situation, the interpretations should be provided in the order in r table_nums(name = "order_2", display = "cite").
region_orders <- function(number_regions) {
number_interpretations <- number_regions * (number_regions + 1) / 2
lower_regions <- rep(1 : number_regions, number_regions : 1)
upper_regions <- unlist(mapply(function(x) { x : number_regions },
1 : number_regions))
regions <- data.frame(1 : number_interpretations,
paste("Region", lower_regions),
paste("Region", upper_regions))
colnames(regions) <- c("Order",
"Lower confidence level",
"Upper confidence level")
rownames(regions) <- LETTERS[1 : number_interpretations]
return(regions)
}
boundary_2 <- region_orders(2)
knitr::kable(boundary_2,
caption = table_nums(name = "order_2",
caption = "Order of interpretations with two regions."))
For a 3 region (2 boundary) situation, the interpretations should be provided in the order in r table_nums(name = "order_3", display = "cite").
boundary_3 <- region_orders(3)
knitr::kable(boundary_3,
caption = table_nums(name = "order_3",
caption = "Order of interpretations with three regions."))
Rather than trying to match interpretations to these descriptions from scratch, an easier approach may be to instead first create and plot a dummy version of your interpretation_set object with the progression of letters through the alphabet (r figure_nums("dummy-interpretation-set-plot", display = "cite")).
dummy_i_s_label <- figure_nums("dummy-interpretation-set-plot", "Plot of a 'dummy' `interpretation_set` object, providing visual support for use while drafting interpretations.")
practical_superiority <- interpretation_set(
boundary_names =c("Actual null", "Practical null"),
placeholders = c(comparison_intervention = "$comp",
tested_intervention = "$test"),
interpretations = list(
# A
list(interpretation_short = "A",
interpretation = "A",
interpretation_md = "A"),
# B
list(interpretation_short = "B",
interpretation = "B",
interpretation_md = "B"),
# C
list(interpretation_short = "C",
interpretation = "C",
interpretation_md = "C"),
# D
list(interpretation_short = "D",
interpretation = "D",
interpretation_md = "D"),
# E
list(interpretation_short = "E",
interpretation = "E",
interpretation_md = "E"),
# F
list(interpretation_short = "F",
interpretation = "F",
interpretation_md = "F")))
# Set a nice colour scheme
grDevices::palette(RColorBrewer::brewer.pal(3,"RdYlBu"))
plot(practical_superiority)
Drafting interpretations
Once you have a visualisation of the labelled interpretation_set in front of you, you can start to establish the interpretations that would apply to each option.
Values for placeholders can be specified to enable sections of text in interpretations to be replaced automatically. This can be used, for example, to allow names or descriptions of interventions to be passed to confinterpret, so that these can be returned in the final interpretation rather than a generic description. confinterpret uses gsub() with fixed=TRUE to do substitutions for placeholders entries, so values should be selected that will match accordingly (and will not match extra items). Use of a non-alphanumeric character within a placeholder can help to reduce accidental matches.
new_i_s_label <- figure_nums("new-interpretation-set-plot", "Plot of the newly-defined `interpretation_set` object, showing short versions of the drafted interpretations.")
practical_superiority <- interpretation_set(
boundary_names =c("Actual null", "Practical null"),
placeholders = c(comparison_intervention = "$comp",
tested_intervention = "$test"),
interpretations = list(
# A
list(interpretation_short = "Inferior",
interpretation = "$test inferior to $comp",
interpretation_md = "$test **inferior** to $comp"),
# B
list(interpretation_short = "Not practically superior",
interpretation = "$test not practically superior to $comp",
interpretation_md = "$test **not practically superior** to $comp"),
# C
list(interpretation_short = "Inconclusive",
interpretation = paste("Inconclusive: $test not shown to",
"be inferior or superior to $comp"),
interpretation_md = paste("**Inconclusive**: $test not shown to",
"be inferior or superior to $comp")),
# D
list(interpretation_short = "Not practically superior",
interpretation = "$test not practically superior to $comp",
interpretation_md = "$test **not practically superior** to $comp"),
# E
list(interpretation_short = "Inconclusive",
interpretation = paste("Inconclusive: $test not inferior",
"to $comp, but not shown to be",
"practically superior"),
interpretation_md = paste("**Inconclusive**: $test not inferior",
"to $comp, but not shown to be",
"practically superior")),
# F
list(interpretation_short = "Superior",
interpretation = paste("$test superior to $comp, to a",
"practically relevant extent"),
interpretation_md = paste("$test **superior** to $comp, to a",
"practically relevant extent"))))
plot(practical_superiority)
As noted before, the function building the interpretation_set will run it through the validator, so errors may be picked up if they have been introduced.
knitr::kable(tabulate_interpretation_set(practical_superiority,
"Practical superiority interpretations."),
caption = "Practical superiority interpretations.")
Using a new interpretation_set object
The newly-created interpretation_set object can be used as with any of the provided ones, although natrually this will only be via the main confinterpret() function rather than being able to rely on a helper variant, unless you write one of those too.
The following example shows how the new object can be used to interpret and visualise results of a randomised controlled trial. The data provided are from [@vine_effects_2016]. Briefly, in this study, alternative letter variants were tested using a randomised controlled trial (RCT) approach, to see whether the new letter resulted in more participants engaging with a free support service than the existing standard letter. The relative effectiveness of the letters was assessed at two stages, a triage stage ('gateway') and then at engagement with a specialist advisor. These were binary / dichotomous outcomes, and the effect size measure was prevalence difference: the difference between the proportion of participants who engaged with the new letter and the old letter. Confidence intervals for these prevalence differences were calculated at the 95% level. The resulting estimates and confidence intervals were:
estimate_gateway <- c("prevalence difference" = 0.1032195)
ci_gateway <- matrix(c(0.04777727, 0.2064016),
nrow = 1,
dimnames = list("estimate", c("2.5 %", "97.5 %")))
estimate_specialist <- c("prevalence difference" = 0.1270894)
ci_specialist <- matrix(c(0.0296644, 0.1767746),
nrow = 1,
dimnames = list("estimate", c("2.5 %", "97.5 %")))
Suppose, using the data from the study introduced above, that our analysis plan had specified that we needed to see a 5% improvement before declaring that the new letter was of practically significant superiority. The interpretation could be conducted as follows, and plotted as shown in r figure_nums("re-interpret-plot", display = "cite").
re_int_label <- figure_nums("re-interpret-plot", "Plot showing a study result, re-interpreted using the newly-defined `interpretation_set` object.")
specialist_prac_sup <- confinterpret(ci_specialist,
practical_superiority,
boundaries = c(0, 0.05),
comparison_labels =
c(comparison_intervention =
"Existing standard letter",
tested_intervention = "New letter"))
plot(specialist_prac_sup)
As well as the short interpretation shown on the plot, the longer version of the result can be extracted using specialist_prac_sup$interpretation or specialist_prac_sup$interpretation_md, which give, respectively:
r specialist_prac_sup$interpretationr specialist_prac_sup$interpretation_md
As described above, the confinterpret package is provided with interpretations_superiority for the more typical approach to superiority interpretations. Whilst that approach can result in some practically insignificant results being deemed "superior", by enabling clear presentation in terms of the actual effect sizes, the small / insignificant extent to which the effect was superior would be on clear display.
References {-}
jimvine/confinterpret documentation built on May 19, 2019, 10:35 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width=6.5, fig.height=6.5 ) library("confinterpret")
#' Arrange an interpretation_set object into a table, for printing etc. tabulate_interpretation_set <- function(interpretation_set, name) { int_md <- as.data.frame(sapply(interpretation_set$interpretations, "[[", "interpretation_md")) rownames(int_md) <- LETTERS[1 : nrow(int_md)] colnames(int_md) <- name int_md } table_nums <- captioner::captioner(prefix = "Table") figure_nums <- captioner::captioner(prefix = "Figure")
As a framework for interpreting range-based results, one of the stengths of the confinterpret package is that it is extensible. The extensibility is primarily provided in the form of being able to systematically define new interpretation_set objects. The following example is provided to demonstrate how users can make use of this extensibility.
Motivating example: 'Practical superiority'
The concept of a minimum practical difference is used in some forms of analyses, such as typically forming the basis for establishing the non-inferiority margin in non-inferiority tests. (See ?interpret_noninferiority for further details.) In this example we demonstrate the process for establishing a new interpretation_set using this concept in the case where we wish to conduct a superiority test, but to insist on superiority above some threshold before declaring the intervention to be meaingfully superior.
Although this is not a typical approach, and hence not provided by default within the package, it is plausible as an option that a user may wish to conduct. A result that is of statistical significance --- which is equivalent to having a confidence interval that is entirely superior --- may not equate to a practically significant result. A narrow confidence interval that is all superior --- for example 0.1% to 0.3% improvement --- would pass a standard superiority test but might not be practically superior if decision-makers are only interested in improvements of more than, say, 1%. The example is, however, provided to illustrate the process of extending the package, rather than to particularly recommend this design of interpretations.
The process of establishing a new interpretation_set object is helpful because it forces the user to consider, and formally specify, the interpretation that will apply in all eventualities. This is particularly useful if conducted before analysis commences, as it gets the analysts to establish in advance how they will describe the findings if any particular outcome is observed. As one part of a strategy to reduce researcher degrees of freedom [@simmons_false-positive_2011] it can contribute to the enhancement of the credibility of findings and ultimately improve the quality of the published literature.
Whilst an interpretation_set object can be built from scratch in one step with a call to the interpretation_set() function, the following process can make it easier to build the object in stages:
- First, build the object with only identifying labels, not meaningful interpretations.
- Then plot the object to see the range of interpretations that need to be provided, based on the boundaries.
- Draft the interpretations and re-build the object with these in place.
- Re-plot and print the object to check your interpretations are correct.
Building an interpretation_set with identifying labels
To build an interpretation_set object you will first need to consider the boundaries that specify the type of analysis you are undertaking. Having established the boundaries you will be able to identify how many interpretations you need to specify, as follows.
The number of boundaries, $n_b$ specified in an interpretation_set can be thought of as establishing a number of regions, $n_r$, within which the lower and upper confidence limits can sit. This is given by:
$$n_r = n_b + 1$$
since each boundary can be thought of as having an associated region above it (running up to the next boundary, or in the case of the top boundary, running to infinity), and in addition there is a region for below the bottom boundary. The number of regions in turn determines the number of interpretations. The valid combinations are those where the upper confidence limit is in a region greater than or equal to the region of the lower confidence limit. Consequently, the number of valid combinations is given by:
$$sum(1 : n_r)$$
which equals:
$$n_r (n_r + 1) / 2$$
An interpretation needs to be provided for each of these combinations. For likely (i.e., small) numbers of boundaries, the numbers of regions and interpretations are as shown in r table_nums(name = "numbers_b_r_i", display = "cite").
number_boundaries <- 1 : 5 number_regions <- number_boundaries + 1L number_interpretations <- as.integer(number_regions * (number_regions + 1L) / 2) numbers <- data.frame(number_boundaries, number_regions, number_interpretations, row.names = NULL) colnames(numbers) <- c("Number of boundaries, $n_b$", "Number of regions, $n_r$", "Number of interpretations") knitr::kable(numbers, caption = table_nums(name = "numbers_b_r_i", caption = "Number of regions and interpretations for each number of boundaries."))
Since the interpretation_set() function calls a validator function, if you make an error in setting up the correct number of interpretations it will be caught and an error reported.
The order in which interpretations should be provided is well-defined. The order is based on first specifying all the cases where the lower confidence limit is in the bottom region, and each of the regions for the upper confidence limit (again, starting from the bottom and increasing to the top); next come all of the cases where the lower confidence limit is in the second-from-bottom region (in this case the valid regions for the upper confidence limit will start at the second-from-bottom and go up to the top region); and so on. So for a 2 region (1 boundary) situation, the interpretations should be provided in the order in r table_nums(name = "order_2", display = "cite").
region_orders <- function(number_regions) { number_interpretations <- number_regions * (number_regions + 1) / 2 lower_regions <- rep(1 : number_regions, number_regions : 1) upper_regions <- unlist(mapply(function(x) { x : number_regions }, 1 : number_regions)) regions <- data.frame(1 : number_interpretations, paste("Region", lower_regions), paste("Region", upper_regions)) colnames(regions) <- c("Order", "Lower confidence level", "Upper confidence level") rownames(regions) <- LETTERS[1 : number_interpretations] return(regions) } boundary_2 <- region_orders(2) knitr::kable(boundary_2, caption = table_nums(name = "order_2", caption = "Order of interpretations with two regions."))
For a 3 region (2 boundary) situation, the interpretations should be provided in the order in r table_nums(name = "order_3", display = "cite").
boundary_3 <- region_orders(3) knitr::kable(boundary_3, caption = table_nums(name = "order_3", caption = "Order of interpretations with three regions."))
Rather than trying to match interpretations to these descriptions from scratch, an easier approach may be to instead first create and plot a dummy version of your interpretation_set object with the progression of letters through the alphabet (r figure_nums("dummy-interpretation-set-plot", display = "cite")).
dummy_i_s_label <- figure_nums("dummy-interpretation-set-plot", "Plot of a 'dummy' `interpretation_set` object, providing visual support for use while drafting interpretations.")
practical_superiority <- interpretation_set( boundary_names =c("Actual null", "Practical null"), placeholders = c(comparison_intervention = "$comp", tested_intervention = "$test"), interpretations = list( # A list(interpretation_short = "A", interpretation = "A", interpretation_md = "A"), # B list(interpretation_short = "B", interpretation = "B", interpretation_md = "B"), # C list(interpretation_short = "C", interpretation = "C", interpretation_md = "C"), # D list(interpretation_short = "D", interpretation = "D", interpretation_md = "D"), # E list(interpretation_short = "E", interpretation = "E", interpretation_md = "E"), # F list(interpretation_short = "F", interpretation = "F", interpretation_md = "F"))) # Set a nice colour scheme grDevices::palette(RColorBrewer::brewer.pal(3,"RdYlBu")) plot(practical_superiority)
Drafting interpretations
Once you have a visualisation of the labelled interpretation_set in front of you, you can start to establish the interpretations that would apply to each option.
Values for placeholders can be specified to enable sections of text in interpretations to be replaced automatically. This can be used, for example, to allow names or descriptions of interventions to be passed to confinterpret, so that these can be returned in the final interpretation rather than a generic description. confinterpret uses gsub() with fixed=TRUE to do substitutions for placeholders entries, so values should be selected that will match accordingly (and will not match extra items). Use of a non-alphanumeric character within a placeholder can help to reduce accidental matches.
new_i_s_label <- figure_nums("new-interpretation-set-plot", "Plot of the newly-defined `interpretation_set` object, showing short versions of the drafted interpretations.")
practical_superiority <- interpretation_set( boundary_names =c("Actual null", "Practical null"), placeholders = c(comparison_intervention = "$comp", tested_intervention = "$test"), interpretations = list( # A list(interpretation_short = "Inferior", interpretation = "$test inferior to $comp", interpretation_md = "$test **inferior** to $comp"), # B list(interpretation_short = "Not practically superior", interpretation = "$test not practically superior to $comp", interpretation_md = "$test **not practically superior** to $comp"), # C list(interpretation_short = "Inconclusive", interpretation = paste("Inconclusive: $test not shown to", "be inferior or superior to $comp"), interpretation_md = paste("**Inconclusive**: $test not shown to", "be inferior or superior to $comp")), # D list(interpretation_short = "Not practically superior", interpretation = "$test not practically superior to $comp", interpretation_md = "$test **not practically superior** to $comp"), # E list(interpretation_short = "Inconclusive", interpretation = paste("Inconclusive: $test not inferior", "to $comp, but not shown to be", "practically superior"), interpretation_md = paste("**Inconclusive**: $test not inferior", "to $comp, but not shown to be", "practically superior")), # F list(interpretation_short = "Superior", interpretation = paste("$test superior to $comp, to a", "practically relevant extent"), interpretation_md = paste("$test **superior** to $comp, to a", "practically relevant extent")))) plot(practical_superiority)
As noted before, the function building the interpretation_set will run it through the validator, so errors may be picked up if they have been introduced.
knitr::kable(tabulate_interpretation_set(practical_superiority, "Practical superiority interpretations."), caption = "Practical superiority interpretations.")
Using a new interpretation_set object
The newly-created interpretation_set object can be used as with any of the provided ones, although natrually this will only be via the main confinterpret() function rather than being able to rely on a helper variant, unless you write one of those too.
The following example shows how the new object can be used to interpret and visualise results of a randomised controlled trial. The data provided are from [@vine_effects_2016]. Briefly, in this study, alternative letter variants were tested using a randomised controlled trial (RCT) approach, to see whether the new letter resulted in more participants engaging with a free support service than the existing standard letter. The relative effectiveness of the letters was assessed at two stages, a triage stage ('gateway') and then at engagement with a specialist advisor. These were binary / dichotomous outcomes, and the effect size measure was prevalence difference: the difference between the proportion of participants who engaged with the new letter and the old letter. Confidence intervals for these prevalence differences were calculated at the 95% level. The resulting estimates and confidence intervals were:
estimate_gateway <- c("prevalence difference" = 0.1032195) ci_gateway <- matrix(c(0.04777727, 0.2064016), nrow = 1, dimnames = list("estimate", c("2.5 %", "97.5 %"))) estimate_specialist <- c("prevalence difference" = 0.1270894) ci_specialist <- matrix(c(0.0296644, 0.1767746), nrow = 1, dimnames = list("estimate", c("2.5 %", "97.5 %")))
Suppose, using the data from the study introduced above, that our analysis plan had specified that we needed to see a 5% improvement before declaring that the new letter was of practically significant superiority. The interpretation could be conducted as follows, and plotted as shown in r figure_nums("re-interpret-plot", display = "cite").
re_int_label <- figure_nums("re-interpret-plot", "Plot showing a study result, re-interpreted using the newly-defined `interpretation_set` object.")
specialist_prac_sup <- confinterpret(ci_specialist, practical_superiority, boundaries = c(0, 0.05), comparison_labels = c(comparison_intervention = "Existing standard letter", tested_intervention = "New letter")) plot(specialist_prac_sup)
As well as the short interpretation shown on the plot, the longer version of the result can be extracted using specialist_prac_sup$interpretation or specialist_prac_sup$interpretation_md, which give, respectively:
r specialist_prac_sup$interpretationr specialist_prac_sup$interpretation_md
As described above, the confinterpret package is provided with interpretations_superiority for the more typical approach to superiority interpretations. Whilst that approach can result in some practically insignificant results being deemed "superior", by enabling clear presentation in terms of the actual effect sizes, the small / insignificant extent to which the effect was superior would be on clear display.
References {-}
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.