In thomasgstewart/ahsqc: Functions Specific to AHSQC Operations

## set echo to TRUE if want to include code-folding
knitr::opts_chunk$set(echo = FALSE, message = FALSE, warning = FALSE)
knitr::opts_chunk$set(error = FALSE)

library(ahsqc)
library(tgsify)
library(Hmisc)
library(rms)
library(kableExtra)
library(knitr)
library(boot)
library(data.table)
library(tidyr)
library(tableone)
library(here)

Project Proposal

Executive Summary

Study Population: Inclusion/Exclusion Criteria

Statistical Analysis Plan

Unadjusted Analysis

List of Abbreviations

| Abbreviation | Description | |--------------|---------------------------------------------------------------------------------------------------------------------------------| | FE | Fisher’s two-sided exact test | | EP | E Pearson’s chi-square test also known as the (N-1) chi-square, the randomization chi-square, or the Mantel-Haenszel chi-square | | WR | Wilcoxon rank sum test | | cata | When responding to survey prompt, surgeons were asked to check all options that apply

FAQ

Q. I have a large (often >0.05) p-value. Now what?

A. This is a friendly reminder that absence of evidence does not mean evidence of absence. A large p-value simply means that we don't have a large enough sample size to accurately quantify the effect. In other words, when the null hypothesis is not rejected (p > 0.05), it implies that either the null hypothesis is actually true or that it is actually false but that there was not enough evidence observed to reject it. Confidence intervals can convey information about magnitude, direction, and accuracy of effects, however. The confidence interval will tell you how big the effect could be, if there is an effect. For example, let's consider a confidence interval for an odds ratio, (0.5, 2). The interval includes the odds ratio of 1 (no effect) and the associated p-value would not be statistically significant. However, the interval contains other plausible estimates given the data and tells us that the odds ratio could be as large as 2 or as small as 0.5 is there is an effect. Therefore, the confidence interval contains both "no effect" (OR = 1) and a potentially clinically important effect (OR = 2, 0.5). For these reasons, the conclusion of "effect or no effect" can't be made and is inconclusive.

Q. Why do I have an odds ratio for a continuous outcome?

A. Non-parametric p-values from the Wilcoxon rank sum test for continuous outcomes are equivalent to p-values you would obtain from a cumulative probability model with logit link, or the proportional odds logistic regression model, with no covariates aside from the predictor of interest. The natural estimate and confidence interval to pair with these p-values are odds ratios obtained from this same model.

Q. What is validation and the optimism corrected statistics? A. When a model is fit to a data set, it will often predict that dataset better than a new one, and therefore statistics and predictions generated from the model are considered "optimistic." There are some techniques to correct for this optimism, and the values obtained from those methods are referred to as "optimism corrected" values. If you're interested in a deeper explanation, I think this blog post explains the idea fairly well.

Q. What are Brier scores? A. The Brier score measures the accuracy of predictions or the calibration of the predictions in binary or categorical outcomes. It measures the mean squared difference between the predicted probability and the observed outcome. The score ranges from 0 to 1 and a Brier score closer to 0 shows better calibration.

Q. What is a g-index (g)? A. It’s predictive discrimination measure based on Gini’s mean difference. This means it’s based on the typical difference in the prediction on the original scale that the regression model produces. A good statistical model will have a wide range of predictions (whereas a not-so-good one might have predictions that are very similar and hard to discriminate). The greater the g-index is, more discrimination you have. Intuitively, the g-index can be thought of as "If I pick two patients at random, what would the expected difference in their prediction be?"

For time-to-event outcomes: A good G-index for a time-to-event outcome would be a large number - there isn't a clear "base" value, just that we want the average predictions to be "far apart". For example, suppose a discrimination g-index (g) = 0.4. This would mean that the average difference in the log relative hazard is 0.4, or the average difference in the hazard ratio is 1.5.

For binary and ordinal outcomes: g-index represent "typical" log odds ratios. If you were to choose two patients at random, what would the typical log odds ratio be? gr would be on the odds ratio scale.

For continuous outcomes: g-index represents the typical difference in the predicted value. If you were to choose two patients at random, what would their typical difference in predicted value be?

g is on the original scale gr is on the ratio scale (e.g. odds/hazard) gp is on the probability or risk scale

Q. What is DXY? A. Dxy, also called Somers' Dxy, is a statistical index to quantify model discrimination ability and it ranges from -1 to 1. There are a lot of ways to calculate and describe this measure, so here's my shot at it. One way to describe it is a rank measure that only measures how well predicted values can rank-order responses. Another way to describe it is the difference between the number of concordant pairs and the number of discordant pairs divided by the total number of pairs not tied on the independent variable. When the outcome is binary, Dxy = 2 * (c-index - 0.5). When Dxy = 0, the model is making random predictions and when Dxy = 1, the predictions are perfectly discriminating. When the outcome is time-to-event, Dxy is the rank correlation between the predicted log relative hazard and observed survival time (you can think of this as the rank version of an R^2). According to Dr. Frank Harrell, there are no acceptable values; it's all relative to what you are trying to do (I think he's suggesting that if you have something that's hard to predict - like death - we would be okay with lower values of Dxy). For example, a Dxy value = 0.5 means that 75% of pairs are concordant and 25% of pairs are discordant. If Dxy = 0.2, that would mean that 60% of pairs are concordant and 40% of pairs are discordant.

Q. How do you interpret coefficients from a cumulative probability model with logit link, or a proportional odds logistic regression model in the odds-ratio scale?

A. The adjusted estimate is the odds ratio of the outcome $\geq$ j verus $<$ j for heavy weight compared to medium weight, for any j. Loosely, it's the odds of having a higher outcome (e.g. QoL score) for heavy weight compared to medium weight mesh.

For PROMIS T-score, a coefficient (odds ratio) less than one will show "favor" for heavy weight mesh as the odds of a higher QoL score (i.e. less favorable score) are lower. This is also true for length of stay. The opposite is true for HerQLes - a coefficient greater than one will favor heavy weight weight mesh as the odds of a higher QoL score (i.e. more favorable score) are higher.

Q. What is a restricted cubic splines? What are knots?

A. Restricted cubic splines are a way of modeling continuous variables that may not be linear and allow flexibility in the relationship between the variable and outcome. Knots are the locations where these flexibile curves can change direction.