Nothing
`rddapp` `shiny` manual"
In rddapp: Regression Discontinuity Design Application
rddapp is an R package with a shiny web app for the analysis of regression discontinuity designs (RDDs). It offers both parametric and non-parametric estimation of causal effects in RDDs, with one or two assignment variables. It also provides numerous assumption checks, and allows the estimation of statistical power.
Note: If any of the equations or mathematical symbols do not align properly on your web page, right click on the disrupted text and select "Math Settings" in the dropdown menu. Then, navigate to the "Math Renderer" option and select "Common HTML".
Use
Online
The rddapp shiny web app can be accessed online. Untested features will first be introduced to the beta version.
The shiny environment is fully point-and-click; no programming in R is required. Data can be uploaded directly to the shiny server for analysis. For sensitive data that must be kept on secure, the use of an offline client is available.
Offline
To use the shiny app offline, R must be installed. Next, open an R session and download the offical rddapp R package from CRAN. Load the package using the library(rddapp) command in the R prompt, and then execute the function shiny_run(). This command will load the point-and-click shiny environment in a local browser. No further interaction with the R console is required and no data will be transferred over the Internet.
Menu
At the very top of the shiny app are three menu options, labeled Model, Power, and More. The Model page is for statistical estimation, the Power page is for power analysis, and the More menu contains this manual, as well as a simple About page.
Model page {#model_page}
The Model page is subdivided into a data panel and a dynamic results panel which appears after the data are loaded.
Data upload panel
Data are uploaded using the pull-down menu in the Data panel. The choices are to upload either a .sav file (native file format of SPSS), or a .csv file (comma separated values).
The user can further fine-tune the data upload by clicking on the small gear button at the top right of the panel to reveal additional options depending on the file format chosen.
The Browse button appears automatically when the user chooses a .sav or .csv file and opens a file explorer from which to select the data for upload.
Once the data are loaded, additional panels will appear for the Outcome variable and Treatment design.
.sav files {#sav_files}
If the user selects an .sav file for upload, the gear button in the top right of the Data panel can be used to display further options.
If the Use Value Labels option is selected, value labels from SPSS will be used for the uploaded data. For instance, a user may have coded a categorical variable 0 and 1, and included corresponding value labels male and female. When Use Value Labels is activated, the actual labels will be displayed; otherwise, the numeric codes will be used.
If the Use User's Missing Value option is selected, any user-specified missing values in SPSS will be treated as NA in the R program.
Finally, the user can indicate that the uploaded file has been multiply imputed (typically to deal with missing values) by selecting a Multiple Imputation ID variable from the data. This variable indicates which imputation event each observation corresponds to. All analyses will be performed on each imputation, estimates from each imputation will be averaged, and standard errors will be derived using Rubin's rules (1987), which accounts for the variation both within and between imputations.
.csv files
If the user selects a .csv file, the options under the gear button allow the user to specify exactly how the .csv file is structured.
The user can specify:
- whether any values are enclosed in quotes (double
", single ', or none),
- what kind of separator is used between values (comma
,, semi-colon ;, tab, or whitespace),
- whether missing values are blank by default or if a special character is used (e.g.,
NA),
- whether the file includes a header in the first line with variable names.
Just as for .sav files, the user can specify that the file is multiply imputed, and if so which variable is the Multiple Imputation ID (see above for details).
CARE example data
The application includes an example data set, named CARE. The data are sampled from the Carolina Abecedarian Project and the Carolina Approach to Responsive Education (CARE) study, with the full data set hosted by ICPSR. The example data are described briefly in the rddapp package manual, and in detail in the vignette.
Outcome panel {#outcome_panel}
The Outcome menu allows the user to define the outcome variable: the variable that the researcher believes is affected by the treatment. Additional options for analyzing the outcome variable appear by clicking the gear button and include the Kernel for local linear fitting, Type of SE (standard error), and Auxiliary Variables.
The kernel option specifies the type of kernel that should be used for the non-parametric (local linear fitting) analysis, described further in the Estimates section. By default, a triangular kernel is chosen, in line with recommendations of Imbens and Kalyanaraman [-@ImbenKalya09; -@ImbenKalya12]. However, the user can change this to a variety of other kernels, defined by the following functions:
- Triangular: $K(u) = (1 - |u|)$,
- Rectangular (uniform): $K(u) = \frac{1}{2}$,
- Epanechnikov: $K(u) = \frac{3}{4} (1 - u^2)$,
- Quartic (biweight): $K(u) = \frac{15}{16} (1 - u^2)^2$,
- Triweight: $K(u) = \frac{35}{32}(1 - u^2)^3$,
- Tricube: $K(u) = \frac{70}{81}(1 - |u|^3)^3$,
- Gaussian: $K(u) = \frac{1}{\sqrt{2\pi}} e^{-u^2/2}$,
- Cosine: $K(u) = \frac{\pi}{4}\cos(\frac{\pi}{2}u)$,
where $|u|\leq 1$, except for with the Gaussian kernel, where $u$ can be any real number.
The second option under the outcome tab is the type of standard error (SE) that should be used. rddapp supports a "regular" standard error, that assumes constant variance ($\sigma^2 (X'X)^{-1}$), as well as all of the types of standard errors that are included in the sandwich package (Zeileis, 2017). By default, a heteroskedasticty-robust standard error (HC1) is chosen. If the user chooses the "One-way Cluster" option, an additional pull-down menu appears, prompting the user to select the cluster ID variable.
The auxiliary variables option allows users to specify the use of auxiliary variables in the RDD. These variables are integrated in the regressions of the parametric RDD. All auxiliary variables must be numeric, including ordinal and nominal variables (e.g., factors). Non-numeric variables should be manually recoded into dummy-indicator variables prior to performing the analysis.
Purely linear terms are used for all auxiliary variables; there is currently no implementation for adding non-linear terms. To add non-linear terms, a user would have to create them by hand first (e.g., polynomials) and enter them as regular auxiliary variables. By default, the auxiliary variables will not be entered as interactive terms (i.e., interacting with the treatment assignment). In a properly executed RDD, it should not be necessary to include auxiliary variables, but in some instances, including them can increase precision of parameter estimates, and thus statistical power.
For some models, in particular the frontier approach for multiple assignment variables, an additional option will be displayed that enables the user to define the number of bootstrap samples that should be performed for estimation of standard errors and confidence intervals.
Treatment design panel {#treatment_design_panel}
The treatment mechanism of the RDD is a (series of) rules used to determine whether an individual is selected for treatment, but it does not necessarily imply that the individual actually received treatment. In a sharp RDD, all individuals that were assigned treatment actual received treatment. In a fuzzy RDD, not all individuals that were assigned the treatment received it and/or some individuals who were not assigned the treatment received it.
The mechanism is described by (a series) of "if" statements, in the Treatment design panel, under the heading Assign treatment if. The user selects the assignment variable in the drop-down menu labeled primary assignment. The box underneath is used to input the threshold for being assigned to the treatment.
By default, every value of the assignment variable that is greater than or equal to the threshold leads to an assignment to the treatment condition. Clicking on the $\geq$ sign switches the condition to less than or equal to ($\leq$), meaning that values of the assignment variable less than or equal to the threshold lead to assignment. To exclude treatment at the threshold level, click the gear button and deselect the Treatment at the cutoff(s) option. This will convert the inequality to a strict greater than symbol ($>$) or less than symbol ($<$).
For example, choosing an assignment variable and then selecting greater than or equal to ($\geq$) with treatment threshold value 40, would mean that treatment was assigned to those individuals whose assignment variable was equal to or greater than 40. On the other hand, choosing strictly less than ($<$) a value of 10 would mean that individuals whose assignment variable was strictly less than 10 were assigned to the treatment.
Clicking the plus button in the top right corner of the Treatment design panel allows the user to select a secondary assignment variable, to accommodate multiple-assignment RDDs. Currently, the software only supports up to two assignment variables. One restriction that is in place for these designs is that both assignment rules must be either strict inequalities or not (i.e., either both $>$ and/or $<$ or both $\geq$ and/or $\leq$). Combinations of inequalities where one is strict but the other is not can lead to designs in which it is at least theoretically possible to have undefined cases for some analytic approaches.
Under the Treatment Receipt heading, the user selects the treatment variable in pull-down menu labeled actual treatment. The treatment variable indicates whether the individual actually received treatment.
Data analysis tab
The Data tab displays a preview of the data upon upload, mainly for the purposes of validating that the data were read into the program correctly, in the Raw data panel. Variable names are displayed as table column headers. Users can sort observations by each of the variables by either clicking the up arrow next to the variable name for ascending order or the down arrow for descending order.
Upon specifying the Outcome variable and the Treatment design, a Model Data button appears at the top right corner of the Raw data panel. Clicking this button restricts the preview table to just the outcome and treatment design variables.
Specifying the Outcome variable and the Treatment design also generates two new result tables.
Table 1.1
Table 1.1 shows descriptive statistics of the treatment variable (T), the assignment variable (A1 - and A2, if a second assignment was chosen), and the outcome variable (O). For each variable, three columns are reported: the sample size (N), the mean (M), and the standard deviation (SD). In addition, a correlation matrix is shown on the side, whose columns are treatment (T) and outcome (O), and whose values are the Pearson correlation coefficient between variables. Clicking on the notepad icon in the top right corner of the Table 1.1 panel downloads the data as a .csv file.
Table 1.2
Table 1.2 summarizes the design of the RDD, and can serve to validate that the design was correctly specified in the Treatment design panel. Entries of this table are sample sizes, n. For an RDD with a single assignment variable, rows are the two different groups given by the treatment design (i.e., having the assignment variable below or above the specified threshold). The columns specify whether the treatment was actually received within each group, based on the specified treatment receipt variable.
In a sharp RDD (a design in which assigned and received treatment always coincide), the off-diagonal cell counts are always zero. In a fuzzy RDD, the off-diagonal elements will contain sample sizes of incorrectly classified individuals (i.e., treatment assignment and received treatment do not coincide). In this case, the probability of receiving treatment, $\pi$, conditional on being on either side of the threshold is also displayed. This quantity is computed as a simple conditional probability, $P(T=1|A1)$; no parametric model is being used for this estimate.
If a second assignment variable is specified, an additional column is added for this variable. All combinations of treatment thresholds are displayed as single rows, with their respective sample sizes, n, and the estimated $\pi$ parameter.
rddapp reports further details of the treatment design under the design summary table. It determines whether the design is sharp or fuzzy, and whether one or two assignment variables were used. If the second assignment variable does not alter any of the treatments assigned based on the first assignment variable, the software notes that the second assignment variable is redundant, and calls this an ineffective assignment.
Assumptions tab
RDDs rely on several assumptions, which are summarized in the Assumptions tab. Depending on whether the design is sharp or fuzzy, and relies on one or two assignment variables, the content of the assumption tab will vary slightly.
Figure 2.1
Figure 2.1 shows the result of McCrary's (2008) sorting test, which relies on the following idea: in a proper RDD, there should be no discontinuity at the threshold for treatment if participants were neutrally selected based on the assignment rule. The presence of a discontinuity could imply that there was self-selection for or against the treatment by some participants or others involved.
As an example, suppose that students are put on the dean's list if their GPA is 3.75 or higher. Teachers who (either implicitly or explicitly) want to help their students may round up a grade that is very close to the treatment threshold. A teacher may grade the final exam of a student with a GPA of 3.74 more leniently, so that the student can obtain a final GPA of 3.75 and make it on the dean's list. If such behavior was wide-spread, we would find fewer scores of 3.74 and just below, and more scores of 3.75 and just above. This effect would manifest itself as a small discontinuity in the distribution of GPA, right at the treatment threshold.
McCrary's sorting test
The sorting test is implemented as described by McCrary (2008). The user specifies the assignment variable and threshold of interest in the pull-down menu labeled Assignment [Cutoff]. The data are aggregated with a default bin size and a non-parametric kernel smoother is applied to both sides of the treatment threshold using the midpoints of bins and the observed count in each bin as data. The test measures the vertical distance of the two edges on both sides of the treatment threshold with a log metric.
Because the sorting test relies on a non-parametric estimation of the kernel density estimate, it is possible to change parameters of the test. Any changes to the parameters automatically update both the plot and the test summary.
Bin size
The bin size determines the width of bins for the kernel density estimate in the sorting test, which is performed on the means of each bin of raw data. As the bin size decreases, the number of bins with frequency at or close to 1 will increase, and so the density estimate will look very flat (near 0). As the bin size increases, the number of bins decreases, and the kernel density estimate will tend to be under-smoothed if it relies on too few bins.
Band width
The bandwidth, also known as the smoothing parameter, determines how many data points are considered at once when computing the smooth approximation to the data. As the bandwidth increases, more data points are considered and so the smoother goes from flat to jagged. The choice of bandwidth will affect the test statistic, and possibly different inferential decisions. Generally speaking, a bandwidth parameter should be chosen so that no over- or under-smoothing occurs. To aid in the selection of the bandwidth parameter, Figure 2.1 will update every time the smoothing parameter is changed.
When changing the parameters, be mindful that it it is considered bad statistical practice to change the bin size or bandwidth to maximize the p-value in order to make the test look good. Instead either the default bin size should be used, or adjusted in accordance with a properly smoothed plot. The software defaults are often good choices, but sometimes minor adjustments are needed.
Test summary {#test_summary}
The software derives a test statistic, theta, which quantifies the vertical distance, and a standard error (se). These quantities are tested using a $z$-test. The z value and the p value are also reported. Ideally, this test should not be significant. A significant test statistic would indicate that the assumption of a neutrally implemented RDD (assignment only due to treatment threshold) has been violated. The test summary is displayed in the lower right corner of the Figure 2.1 panel.
Both the plot and the test summary can be downloaded using the buttons in the top right corner of the Figure 2.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
Table 2.1
Table 2.1 displays the results of an attrition analysis. In each row, it provides a count, N, of missing data and corresponding percentages of the total sample size for each variable (treatment, outcome, and assignment). The three columns give these counts and percentages:
- for all individuals (Overall);
- restricted to individuals in the control group (Control);
- restricted to individuals who received treatment (Treatment).
This information can help to determine whether any differential attrition occurred. The table can be downloaded as a .csv file using the button in the top right corner of the Table 2.1 panel.
Estimates tab {#estimates_tab}
The Estimates tab is where the user will find point estimates and inferential statistics for the treatment effect. We begin by discussing the basic estimates interface in detail in the section on RDD with one assignment rule. We then discuss specifics for the multivariate case in the RDD with two assignment rule section. We recommend that users read through the single variable section before moving on to the section on multivariate analysis.
RDD with one assignment rule {#rdd_one_asst}
Table 3.1
Table 3.1 contains all results, treatment effect estimates, and inferential statistics for both the parametric and non-parametric models. The user can expand or reduce the amount of information that is being displayed using several toggle switches at the top right of the table, including those for Parametric (global) and Non-parametric (local) models.
Parametric models {#parametric_models}
The parametric model results are subdivided into three rows, where each row gives results of the outcome variable modeled as a function of the assignment variable, but with different functional forms in each case. The available functional forms are:
- linear, which only has a linear term;
- quadratic, which in addition to the linear term from above also has a quadratic term;
- cubic, which in addition to the linear and quadratic terms from above also has a cubic term.
Each term listed above has an associated regression coefficient. Moreover, these coefficients are allowed to differ before and after the assignment threshold. As a result, the model always includes an interaction effect between the assignment variable and the treatment status.
By default, all parametric models used in the estimation of the treatment effect are performed using two-stage least-squares (2SLS) estimation, with robust standard errors. The first stage involves a model that uses the assignment variable to predict the treatment variable. Note that in sharp RDDs, this prediction of treatment assignment will always be perfect by definition, which renders this first step unnecessary; the whole process reduces down to a single-stage least-squares regression model. In the second stage, the predicted values from the first stage are used as predictors of the outcome variable. The treatment effect is captured in the regression coefficient that is associated with the treatment variable in the second stage of estimation.
For each parametric model, several inferential statistics are reported in labeled columns:
- N: the total sample size;
- Est.: the treatment effect estimate;
- SE: the standard error associated with the treatment effect estimate;
- z: the z-score associated with the treatment effect estimate;
- p: the p-value for the estimate;
- 95% CI: the 95% confidence interval for the treatment effect estimate;
- Cohen's d: the Cohen's d effect size, which is computed as the mean difference at the treatment threshold divided by the standard deviation of all observed outcome values (same across all models to make direct comparisons of Cohen's d meaningful).
Non-parametric models
The non-parametric models use a weighted least-squares linear regression. A bandwidth function determines how to weigh each data point in the regression, which determines both the smoothness of the regression, as well as the window around the treatment threshold from which to use data (points are excluded by being assigning a weight of 0).
The non-parametric model results are subdivided into three rows, where each row gives results of the outcome variable modeled as a function of the assignment variable, but with a different bandwidth function. The bandwidth functions include:
- optimal: the "optimal" bandwidth, as defined by Imbens and Kalyanaraman [-@ImbenKalya09; -@ImbenKalya12];
- half-optimal: half the optimal bandwidth;
- double-optimal: double the optimal bandwidth.
Note that, unlike parametric models, these non-parametric models differ in the sample size being used in each regression (since the bandwidth function can assign 0 weight to some points).
The three different non-parametric models are run by default to allow the researcher to see whether results hold under a variety of bandwidths. Additional robustness checks are performed in the sensitivities tab.
In the case of a fuzzy RDD, the user may also request the intent-to-treat estimate, using the ITT toggle switch in the top right of the Table 3.1 panel. This estimate ignores the fuzziness of the design, and simply assigns each individual the treatment that they were supposed to receive based on the treatment assignment rule. Note that ITT model is only useful for fuzzy designs since in sharp designs, treatment assignment and actual treatment always coincide.
If any covariates are specified in the auxilliary variables field found in the additional options of the outcome panel, an additional toggle button labeled COV is shown at the top right of the table. It is deselected by default, but if the user selects it, additional rows are added to Table 3.1 specifying the regression coefficient that describes the (linear) relationship between the covariate and the outcome is displayed, along with all inferential statistics. Note that even though in the table the regression coefficient is displayed under the same column heading (Est.), it does not mean that this is a treatment effect estimate. It is simply the linear relationship between covariate and the outcome variable. The regression coefficient for the covariate is obtained from the same model that is used to estimate the RDD treatment effect. That means, the treatment and assignment variables are also predictors in this model. It is currently not possible to model the relationship between the covariate and the outcome variable in a non-linear fashion (e.g., polynomial regression, interacted with the treatment, etc.).
The table can be downloaded as a .csv file using the button in the top right corner of the Table 3.1 panel.
Figure 3.1
Figure 3.1 provides a visual display of the treatment effect summarized in Table 3.1, typically through some combination of a scatterplot of the assignment vs. outcome variables along with a plot of one or more fitted models and their associated confidence intervals over top.
The user can customize this plot in three different ways: by changing the model type, predicted lines, and data points.
Model type
The model type option is only useful for fuzzy designs as there is no difference between the compiler average and the intention-to-treat options for sharp design; in these cases, treatment assignment and actual treatment always coincide.
Predicted lines {#predicted_lines}
The user can toggle which of the models summarized in Table 3.1 are plotted as predicted lines in Figure 3.1 by selecting their labels from the drop-down menu or by deleting their labels from this field by clicking the label and hitting the backspace key on the keyboard. By default, the non-parametric optimal model is overlaid.
Furthermore, the user can specify a percentage for the confidence interval(s) plotted in the CI input box, and toggle whether or not the area between the confidence interval boundaries should be colored in (by selecting the fill area button).
Data points {#data_points}
The user can choose whether to scatterplot all data points (with the raw option), or to bin them evenly or by quartile. The latter option may be preferred if there are many data points to prevent over-plotting. In each bin, an average of the outcome variable is computed and plotted as a single point.
Binning evenly means that the assignment variable is sub-divided into bins of equal width, while binning by quantiles uses quantiles of the assignment variable. In either case, the binning algorithm will always avoid spanning a bin over the treatment threshold value, which can sometimes lead to slight distortions in the bin width and sample size. For instance, there are situations in which even binning does not result in bins of equal width, or quantile binning does not result in the same sample size across all bins, as one might expect.
The user can specify the number of bins in the input box labeled bins. The point size can be scaled with respect to the number of points in each bin using the scale point size switch.
A (normal) confidence interval is plotted around the estimate of the mean, at whatever percentage specified in the CI field. The user can toggle whether or not to hide error bars with the eponymous button.
Plot labels (e.g., regression line labels, point legends, axis labels) are generated automatically and cannot be customized by the user at this time.
The plot can be downloaded as a .png, .svg, and a .pdf using the buttons in the top right of the Figure 3.1 panel.
RDD with two assignment rules {#rdd_two_asst}
An RDD with two assignment variables requires a more complicated analysis because several estimates have been suggested in the literature. For an overview, see the paper by Wong, Steiner, and Cook (2013), which describes a variety of designs and estimates. The three estimate approaches implemented in this software are directly derived from the Wong et al. (2013) paper, as well as the Stata code that was published by Wong.
The top right of the Table 3.1 panel features toggle buttons for the three multivariate approaches: Centering, Univariate, and Frontier. For each of these approaches, the user can also the toggle parametric and non-parametric models described in the previous section, and the same results are displayed as previously discussed.
Centering approach {#centering_approach}
In the centering approach, the multiple assignment RDD is transformed into a single assignment RDD through the use of a centering function. In particular, the program centers all assignment variables around the treatment threshold value by subtracting the value of the treatment threshold from each assignment variable; these differences are called deviation scores. For each observation, we take minimum of the two deviation scores calculated from the values of each of the two assignment variables. In designs in which treatment is assigned to units above or below the treatment threshold (or any combination of those), sometimes the maximum is taken.
Following this transformation, each observation has a single (centered) assignment variable (the minimum deviation score), which is then used to compute a single assignment RDD. Note that the treatment effect estimate is sensitive to the metric used for each assignment variable. Wong et al. (2013) show that even standardization cannot typically mitigate this issue, and advise that the centering approach should only be used when the assignment variables share the same metric.
The centering approach computes what Wong et al. call the frontier average treatment effect, which is essentially a treatment effect along the whole treatment threshold of the two-dimensional response (outcome) surface defined by the two assignment variables.
The plot for the centering approach can be displayed in Figure 3.1 by selecting Centering in the Model Type drop-down menu of that panel. The options for Predicted lines and Data points remain the same as in the case of an RDD with a single assignment variable.
Univariate approach {#univariate_approach}
The univariate approach focuses on treatment effects that are exclusively attributable to one of the two assignment variables. Wong et al. (2013) call these the frontier-specific effects. In a design with two assignment variables, two frontier-specific effects will be computed: one for each assignment variable.
To estimate these effects, the observations are divided into subsets based on which rule assigned them to treatment. For example, to compute the frontier-specific effect of the first assignment variable, the only observations used are those assigned to treatment based solely on the first assignment variable. Table 3.1 shows the sample sizes for each subset in the N column.
These subsets of the data for each assignment variable, labeled A1 and A2, are then analyzed using an RDD with one assignment variable.
There is an alternative to the univariate approach, which retains all subjects and then treats the resulting design as a fuzzy RDD with a single assignment variable. Wong et al. (2013) demonstrated that this approach generally has no advantages with respect to increasing statistical power or decreasing bias when compared to simply excluding the mis-classified observations from the analysis.
The plot for the univariate approach can be displayed in Figure 3.1 by selecting Univariate (A1) or Univariate (A2) in the Model Type drop-down menu of that panel. The options for Predicted lines and Data points remain the same as in the case of an RDD with a single assignment variable.
Frontier approach {#frontier_approach}
The frontier approach estimates the treatment effect along the whole discontinuity frontier of the response (outcome) surface. As described by Wong et al. (2013), this approach is performed by first estimating the response surface (e.g., using a parametric model), and then using kernel density estimates for numerical integration of the treatment effect.
The frontier approach yields three different estimates of the average treatment effect:
- the average frontier effect, computed for the whole frontier
- the frontier 1 effect, computed for the frontier along the first assignment variable, A1;
- the frontier 2 effect, computer for the frontier along the second assignment variable, A2.
All of these effects are estimated simultaneously using the full sample.
This method usually results in higher statistical power than other approaches, but requires the assumption that the response surface is correctly modeled by the linear parametric model used in the computation. Non-parametric models are not supported at this time and so its toggle button is inactive in this case.
There are three linear parametric models available: Unconstrained, Heterogeneous effects, and Constant Effect. Note that non-linear parametric models are not supported at this time.
The constant effect model is the most restrictive, as it imposes two big constraints:
- the heights of the discontinuities at the treatment thresholds must be equal across both dimensions given by the assignment variables;
- the regression slope on all parts of the response surface (away from the discontinuity) must be equal.
Despite these constraints, the average frontier treatment effect and the frontier-specific effects are not necessarily identical due to the fact that the raw data may not be identically distributed along the regression surface, and so the kernel density estimate for the numerical integration is not weighted equally on all parts of the frontier.
The heterogeneous effects model relaxes constraint 1 of the constant effect model. In particular, the heights of the discontinuities are allowed to differ in different segments of the frontier. However, the slope of the regression surface must still be equal everywhere (away from the discontinuities).
The fully unconstrained model eschews both constraints 1 and 2, which means that it is the most complex of these linear parametric models. Wong et al. (2013) does not provide closed form (analytic) solutions for standard errors for the frontier treatment effects. We use bootstrapping to compute the inferential statistics in Table 3.1.
In order to perform these computations, the number of bootstrap samples must be specified in the additional options of the Outcome panel (in the Data tab), which is displayed using the gear button in the top right corner of the panel. The input field for the number of bootstrap samples is labeled SE and CI for frontier estimate. Once the number of samples is specified and the user navigates back to the Estimates tab, the computation begins and Table 3.1 is populated when the calculations are complete. Using a large number of bootstrap samples is computationally expensive, and it make take a long time to complete the calculations.
Figure 3.1 displays a three-dimensional plot of the response surface when the Frontier option is selected in the Model type pull-down menu. New pull-down menus appear below, replacing the Predicted lines and Data points options.
The first new menu is labeled Model Specification, where the user can select any of the three linear parametric models implemented for the frontier approach (Unconstrained, Heterogenous effect, Constant effect). These options help the user visualize each model's constraints (or lack thereof).
The remaining new menus enable the user to modify the appearance of the plot. The View pull-down menu has four options: Perspective,which displays a three-dimensional plot of the response surface, as well as Top, Front, and Side, which show various two-dimensional projections of the surface (flattening one dimension). For example, the Top view shows the response surface directly from above, flattening the outcome variable dimension, which can help the user visualize the treatment thresholds.
The three-dimensional perspective plot can be rotated using the sliders labeled with arrows. The graph is regenerated every time these sliders are adjusted, and it takes for the plot to refresh.
The panel buttons allow the user to color all surfaces that received the treatment assignment, add a grid to the response surface, and shade the surface slightly.
Finally, data buttons allow the user to scale the axes to the original scale (by default, the axes are scaled to the same length), and to add the raw data points to the plot.
Fuzzy RDD
In the case of an RDD with multiple assignments that is fuzzy, the design becomes slightly more complicated. However, the overall interface and outputs are identical to the sharp counterpart of this design. All of the models that were described above, are now being estimated using the two-stage least squares approach, in which the fuzzy treatment assignment is estimated in a first step, and then the estimated treatment assignments are used for all of the models.
Sensitivities tab {#sensitivities_tab}
In this tab, the user can perform various sensitivity analyses, including those for the treatment threshold and the bandwidth. Note that neither of Figures 4.1 or 4.2 will be generated automatically; they will only appear after the corresponding blue circular arrow button is pressed (found in the lower right corner of each figure panel).
Figure 4.1
Figure 4.1. performs a sensitivity analysis for the treatment threshold by plotting the treatment effect estimate as a function of a range of hypothetical treatment thresholds for various models.
These tests of treatment effects at wrong treatment threshold values are sometimes also called placebo tests. The assumption is that in a properly-run RDD, the treatment effect should only be visible at the actual treatment threshold. Hence, seeing many non-significant results at treatment thresholds away from the actual treatment threshold value is desirable.
The user can change the span of treatment thresholds to test by defining the endpoints using the min and max boxes below the plot, and the number of estimates to compute between these endpoints using the steps box. The treatment effect at the true treatment threshold is always estimated and denoted with a dashed vertical line labeled designed cutoff. By default, a 95% confidence interval is shown around the estimates, but the user can change this percentage in the CI box at the top right of the Figure 4.1 panel.
The user can select the models with which to perform this analysis (e.g., the parametric linear model, the non-parametric optimal model etc.) in the box directly under the plot. Clicking this box will open a drop-down menu featuring all models that can be added to the plot. To delete a model from the plot, click on its label and press the keyboard's backspace button. The labels in this menu correspond to the ones used in the Estimates tab.
Both the plot and the data being plotted can be downloaded using the buttons in the top right corner of the Figure 4.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
Figure 4.2
Figure 4.2. is a robustness check for the non-parametric model; it shows how the treatment effect estimate changes with different bandwidths. Ideally, the graph should show a somewhat flat line that indicates that over a wide range of bandwidth choices, the estimate does not change dramatically.
The user can change the span of bandwidths to test by defining the endpoints using the min and max boxes below the plot, and the number of estimates to compute between these endpoints using the steps box. The treatment effect at the optimal bandwidth is always estimated and denoted with a dashed vertical line labeled optimal bandwidth. By default, a 95% confidence interval is shown around the estimates, but the user can change this percentage in the CI box at the top right of the Figure 4.2 panel.
Both the plot and the data being plotted can be downloaded using the buttons in the top right corner of the Figure 4.2 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
R code tab
In this tab, the current analysis that the user selected via the drop-down menu is translated into R code. Users can copy and paste this code into an R script for customization, automation, and for conducting reproducible research.
Power page {#power_page}
Users may wish to perform a formal (frequentist) power analysis, to determine a priori sample sizes, or to construct power curves for various sample sizes. All power analyses are performed using Monte Carlo simulations. The user defines population parameters, which are used to simulate a model. The treatment effect is estimated from the simulation and its significance is calculated. This procedure is repeated many times, and a mean and variance are calculated for the treatment effects. The statistical power is estimated as the proportion of significant results across all Monte Carlo simulations.
The success of this process hinges on properly defining the population parameters, which is often challenging. We attempt to guide the user in this process using various graphical displays. These displays change depending on the particular design that is chosen (i.e., whether the RDD has a single or multiple assignment variable(s)).
Assignment panel
The Assignment panel is where the user defines the assumed distribution of the assignment variable. The plot shows a histogram with kernel density estimate. The shading of the histogram bars indicates group membership (treatment or control).
The settings under the Distribution label allow the user to specify the distribution type and parameters for the assignment variable, labeled A1 by default. The distribution types available are normal and uniform, and the user can toggle between them using the eponymous toggle button. For a normal distribution, a mean $\mu$ and a standard deviation $\sigma$ must be provided. For a uniform distribution, a minimum value and a maximum value must be provided.
Under the Treatment design label, the user can set the assignment rule, i.e., the values of the assignment variable A1 at which treatment is administered. The assignment rule changes the proportion of treated and untreated, which affects power estimates. To change the assignment rule, the user can click on the inequality symbol to toggle between greater than or equal to and less than or equal to and change the value of the assignment threshold in the adjacent text box. Note that there is no setting to exclude treatment at the threshold (i.e., to remove the equality from the inequality). If the assignment variable is discrete and the desired treatment assignment is A1 $> x$, then the user could use the equivalent assignment rule A1 $\geq x+1$.
The user may also change the amount of fuzziness in the design, which is defined as the proportion of misclassified cases around the treatment threshold. The user can set this proportion (as a decimal) below and above the threshold independently, using the text boxes adjacent to the fuzziness label. The left box is for the proportion of control cases that were mis-classifieds and instead treated, while the right box is for cases assigned treatment but actually left untreated. Note that the left box is always associated with control cases and the right box is always associated with treatment cases even if the assignment rule flips the plot so that cases under (i.e., to the left of) the threshold are treated.
The sample size can be specified in the box found in the top right of the Assignment panel.
Clicking the plus button in the top right corner of the Assignment panel allows the user to add a second assignment variable (labeled A2). The distribution and treatment design for A2 can be set in the same way as they were for A1. All parameters (including distribution, and fuzzyness) of the second variable can be modified in the same way. The plot will update to show an annotated scatterplot with each assignment variable on one axis, along with a sketch of their distributions.
Parametric model for outcome panel
Once the assignment is specified, the user must specify the parametric model for the outcome variable as a function of the assignment and treatment.
Currently, only linear models are implemented, though there is a fully unconstrained parametric model with respect to interactive effects. In this case, every segment of the regression surface can have a different slope.
The plot in this panel shows the simulated outcome variable as a function of the assignment and will update when simulation parameters are changed.
In the case of an RDD with one assignment variable, a scatterplot with a regression line is shown. The user can modify the treatment effect (labeled Treatment), the intercept of the regression slope (labeled Constant), the slope on the control side of the treatment threshold (labeled Slope), and the difference in slopes on the treatment side of the threshold (labeled Interaction). Lastly, the user can define the Partial $\eta^2$, which measures the effect size of the treatment. As the partial $\eta^2$ increases, the simulated data will lie closer to the regression lines. Defining the effect size is an indirect way of changing the standard error of the treatment effect.
In the case of an RDD with two assignment variables, the parametric model is substantially more complex. It now involves treatment effects at both frontiers, and slopes that are defined for every piece of the regression surface divided by the threshold for each assignment variable.
There are a total of 13 regression coefficients that need to be defined by the user:
- the intercept, i.e., the predicted outcome when both assignment variables at their respective treatment thresholds (labeled constant);
- the two treatment effects (labeled T1 and T2 for the assignment variables A1 and A2 respectively), which set the effect size of the treatment at the threshold of each assignment variable;
- the two regression coefficients that relate the assignment variables to the outcome, while holding treatments constant at the control group (labeled A1 and A2);
- a total of seven interaction effects are included, that define how the slopes of the assignment variables change with different values of the treatment and control values for the two cut-offs.
Table 5.1
Once the parametric models are specified in the Assignment and Parametric Model for Outcome panels, the user can generate Monte Carlo simulations in the Table 5.1 Powers of RDD estimates panel.
The user can specify the number of Monte Carlo simulations to be performed in the field labeled Iterations in the top right corner of the panel. The default number of iterations is 500. In general, as the number of iterations increases, the power estimate becomes more precise; however, a large number of iterations may take a very long time to complete.
The simulations are executed by pressing the blue circular arrow button in the top right corner of the panel. Once the simulations are complete, Table 5.1. is populated with results.
For an RDD with one assignment variable, the rows of the table give the results for the linear parametric model and the non-parametric model with optimal bandwidth.
For an RDD with two assignment variables, the rows of the table give the results for three linear parametric models (centering, and two univariate models), as well as three non-parametric models (same as the parametric ones but with an optimal bandwidth). Other possible effects (e.g., the frontier average effect) are not estimated as they take prohibitively long to compute.
In both cases, the columns of the table are:
- S: the number of successful Monte Carlo simulations; typically, this value will be equal to the number of iterations, but for some usual models (e.g., lots of fuzziness in the assignment), it is possible that some simulations do not yield a valid estimate;
- Treatment effects: the sample Mean and Variance of the treatment effect from the simulations of each model; for a large enough number of iterations, the user-defined treatment effect should be recovered;
- Power: the estimated statistical power (Type II error rates) for various significance levels ($\alpha = .05, .01, .001$).
The tab above the Table 5.1 panel, labeled R Code gives the code used to perform the simulation and populate the table.
Figure 5.1
Figure 5.1. allows the user to investigate how statistical power is affected by sample size. Inside the panel, the user can set the minimum sample size and the maximum sample size, while the number of sample sizes to take in this interval can be set in the Steps box, in the top right corner of the panel.
The number of Iterations in each step can be set in the top right corner of the panel as well. The default number of iterations is 500. In general, as the number of iterations increases, the power estimate becomes more precise; however, a large number of iterations may take a very long time to complete.
The simulations are executed by pressing the blue circular arrow button in the top right corner of the panel. Figure 5.1. is populated with results as the simulations for each sample size are complete.
Once the simulations are fully complete, the user can adjust what is plotted in the figure. The Group lines by drop-down menu reveals two main options:
- Model type: the results for each model are plotted as a different curve;
- Alpha level: the results the three levels of significance ("alpha levels") are plotted as different curves.
Depending on the Group lines by option selected, a second drop-down menu appears below it. If Model type is selected, the menu Show alpha level appears, where the user chooses the significance level for which to plot the estimated power. If Alpha leve is selected, the menu Show model type appears, where the user chooses the model for which to plot the estimated power.
Both the currently displayed plot and the full simulation data can be downloaded using the buttons in the top right corner of the Figure 5.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The simulation data can be downloaded as a .csv file.
About
This software was developed by Felix Thoemmes, Wang Liao, Ze Jin, Wenyu Zhang, Irena Papst, and Kim Hochstedler all at Cornell University. We are happy to receive feedback! Please send it to
The project was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305D150029. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.
Citations
Try the rddapp package in your browser
Any scripts or data that you put into this service are public.
rddapp documentation built on April 6, 2023, 1:15 a.m.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
rddapp is an R package with a shiny web app for the analysis of regression discontinuity designs (RDDs). It offers both parametric and non-parametric estimation of causal effects in RDDs, with one or two assignment variables. It also provides numerous assumption checks, and allows the estimation of statistical power.
Note: If any of the equations or mathematical symbols do not align properly on your web page, right click on the disrupted text and select "Math Settings" in the dropdown menu. Then, navigate to the "Math Renderer" option and select "Common HTML".
Use
Online
The rddapp shiny web app can be accessed online. Untested features will first be introduced to the beta version.
The shiny environment is fully point-and-click; no programming in R is required. Data can be uploaded directly to the shiny server for analysis. For sensitive data that must be kept on secure, the use of an offline client is available.
Offline
To use the shiny app offline, R must be installed. Next, open an R session and download the offical rddapp R package from CRAN. Load the package using the library(rddapp) command in the R prompt, and then execute the function shiny_run(). This command will load the point-and-click shiny environment in a local browser. No further interaction with the R console is required and no data will be transferred over the Internet.
Menu
At the very top of the shiny app are three menu options, labeled Model, Power, and More. The Model page is for statistical estimation, the Power page is for power analysis, and the More menu contains this manual, as well as a simple About page.
Model page {#model_page}
The Model page is subdivided into a data panel and a dynamic results panel which appears after the data are loaded.
Data upload panel
Data are uploaded using the pull-down menu in the Data panel. The choices are to upload either a .sav file (native file format of SPSS), or a .csv file (comma separated values).
The user can further fine-tune the data upload by clicking on the small gear button at the top right of the panel to reveal additional options depending on the file format chosen.
The Browse button appears automatically when the user chooses a .sav or .csv file and opens a file explorer from which to select the data for upload.
Once the data are loaded, additional panels will appear for the Outcome variable and Treatment design.
.sav files {#sav_files}
If the user selects an .sav file for upload, the gear button in the top right of the Data panel can be used to display further options.
If the Use Value Labels option is selected, value labels from SPSS will be used for the uploaded data. For instance, a user may have coded a categorical variable 0 and 1, and included corresponding value labels male and female. When Use Value Labels is activated, the actual labels will be displayed; otherwise, the numeric codes will be used.
If the Use User's Missing Value option is selected, any user-specified missing values in SPSS will be treated as NA in the R program.
Finally, the user can indicate that the uploaded file has been multiply imputed (typically to deal with missing values) by selecting a Multiple Imputation ID variable from the data. This variable indicates which imputation event each observation corresponds to. All analyses will be performed on each imputation, estimates from each imputation will be averaged, and standard errors will be derived using Rubin's rules (1987), which accounts for the variation both within and between imputations.
.csv files
If the user selects a .csv file, the options under the gear button allow the user to specify exactly how the .csv file is structured.
The user can specify:
- whether any values are enclosed in quotes (double
", single', or none), - what kind of separator is used between values (comma
,, semi-colon;, tab, or whitespace), - whether missing values are blank by default or if a special character is used (e.g.,
NA), - whether the file includes a header in the first line with variable names.
Just as for .sav files, the user can specify that the file is multiply imputed, and if so which variable is the Multiple Imputation ID (see above for details).
CARE example data
The application includes an example data set, named CARE. The data are sampled from the Carolina Abecedarian Project and the Carolina Approach to Responsive Education (CARE) study, with the full data set hosted by ICPSR. The example data are described briefly in the rddapp package manual, and in detail in the vignette.
Outcome panel {#outcome_panel}
The Outcome menu allows the user to define the outcome variable: the variable that the researcher believes is affected by the treatment. Additional options for analyzing the outcome variable appear by clicking the gear button and include the Kernel for local linear fitting, Type of SE (standard error), and Auxiliary Variables.
The kernel option specifies the type of kernel that should be used for the non-parametric (local linear fitting) analysis, described further in the Estimates section. By default, a triangular kernel is chosen, in line with recommendations of Imbens and Kalyanaraman [-@ImbenKalya09; -@ImbenKalya12]. However, the user can change this to a variety of other kernels, defined by the following functions:
- Triangular: $K(u) = (1 - |u|)$,
- Rectangular (uniform): $K(u) = \frac{1}{2}$,
- Epanechnikov: $K(u) = \frac{3}{4} (1 - u^2)$,
- Quartic (biweight): $K(u) = \frac{15}{16} (1 - u^2)^2$,
- Triweight: $K(u) = \frac{35}{32}(1 - u^2)^3$,
- Tricube: $K(u) = \frac{70}{81}(1 - |u|^3)^3$,
- Gaussian: $K(u) = \frac{1}{\sqrt{2\pi}} e^{-u^2/2}$,
- Cosine: $K(u) = \frac{\pi}{4}\cos(\frac{\pi}{2}u)$,
where $|u|\leq 1$, except for with the Gaussian kernel, where $u$ can be any real number.
The second option under the outcome tab is the type of standard error (SE) that should be used. rddapp supports a "regular" standard error, that assumes constant variance ($\sigma^2 (X'X)^{-1}$), as well as all of the types of standard errors that are included in the sandwich package (Zeileis, 2017). By default, a heteroskedasticty-robust standard error (HC1) is chosen. If the user chooses the "One-way Cluster" option, an additional pull-down menu appears, prompting the user to select the cluster ID variable.
The auxiliary variables option allows users to specify the use of auxiliary variables in the RDD. These variables are integrated in the regressions of the parametric RDD. All auxiliary variables must be numeric, including ordinal and nominal variables (e.g., factors). Non-numeric variables should be manually recoded into dummy-indicator variables prior to performing the analysis.
Purely linear terms are used for all auxiliary variables; there is currently no implementation for adding non-linear terms. To add non-linear terms, a user would have to create them by hand first (e.g., polynomials) and enter them as regular auxiliary variables. By default, the auxiliary variables will not be entered as interactive terms (i.e., interacting with the treatment assignment). In a properly executed RDD, it should not be necessary to include auxiliary variables, but in some instances, including them can increase precision of parameter estimates, and thus statistical power.
For some models, in particular the frontier approach for multiple assignment variables, an additional option will be displayed that enables the user to define the number of bootstrap samples that should be performed for estimation of standard errors and confidence intervals.
Treatment design panel {#treatment_design_panel}
The treatment mechanism of the RDD is a (series of) rules used to determine whether an individual is selected for treatment, but it does not necessarily imply that the individual actually received treatment. In a sharp RDD, all individuals that were assigned treatment actual received treatment. In a fuzzy RDD, not all individuals that were assigned the treatment received it and/or some individuals who were not assigned the treatment received it.
The mechanism is described by (a series) of "if" statements, in the Treatment design panel, under the heading Assign treatment if. The user selects the assignment variable in the drop-down menu labeled primary assignment. The box underneath is used to input the threshold for being assigned to the treatment. By default, every value of the assignment variable that is greater than or equal to the threshold leads to an assignment to the treatment condition. Clicking on the $\geq$ sign switches the condition to less than or equal to ($\leq$), meaning that values of the assignment variable less than or equal to the threshold lead to assignment. To exclude treatment at the threshold level, click the gear button and deselect the Treatment at the cutoff(s) option. This will convert the inequality to a strict greater than symbol ($>$) or less than symbol ($<$).
For example, choosing an assignment variable and then selecting greater than or equal to ($\geq$) with treatment threshold value 40, would mean that treatment was assigned to those individuals whose assignment variable was equal to or greater than 40. On the other hand, choosing strictly less than ($<$) a value of 10 would mean that individuals whose assignment variable was strictly less than 10 were assigned to the treatment.
Clicking the plus button in the top right corner of the Treatment design panel allows the user to select a secondary assignment variable, to accommodate multiple-assignment RDDs. Currently, the software only supports up to two assignment variables. One restriction that is in place for these designs is that both assignment rules must be either strict inequalities or not (i.e., either both $>$ and/or $<$ or both $\geq$ and/or $\leq$). Combinations of inequalities where one is strict but the other is not can lead to designs in which it is at least theoretically possible to have undefined cases for some analytic approaches.
Under the Treatment Receipt heading, the user selects the treatment variable in pull-down menu labeled actual treatment. The treatment variable indicates whether the individual actually received treatment.
Data analysis tab
The Data tab displays a preview of the data upon upload, mainly for the purposes of validating that the data were read into the program correctly, in the Raw data panel. Variable names are displayed as table column headers. Users can sort observations by each of the variables by either clicking the up arrow next to the variable name for ascending order or the down arrow for descending order.
Upon specifying the Outcome variable and the Treatment design, a Model Data button appears at the top right corner of the Raw data panel. Clicking this button restricts the preview table to just the outcome and treatment design variables.
Specifying the Outcome variable and the Treatment design also generates two new result tables.
Table 1.1
Table 1.1 shows descriptive statistics of the treatment variable (T), the assignment variable (A1 - and A2, if a second assignment was chosen), and the outcome variable (O). For each variable, three columns are reported: the sample size (N), the mean (M), and the standard deviation (SD). In addition, a correlation matrix is shown on the side, whose columns are treatment (T) and outcome (O), and whose values are the Pearson correlation coefficient between variables. Clicking on the notepad icon in the top right corner of the Table 1.1 panel downloads the data as a .csv file.
Table 1.2
Table 1.2 summarizes the design of the RDD, and can serve to validate that the design was correctly specified in the Treatment design panel. Entries of this table are sample sizes, n. For an RDD with a single assignment variable, rows are the two different groups given by the treatment design (i.e., having the assignment variable below or above the specified threshold). The columns specify whether the treatment was actually received within each group, based on the specified treatment receipt variable.
In a sharp RDD (a design in which assigned and received treatment always coincide), the off-diagonal cell counts are always zero. In a fuzzy RDD, the off-diagonal elements will contain sample sizes of incorrectly classified individuals (i.e., treatment assignment and received treatment do not coincide). In this case, the probability of receiving treatment, $\pi$, conditional on being on either side of the threshold is also displayed. This quantity is computed as a simple conditional probability, $P(T=1|A1)$; no parametric model is being used for this estimate.
If a second assignment variable is specified, an additional column is added for this variable. All combinations of treatment thresholds are displayed as single rows, with their respective sample sizes, n, and the estimated $\pi$ parameter.
rddapp reports further details of the treatment design under the design summary table. It determines whether the design is sharp or fuzzy, and whether one or two assignment variables were used. If the second assignment variable does not alter any of the treatments assigned based on the first assignment variable, the software notes that the second assignment variable is redundant, and calls this an ineffective assignment.
Assumptions tab
RDDs rely on several assumptions, which are summarized in the Assumptions tab. Depending on whether the design is sharp or fuzzy, and relies on one or two assignment variables, the content of the assumption tab will vary slightly.
Figure 2.1
Figure 2.1 shows the result of McCrary's (2008) sorting test, which relies on the following idea: in a proper RDD, there should be no discontinuity at the threshold for treatment if participants were neutrally selected based on the assignment rule. The presence of a discontinuity could imply that there was self-selection for or against the treatment by some participants or others involved.
As an example, suppose that students are put on the dean's list if their GPA is 3.75 or higher. Teachers who (either implicitly or explicitly) want to help their students may round up a grade that is very close to the treatment threshold. A teacher may grade the final exam of a student with a GPA of 3.74 more leniently, so that the student can obtain a final GPA of 3.75 and make it on the dean's list. If such behavior was wide-spread, we would find fewer scores of 3.74 and just below, and more scores of 3.75 and just above. This effect would manifest itself as a small discontinuity in the distribution of GPA, right at the treatment threshold.
McCrary's sorting test
The sorting test is implemented as described by McCrary (2008). The user specifies the assignment variable and threshold of interest in the pull-down menu labeled Assignment [Cutoff]. The data are aggregated with a default bin size and a non-parametric kernel smoother is applied to both sides of the treatment threshold using the midpoints of bins and the observed count in each bin as data. The test measures the vertical distance of the two edges on both sides of the treatment threshold with a log metric.
Because the sorting test relies on a non-parametric estimation of the kernel density estimate, it is possible to change parameters of the test. Any changes to the parameters automatically update both the plot and the test summary.
Bin size
The bin size determines the width of bins for the kernel density estimate in the sorting test, which is performed on the means of each bin of raw data. As the bin size decreases, the number of bins with frequency at or close to 1 will increase, and so the density estimate will look very flat (near 0). As the bin size increases, the number of bins decreases, and the kernel density estimate will tend to be under-smoothed if it relies on too few bins.
Band width
The bandwidth, also known as the smoothing parameter, determines how many data points are considered at once when computing the smooth approximation to the data. As the bandwidth increases, more data points are considered and so the smoother goes from flat to jagged. The choice of bandwidth will affect the test statistic, and possibly different inferential decisions. Generally speaking, a bandwidth parameter should be chosen so that no over- or under-smoothing occurs. To aid in the selection of the bandwidth parameter, Figure 2.1 will update every time the smoothing parameter is changed.
When changing the parameters, be mindful that it it is considered bad statistical practice to change the bin size or bandwidth to maximize the p-value in order to make the test look good. Instead either the default bin size should be used, or adjusted in accordance with a properly smoothed plot. The software defaults are often good choices, but sometimes minor adjustments are needed.
Test summary {#test_summary}
The software derives a test statistic, theta, which quantifies the vertical distance, and a standard error (se). These quantities are tested using a $z$-test. The z value and the p value are also reported. Ideally, this test should not be significant. A significant test statistic would indicate that the assumption of a neutrally implemented RDD (assignment only due to treatment threshold) has been violated. The test summary is displayed in the lower right corner of the Figure 2.1 panel.
Both the plot and the test summary can be downloaded using the buttons in the top right corner of the Figure 2.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
Table 2.1
Table 2.1 displays the results of an attrition analysis. In each row, it provides a count, N, of missing data and corresponding percentages of the total sample size for each variable (treatment, outcome, and assignment). The three columns give these counts and percentages:
- for all individuals (Overall);
- restricted to individuals in the control group (Control);
- restricted to individuals who received treatment (Treatment).
This information can help to determine whether any differential attrition occurred. The table can be downloaded as a .csv file using the button in the top right corner of the Table 2.1 panel.
Estimates tab {#estimates_tab}
The Estimates tab is where the user will find point estimates and inferential statistics for the treatment effect. We begin by discussing the basic estimates interface in detail in the section on RDD with one assignment rule. We then discuss specifics for the multivariate case in the RDD with two assignment rule section. We recommend that users read through the single variable section before moving on to the section on multivariate analysis.
RDD with one assignment rule {#rdd_one_asst}
Table 3.1
Table 3.1 contains all results, treatment effect estimates, and inferential statistics for both the parametric and non-parametric models. The user can expand or reduce the amount of information that is being displayed using several toggle switches at the top right of the table, including those for Parametric (global) and Non-parametric (local) models.
Parametric models {#parametric_models}
The parametric model results are subdivided into three rows, where each row gives results of the outcome variable modeled as a function of the assignment variable, but with different functional forms in each case. The available functional forms are:
- linear, which only has a linear term;
- quadratic, which in addition to the linear term from above also has a quadratic term;
- cubic, which in addition to the linear and quadratic terms from above also has a cubic term.
Each term listed above has an associated regression coefficient. Moreover, these coefficients are allowed to differ before and after the assignment threshold. As a result, the model always includes an interaction effect between the assignment variable and the treatment status.
By default, all parametric models used in the estimation of the treatment effect are performed using two-stage least-squares (2SLS) estimation, with robust standard errors. The first stage involves a model that uses the assignment variable to predict the treatment variable. Note that in sharp RDDs, this prediction of treatment assignment will always be perfect by definition, which renders this first step unnecessary; the whole process reduces down to a single-stage least-squares regression model. In the second stage, the predicted values from the first stage are used as predictors of the outcome variable. The treatment effect is captured in the regression coefficient that is associated with the treatment variable in the second stage of estimation.
For each parametric model, several inferential statistics are reported in labeled columns:
- N: the total sample size;
- Est.: the treatment effect estimate;
- SE: the standard error associated with the treatment effect estimate;
- z: the z-score associated with the treatment effect estimate;
- p: the p-value for the estimate;
- 95% CI: the 95% confidence interval for the treatment effect estimate;
- Cohen's d: the Cohen's d effect size, which is computed as the mean difference at the treatment threshold divided by the standard deviation of all observed outcome values (same across all models to make direct comparisons of Cohen's d meaningful).
Non-parametric models
The non-parametric models use a weighted least-squares linear regression. A bandwidth function determines how to weigh each data point in the regression, which determines both the smoothness of the regression, as well as the window around the treatment threshold from which to use data (points are excluded by being assigning a weight of 0).
The non-parametric model results are subdivided into three rows, where each row gives results of the outcome variable modeled as a function of the assignment variable, but with a different bandwidth function. The bandwidth functions include:
- optimal: the "optimal" bandwidth, as defined by Imbens and Kalyanaraman [-@ImbenKalya09; -@ImbenKalya12];
- half-optimal: half the optimal bandwidth;
- double-optimal: double the optimal bandwidth.
Note that, unlike parametric models, these non-parametric models differ in the sample size being used in each regression (since the bandwidth function can assign 0 weight to some points).
The three different non-parametric models are run by default to allow the researcher to see whether results hold under a variety of bandwidths. Additional robustness checks are performed in the sensitivities tab.
In the case of a fuzzy RDD, the user may also request the intent-to-treat estimate, using the ITT toggle switch in the top right of the Table 3.1 panel. This estimate ignores the fuzziness of the design, and simply assigns each individual the treatment that they were supposed to receive based on the treatment assignment rule. Note that ITT model is only useful for fuzzy designs since in sharp designs, treatment assignment and actual treatment always coincide.
If any covariates are specified in the auxilliary variables field found in the additional options of the outcome panel, an additional toggle button labeled COV is shown at the top right of the table. It is deselected by default, but if the user selects it, additional rows are added to Table 3.1 specifying the regression coefficient that describes the (linear) relationship between the covariate and the outcome is displayed, along with all inferential statistics. Note that even though in the table the regression coefficient is displayed under the same column heading (Est.), it does not mean that this is a treatment effect estimate. It is simply the linear relationship between covariate and the outcome variable. The regression coefficient for the covariate is obtained from the same model that is used to estimate the RDD treatment effect. That means, the treatment and assignment variables are also predictors in this model. It is currently not possible to model the relationship between the covariate and the outcome variable in a non-linear fashion (e.g., polynomial regression, interacted with the treatment, etc.).
The table can be downloaded as a .csv file using the button in the top right corner of the Table 3.1 panel.
Figure 3.1
Figure 3.1 provides a visual display of the treatment effect summarized in Table 3.1, typically through some combination of a scatterplot of the assignment vs. outcome variables along with a plot of one or more fitted models and their associated confidence intervals over top.
The user can customize this plot in three different ways: by changing the model type, predicted lines, and data points.
Model type
The model type option is only useful for fuzzy designs as there is no difference between the compiler average and the intention-to-treat options for sharp design; in these cases, treatment assignment and actual treatment always coincide.
Predicted lines {#predicted_lines}
The user can toggle which of the models summarized in Table 3.1 are plotted as predicted lines in Figure 3.1 by selecting their labels from the drop-down menu or by deleting their labels from this field by clicking the label and hitting the backspace key on the keyboard. By default, the non-parametric optimal model is overlaid.
Furthermore, the user can specify a percentage for the confidence interval(s) plotted in the CI input box, and toggle whether or not the area between the confidence interval boundaries should be colored in (by selecting the fill area button).
Data points {#data_points}
The user can choose whether to scatterplot all data points (with the raw option), or to bin them evenly or by quartile. The latter option may be preferred if there are many data points to prevent over-plotting. In each bin, an average of the outcome variable is computed and plotted as a single point.
Binning evenly means that the assignment variable is sub-divided into bins of equal width, while binning by quantiles uses quantiles of the assignment variable. In either case, the binning algorithm will always avoid spanning a bin over the treatment threshold value, which can sometimes lead to slight distortions in the bin width and sample size. For instance, there are situations in which even binning does not result in bins of equal width, or quantile binning does not result in the same sample size across all bins, as one might expect.
The user can specify the number of bins in the input box labeled bins. The point size can be scaled with respect to the number of points in each bin using the scale point size switch.
A (normal) confidence interval is plotted around the estimate of the mean, at whatever percentage specified in the CI field. The user can toggle whether or not to hide error bars with the eponymous button.
Plot labels (e.g., regression line labels, point legends, axis labels) are generated automatically and cannot be customized by the user at this time.
The plot can be downloaded as a .png, .svg, and a .pdf using the buttons in the top right of the Figure 3.1 panel.
RDD with two assignment rules {#rdd_two_asst}
An RDD with two assignment variables requires a more complicated analysis because several estimates have been suggested in the literature. For an overview, see the paper by Wong, Steiner, and Cook (2013), which describes a variety of designs and estimates. The three estimate approaches implemented in this software are directly derived from the Wong et al. (2013) paper, as well as the Stata code that was published by Wong.
The top right of the Table 3.1 panel features toggle buttons for the three multivariate approaches: Centering, Univariate, and Frontier. For each of these approaches, the user can also the toggle parametric and non-parametric models described in the previous section, and the same results are displayed as previously discussed.
Centering approach {#centering_approach}
In the centering approach, the multiple assignment RDD is transformed into a single assignment RDD through the use of a centering function. In particular, the program centers all assignment variables around the treatment threshold value by subtracting the value of the treatment threshold from each assignment variable; these differences are called deviation scores. For each observation, we take minimum of the two deviation scores calculated from the values of each of the two assignment variables. In designs in which treatment is assigned to units above or below the treatment threshold (or any combination of those), sometimes the maximum is taken.
Following this transformation, each observation has a single (centered) assignment variable (the minimum deviation score), which is then used to compute a single assignment RDD. Note that the treatment effect estimate is sensitive to the metric used for each assignment variable. Wong et al. (2013) show that even standardization cannot typically mitigate this issue, and advise that the centering approach should only be used when the assignment variables share the same metric.
The centering approach computes what Wong et al. call the frontier average treatment effect, which is essentially a treatment effect along the whole treatment threshold of the two-dimensional response (outcome) surface defined by the two assignment variables.
The plot for the centering approach can be displayed in Figure 3.1 by selecting Centering in the Model Type drop-down menu of that panel. The options for Predicted lines and Data points remain the same as in the case of an RDD with a single assignment variable.
Univariate approach {#univariate_approach}
The univariate approach focuses on treatment effects that are exclusively attributable to one of the two assignment variables. Wong et al. (2013) call these the frontier-specific effects. In a design with two assignment variables, two frontier-specific effects will be computed: one for each assignment variable.
To estimate these effects, the observations are divided into subsets based on which rule assigned them to treatment. For example, to compute the frontier-specific effect of the first assignment variable, the only observations used are those assigned to treatment based solely on the first assignment variable. Table 3.1 shows the sample sizes for each subset in the N column.
These subsets of the data for each assignment variable, labeled A1 and A2, are then analyzed using an RDD with one assignment variable.
There is an alternative to the univariate approach, which retains all subjects and then treats the resulting design as a fuzzy RDD with a single assignment variable. Wong et al. (2013) demonstrated that this approach generally has no advantages with respect to increasing statistical power or decreasing bias when compared to simply excluding the mis-classified observations from the analysis.
The plot for the univariate approach can be displayed in Figure 3.1 by selecting Univariate (A1) or Univariate (A2) in the Model Type drop-down menu of that panel. The options for Predicted lines and Data points remain the same as in the case of an RDD with a single assignment variable.
Frontier approach {#frontier_approach}
The frontier approach estimates the treatment effect along the whole discontinuity frontier of the response (outcome) surface. As described by Wong et al. (2013), this approach is performed by first estimating the response surface (e.g., using a parametric model), and then using kernel density estimates for numerical integration of the treatment effect.
The frontier approach yields three different estimates of the average treatment effect:
- the average frontier effect, computed for the whole frontier
- the frontier 1 effect, computed for the frontier along the first assignment variable, A1;
- the frontier 2 effect, computer for the frontier along the second assignment variable, A2.
All of these effects are estimated simultaneously using the full sample.
This method usually results in higher statistical power than other approaches, but requires the assumption that the response surface is correctly modeled by the linear parametric model used in the computation. Non-parametric models are not supported at this time and so its toggle button is inactive in this case.
There are three linear parametric models available: Unconstrained, Heterogeneous effects, and Constant Effect. Note that non-linear parametric models are not supported at this time.
The constant effect model is the most restrictive, as it imposes two big constraints:
- the heights of the discontinuities at the treatment thresholds must be equal across both dimensions given by the assignment variables;
- the regression slope on all parts of the response surface (away from the discontinuity) must be equal.
Despite these constraints, the average frontier treatment effect and the frontier-specific effects are not necessarily identical due to the fact that the raw data may not be identically distributed along the regression surface, and so the kernel density estimate for the numerical integration is not weighted equally on all parts of the frontier.
The heterogeneous effects model relaxes constraint 1 of the constant effect model. In particular, the heights of the discontinuities are allowed to differ in different segments of the frontier. However, the slope of the regression surface must still be equal everywhere (away from the discontinuities).
The fully unconstrained model eschews both constraints 1 and 2, which means that it is the most complex of these linear parametric models. Wong et al. (2013) does not provide closed form (analytic) solutions for standard errors for the frontier treatment effects. We use bootstrapping to compute the inferential statistics in Table 3.1.
In order to perform these computations, the number of bootstrap samples must be specified in the additional options of the Outcome panel (in the Data tab), which is displayed using the gear button in the top right corner of the panel. The input field for the number of bootstrap samples is labeled SE and CI for frontier estimate. Once the number of samples is specified and the user navigates back to the Estimates tab, the computation begins and Table 3.1 is populated when the calculations are complete. Using a large number of bootstrap samples is computationally expensive, and it make take a long time to complete the calculations.
Figure 3.1 displays a three-dimensional plot of the response surface when the Frontier option is selected in the Model type pull-down menu. New pull-down menus appear below, replacing the Predicted lines and Data points options.
The first new menu is labeled Model Specification, where the user can select any of the three linear parametric models implemented for the frontier approach (Unconstrained, Heterogenous effect, Constant effect). These options help the user visualize each model's constraints (or lack thereof).
The remaining new menus enable the user to modify the appearance of the plot. The View pull-down menu has four options: Perspective,which displays a three-dimensional plot of the response surface, as well as Top, Front, and Side, which show various two-dimensional projections of the surface (flattening one dimension). For example, the Top view shows the response surface directly from above, flattening the outcome variable dimension, which can help the user visualize the treatment thresholds.
The three-dimensional perspective plot can be rotated using the sliders labeled with arrows. The graph is regenerated every time these sliders are adjusted, and it takes for the plot to refresh.
The panel buttons allow the user to color all surfaces that received the treatment assignment, add a grid to the response surface, and shade the surface slightly.
Finally, data buttons allow the user to scale the axes to the original scale (by default, the axes are scaled to the same length), and to add the raw data points to the plot.
Fuzzy RDD
In the case of an RDD with multiple assignments that is fuzzy, the design becomes slightly more complicated. However, the overall interface and outputs are identical to the sharp counterpart of this design. All of the models that were described above, are now being estimated using the two-stage least squares approach, in which the fuzzy treatment assignment is estimated in a first step, and then the estimated treatment assignments are used for all of the models.
Sensitivities tab {#sensitivities_tab}
In this tab, the user can perform various sensitivity analyses, including those for the treatment threshold and the bandwidth. Note that neither of Figures 4.1 or 4.2 will be generated automatically; they will only appear after the corresponding blue circular arrow button is pressed (found in the lower right corner of each figure panel).
Figure 4.1
Figure 4.1. performs a sensitivity analysis for the treatment threshold by plotting the treatment effect estimate as a function of a range of hypothetical treatment thresholds for various models.
These tests of treatment effects at wrong treatment threshold values are sometimes also called placebo tests. The assumption is that in a properly-run RDD, the treatment effect should only be visible at the actual treatment threshold. Hence, seeing many non-significant results at treatment thresholds away from the actual treatment threshold value is desirable.
The user can change the span of treatment thresholds to test by defining the endpoints using the min and max boxes below the plot, and the number of estimates to compute between these endpoints using the steps box. The treatment effect at the true treatment threshold is always estimated and denoted with a dashed vertical line labeled designed cutoff. By default, a 95% confidence interval is shown around the estimates, but the user can change this percentage in the CI box at the top right of the Figure 4.1 panel.
The user can select the models with which to perform this analysis (e.g., the parametric linear model, the non-parametric optimal model etc.) in the box directly under the plot. Clicking this box will open a drop-down menu featuring all models that can be added to the plot. To delete a model from the plot, click on its label and press the keyboard's backspace button. The labels in this menu correspond to the ones used in the Estimates tab.
Both the plot and the data being plotted can be downloaded using the buttons in the top right corner of the Figure 4.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
Figure 4.2
Figure 4.2. is a robustness check for the non-parametric model; it shows how the treatment effect estimate changes with different bandwidths. Ideally, the graph should show a somewhat flat line that indicates that over a wide range of bandwidth choices, the estimate does not change dramatically.
The user can change the span of bandwidths to test by defining the endpoints using the min and max boxes below the plot, and the number of estimates to compute between these endpoints using the steps box. The treatment effect at the optimal bandwidth is always estimated and denoted with a dashed vertical line labeled optimal bandwidth. By default, a 95% confidence interval is shown around the estimates, but the user can change this percentage in the CI box at the top right of the Figure 4.2 panel.
Both the plot and the data being plotted can be downloaded using the buttons in the top right corner of the Figure 4.2 panel. The plot can be downloaded as a .png, .svg, or .pdf. The test data displayed in the plot can be downloaded as a .csv file.
R code tab
In this tab, the current analysis that the user selected via the drop-down menu is translated into R code. Users can copy and paste this code into an R script for customization, automation, and for conducting reproducible research.
Power page {#power_page}
Users may wish to perform a formal (frequentist) power analysis, to determine a priori sample sizes, or to construct power curves for various sample sizes. All power analyses are performed using Monte Carlo simulations. The user defines population parameters, which are used to simulate a model. The treatment effect is estimated from the simulation and its significance is calculated. This procedure is repeated many times, and a mean and variance are calculated for the treatment effects. The statistical power is estimated as the proportion of significant results across all Monte Carlo simulations.
The success of this process hinges on properly defining the population parameters, which is often challenging. We attempt to guide the user in this process using various graphical displays. These displays change depending on the particular design that is chosen (i.e., whether the RDD has a single or multiple assignment variable(s)).
Assignment panel
The Assignment panel is where the user defines the assumed distribution of the assignment variable. The plot shows a histogram with kernel density estimate. The shading of the histogram bars indicates group membership (treatment or control).
The settings under the Distribution label allow the user to specify the distribution type and parameters for the assignment variable, labeled A1 by default. The distribution types available are normal and uniform, and the user can toggle between them using the eponymous toggle button. For a normal distribution, a mean $\mu$ and a standard deviation $\sigma$ must be provided. For a uniform distribution, a minimum value and a maximum value must be provided.
Under the Treatment design label, the user can set the assignment rule, i.e., the values of the assignment variable A1 at which treatment is administered. The assignment rule changes the proportion of treated and untreated, which affects power estimates. To change the assignment rule, the user can click on the inequality symbol to toggle between greater than or equal to and less than or equal to and change the value of the assignment threshold in the adjacent text box. Note that there is no setting to exclude treatment at the threshold (i.e., to remove the equality from the inequality). If the assignment variable is discrete and the desired treatment assignment is A1 $> x$, then the user could use the equivalent assignment rule A1 $\geq x+1$.
The user may also change the amount of fuzziness in the design, which is defined as the proportion of misclassified cases around the treatment threshold. The user can set this proportion (as a decimal) below and above the threshold independently, using the text boxes adjacent to the fuzziness label. The left box is for the proportion of control cases that were mis-classifieds and instead treated, while the right box is for cases assigned treatment but actually left untreated. Note that the left box is always associated with control cases and the right box is always associated with treatment cases even if the assignment rule flips the plot so that cases under (i.e., to the left of) the threshold are treated.
The sample size can be specified in the box found in the top right of the Assignment panel.
Clicking the plus button in the top right corner of the Assignment panel allows the user to add a second assignment variable (labeled A2). The distribution and treatment design for A2 can be set in the same way as they were for A1. All parameters (including distribution, and fuzzyness) of the second variable can be modified in the same way. The plot will update to show an annotated scatterplot with each assignment variable on one axis, along with a sketch of their distributions.
Parametric model for outcome panel
Once the assignment is specified, the user must specify the parametric model for the outcome variable as a function of the assignment and treatment.
Currently, only linear models are implemented, though there is a fully unconstrained parametric model with respect to interactive effects. In this case, every segment of the regression surface can have a different slope.
The plot in this panel shows the simulated outcome variable as a function of the assignment and will update when simulation parameters are changed.
In the case of an RDD with one assignment variable, a scatterplot with a regression line is shown. The user can modify the treatment effect (labeled Treatment), the intercept of the regression slope (labeled Constant), the slope on the control side of the treatment threshold (labeled Slope), and the difference in slopes on the treatment side of the threshold (labeled Interaction). Lastly, the user can define the Partial $\eta^2$, which measures the effect size of the treatment. As the partial $\eta^2$ increases, the simulated data will lie closer to the regression lines. Defining the effect size is an indirect way of changing the standard error of the treatment effect.
In the case of an RDD with two assignment variables, the parametric model is substantially more complex. It now involves treatment effects at both frontiers, and slopes that are defined for every piece of the regression surface divided by the threshold for each assignment variable.
There are a total of 13 regression coefficients that need to be defined by the user:
- the intercept, i.e., the predicted outcome when both assignment variables at their respective treatment thresholds (labeled constant);
- the two treatment effects (labeled T1 and T2 for the assignment variables A1 and A2 respectively), which set the effect size of the treatment at the threshold of each assignment variable;
- the two regression coefficients that relate the assignment variables to the outcome, while holding treatments constant at the control group (labeled A1 and A2);
- a total of seven interaction effects are included, that define how the slopes of the assignment variables change with different values of the treatment and control values for the two cut-offs.
Table 5.1
Once the parametric models are specified in the Assignment and Parametric Model for Outcome panels, the user can generate Monte Carlo simulations in the Table 5.1 Powers of RDD estimates panel.
The user can specify the number of Monte Carlo simulations to be performed in the field labeled Iterations in the top right corner of the panel. The default number of iterations is 500. In general, as the number of iterations increases, the power estimate becomes more precise; however, a large number of iterations may take a very long time to complete.
The simulations are executed by pressing the blue circular arrow button in the top right corner of the panel. Once the simulations are complete, Table 5.1. is populated with results.
For an RDD with one assignment variable, the rows of the table give the results for the linear parametric model and the non-parametric model with optimal bandwidth.
For an RDD with two assignment variables, the rows of the table give the results for three linear parametric models (centering, and two univariate models), as well as three non-parametric models (same as the parametric ones but with an optimal bandwidth). Other possible effects (e.g., the frontier average effect) are not estimated as they take prohibitively long to compute.
In both cases, the columns of the table are:
- S: the number of successful Monte Carlo simulations; typically, this value will be equal to the number of iterations, but for some usual models (e.g., lots of fuzziness in the assignment), it is possible that some simulations do not yield a valid estimate;
- Treatment effects: the sample Mean and Variance of the treatment effect from the simulations of each model; for a large enough number of iterations, the user-defined treatment effect should be recovered;
- Power: the estimated statistical power (Type II error rates) for various significance levels ($\alpha = .05, .01, .001$).
The tab above the Table 5.1 panel, labeled R Code gives the code used to perform the simulation and populate the table.
Figure 5.1
Figure 5.1. allows the user to investigate how statistical power is affected by sample size. Inside the panel, the user can set the minimum sample size and the maximum sample size, while the number of sample sizes to take in this interval can be set in the Steps box, in the top right corner of the panel.
The number of Iterations in each step can be set in the top right corner of the panel as well. The default number of iterations is 500. In general, as the number of iterations increases, the power estimate becomes more precise; however, a large number of iterations may take a very long time to complete.
The simulations are executed by pressing the blue circular arrow button in the top right corner of the panel. Figure 5.1. is populated with results as the simulations for each sample size are complete.
Once the simulations are fully complete, the user can adjust what is plotted in the figure. The Group lines by drop-down menu reveals two main options:
- Model type: the results for each model are plotted as a different curve;
- Alpha level: the results the three levels of significance ("alpha levels") are plotted as different curves.
Depending on the Group lines by option selected, a second drop-down menu appears below it. If Model type is selected, the menu Show alpha level appears, where the user chooses the significance level for which to plot the estimated power. If Alpha leve is selected, the menu Show model type appears, where the user chooses the model for which to plot the estimated power.
Both the currently displayed plot and the full simulation data can be downloaded using the buttons in the top right corner of the Figure 5.1 panel. The plot can be downloaded as a .png, .svg, or .pdf. The simulation data can be downloaded as a .csv file.
About
This software was developed by Felix Thoemmes, Wang Liao, Ze Jin, Wenyu Zhang, Irena Papst, and Kim Hochstedler all at Cornell University. We are happy to receive feedback! Please send it to
The project was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305D150029. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.
Citations
Try the rddapp package in your browser
Any scripts or data that you put into this service are public.
R Package Documentation
Browse R Packages
We want your feedback!
Note that we can't provide technical support on individual packages. You should contact the package authors for that.
Embedding an R snippet on your website
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.