Introduction to mobilityIndexR
In mobilityIndexR: Calculates Transition Matrices and Mobility Indices

This vignette provides a guide for using the mobilityIndexR package. mobilityIndexR allows users to both calculate transition matrices (relative, mixed, and absolute) as well as calculate and compare mobility indices (Prais-Bibby, Average Movement, Weighted Group Mobility, and Origin Specific). Below are examples and brief explanations of each.

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

library(mobilityIndexR)

Transition Matrices

Two dimensional transition matrices are a tool for representing mobility within a group between two points in time with respect to some observable value such as income of workers or grades of students. Generally, this approach in some way ranks the observable value at each point in time and displays a contingency table with aggregated counts or proportions of each combination of ranks. mobilityIndexR offers three methods of constructing ranks using the getTMatrix function: relative, mixed, and absolute.

As example data, we include three datasets: mobilityIndexR::incomeMobility, mobilityIndexR::incomeZeroInfMobility, and mobilityIndexR::gradeMobility. These datasets each contain 125 records, an "id" column, and observations of some value at ten points in time, denoted "t0", ..., "t9". mobilityIndexR::incomeMobility simulates income data over ten years; mobilityIndexR::incomeZeroInfMobility simulates income data over ten years that has an inflated number of zeros; and mobilityIndexR::gradeMobility simulates student grade data over ten assignments.

head(incomeMobility)

Relative

Types of transition matrices differ by how values are binned into ranks. With relative transition matrices, values in each of the two specified columns (col_x, col_y) are binned into ranks as quantiles. The number of quantiles is specified in the num_ranks argument. Note: if one value in the data occurs more often than the size of the quantile ranks, then getTMatrix will throw an error; see Exclude Value.

getTMatrix returns a list containing the transition matrix as well as the bounds used to bin the data into ranks.

getTMatrix(dat = incomeMobility, col_x = "t0", col_y = "t5", 
           type = "relative", num_ranks = 5, probs = FALSE)

The above example produces a $5 \times 5$ contingency table. Setting the argument probs = TRUE, we obtain a transition matrix with unconditional probabilities rather than counts.

getTMatrix(dat = incomeMobility, col_x = "t0", col_y = "t5", 
           type = "relative", num_ranks = 5, probs = TRUE)

Mixed

With mixed transition matrices, values in col_x are first binned into ranks as quantiles, then the bounds for col_x are used to bin col_y into ranks. In the example below, observe that col_x_bounds and col_y_bounds are equal except for the minimum and maximum values.

getTMatrix(dat = incomeMobility, col_x = "t0", col_y = "t5", 
           type = "mixed", num_ranks = 5, probs = FALSE)

Absolute

With absolute transition matrices, values in col_x and col_y are binned into ranks using user-specified bounds with the bounds argument.

getTMatrix(dat = gradeMobility, col_x = "t0", col_y = "t5", 
           type = "absolute", probs = FALSE, bounds = c(0, 0.6, 0.7, 0.8, 0.9, 1.0))

Mobility Indices

Mobility indices are measures of mobility within a group between two points in time with respect to some observable value such as income of workers or grades of students. mobilityIndexR calculates indices using the getMobilityIndices function. By default, the getMobilityIndices returns all available indices. Note that getMobilityIndices can additionally return bootstrapped intervals for each of the indices; see Bootstrapped Intervals.

getMobilityIndices(dat = incomeMobility, col_x = "t0", col_y = "t5", 
                   type = "relative", num_ranks = 5)

Below is a brief description of the indices. Let $s(i)$ be the event that one is in rank $i$ at the first or starting point in time and $e(j)$ be the event that one is in rank $j$ at the second or ending point in time. Suppose $k$ is the largest rank in the data.

The Prais-Bibby index (type = "prais_bibby") is the proportion of records that change ranks, i.e. $\Sigma_{i = 1}^kPr[s(i) \neq e(i)]$.

The Average Mobility index (type = "average_movement") is the average number of ranks records move.

The Weighted Group Mobility index (type = "wgm") is a weighted version of the proportion of records that change ranks, i.e. $k^{-1} \Sigma_{i = 1}^k \frac{Pr[\neg e(i) | s(i)]}{Pr[\neg e(i)]}$.

There are four Origin Specific indices (type = "origin_specific"): top, bottom, top far, and bottom far. The top (bottom) index is the proportion of records that begin in the top (bottom) and end outside of the top (bottom). The far versions of these indices are the proportions of records that begin in the top (bottom) and end at least two ranks away from the top (bottom).

getMobilityIndices(dat = incomeMobility, col_x = "t0", col_y = "t5", 
                   type = "relative", num_ranks = 5, indices = "wgm")

A single index or subset of the indices can be returned by passing a string or vector of strings in the indices argument.

Hypothesis Testing

The function getHypothesisTest compares mobility indices across datasets. The user specifies two datasets (dat_A, dat_B) as well as a pair of columns from each dataset as a vector (cols_A, cols_B). This function performs a non-parametric one-sided hypothesis test that the index value for dat_A is greater than the index value for dat_B. The parameter bootstrap_iter specifies the number of bootstrap iterations; the default value is 100.

getHypothesisTest(dat_A = incomeMobility, dat_B = incomeMobility, 
                  cols_A = c("t0", "t3"), cols_B = c("t5", "t8"),
                  type = "relative", num_ranks = 5, bootstrap_iter = 100)

By default, all available indices are returned. A single index or subset of the indices can be returned by passing a string or vector of strings in the indices argument.

Advanced Options

Bootstrapped Intervals

The function getMobilityIndices produces non-parametric bootstrapped confidence intervals using the percentile method by setting intervals = TRUE. By default, intervals = FALSE. The parameter interval_pct specifies the size of the confidence interval in terms of the proportion of the data covered; the default value is 0.95. The parameter bootstrap_iter specifies the number of bootstrap iterations; the default value is 100.

getMobilityIndices(dat = incomeMobility, col_x = "t0", col_y = "t5", 
                   type = "relative", num_ranks = 5, indices = "wgm",
                   intervals = TRUE, interval_pct = 0.95, bootstrap_iter = 100)

Exclude Value

For type = "relative" or type = "mixed", if one value in the data occurs more often than the size of the ranks then an error will result. This error will occur with getTMatrix, getMobilityIndices, and getHypothesisTest since binning into quantile ranks is not well defined in this case. See the example below where $0$ makes up more than $20\%$ of the values in the "t0" column.

getTMatrix(dat = incomeZeroInfMobility, col_x = "t0", col_y = "t5",
           type = "relative", num_ranks = 5, probs = FALSE)
table(incomeZeroInfMobility$t0)

One approach is to bin some elements of the problematic value in one rank and the others in another rank. This is done by setting the parameter strict = FALSE. By default, strict = TRUE. This setting adds a negligible amount of random noise to the problematic values to make binning the values into quantile ranks well defined.

getTMatrix(dat = incomeZeroInfMobility, col_x = "t0", col_y = "t5",
           type = "relative", num_ranks = 5, probs = FALSE, strict = FALSE)

A second approach is to exclude the problematic value from binning the values into quantile ranks then either give the problematic value its own rank, add the problematic value to an existing rank, or exclude the problematic value altogether. A problematic value can be specified in the exclude_value argument. Note: currently, mobilityIndexR only supports one exclude value.

If an exclude value is specified, the user must also specify how to handle the exclude value using the rerank_exclude_value parameter. Setting rerank_exclude_value = "as_new_rank" creates an new rank for the exclude value and modifies the other ranks to remain ordinally consistent. Setting rerank_exclude_value = "as_existing_rank" bins the exclude value into an existing rank. Setting rerank_exclude_value = "exclude" drops any row that has the exclude value from the calculation.

getTMatrix(dat = incomeZeroInfMobility, col_x = "t0", col_y = "t5",
           type = "relative", num_ranks = 5, probs = FALSE,
           exclude_value = 0, rerank_exclude_value = "as_new_rank")

getTMatrix(dat = incomeZeroInfMobility, col_x = "t0", col_y = "t5",
           type = "relative", num_ranks = 5, probs = FALSE,
           exclude_value = 0, rerank_exclude_value = "as_existing_rank")

getTMatrix(dat = incomeZeroInfMobility, col_x = "t0", col_y = "t5",
           type = "relative", num_ranks = 5, probs = FALSE,
           exclude_value = 0, rerank_exclude_value = "exclude")