knitr::opts_chunk$set( collapse = TRUE, comment = "#>" )
Census-level data refers to a data set wherein there is one row per policy. Exposure-level data expands census-level data such that there is one record per policy per observation period. Observation periods could be any meaningful period of time such as a policy year, policy month, calendar year, calendar quarter, calendar month, etc.
A common step in experience studies is converting census-level data into exposure-level data. The expose()
family of functions assists with this task. Specifically, the expose()
family:
NA
for all periods except the last.exposed_df
class, which is a format expected by several actxps functions.If you already have exposure-level data available, the expose()
functions are not necessary. However, we recommend converting your data to the exposed_df
format using the function as_exposed_df()
.
To get started, we're going to use a toy census data frame from the actxps package that contains 3 policies: one active, one that terminated due to death, and one that terminated due to surrender.
toy_census
contains the 4 columns necessary to compute exposures:
pol_num
: a unique identifier for individual policiesstatus
: the policy statusissue_date
: issue dateterm_date
: termination date, if any. Otherwise NA
library(actxps) library(dplyr) toy_census
Let's assume we're performing an experience study as of 2022-12-31 and we're interested in policy year exposures. Here's what we should expect for our 3 policies.
To calculate exposures, we pass our data to the expose()
function and we specify a study end_date
.
exposed_data <- expose(toy_census, end_date = "2022-12-31")
This creates an exposed_df
object, which is a type of data frame with some additional attributes related to the experience study.
is_exposed_df(exposed_data)
Let's examine what happened to each policy.
Policy 1: As expected, there are 13 rows for this policy. New columns were added for the policy year (pol_yr
), date ranges (pol_date_yr
, pol_date_yr_end
), and
exposure. All exposures are 100% since this policy was active for all 13 years.
When the data is printed, additional attributes from the exposed_df
class are displayed.
exposed_data |> filter(pol_num == 1)
Policy 2: There are 10 rows for this policy. The first 9 periods show the policy in an active status
and the termination date (term_date
) is set to NA
. The last period includes the final status of "Death" and the actual termination date. The last exposure is less than one because roughly a third of a year elapsed between the last anniversary date on 2020-05-27 and the termination date on 2020-09-14.
exposed_data |> filter(pol_num == 2)
Policy 3: There are 13 rows for this policy. The first 12 periods show the policy in an active status
and the termination date (term_date
) is set to NA
. The last period includes the final status of "Surrender" and the actual termination date. The last exposure is less than one because the roughly a third of a year elapsed between the last anniversary date on 2021-11-10 and the termination date on 2022-02-25.
exposed_data |> filter(pol_num == 3)
The previous section only supplied data and a study end_date
to expose()
. Optionally, a start_date
can be supplied that will drop exposure periods that begin before a specified date.
expose(toy_census, end_date = "2022-12-31", start_date = "2019-12-31")
Most experience studies use the annual exposure method which allocates a full period of exposure for the particular termination event of interest in the scope of the study.
The intuition for this approach is simple: let's assume we have an unrealistically small study with a single data point for one policy over the course of one year. Let's assume that policy terminated due to surrender half way through the year.
If we don't apply the annual exposure method, we would calculate a termination rate as:
$$ q^{surr} = \frac{claims}{exposures} = \frac{1}{0.5} = 200\% $$
A termination rate of 200% doesn't make any sense. Under the annual exposure method we would see a rate of 100%, which is intuitive.
$$ q^{surr} = \frac{claims}{exposures} = \frac{1}{1} = 100\% $$
The annual exposure method can be applied by passing a character vector of target statuses to the expose()
function.
Let's assume we are performing a surrender study.
exposed_data2 <- expose(toy_census, end_date = "2022-12-31", target_status = "Surrender")
Now let's verify that the exposure on the surrendered policy increased to 100% in the last exposure period.
exposed_data2 |> group_by(pol_num) |> slice_max(pol_yr)
The default exposure basis used by expose()
is policy years. Using the arguments cal_expo
and expo_length
other exposure periods can be used.
If cal_expo
is set to TRUE
, calendar year exposures will be calculated.
Looking at the second policy, we can see that the first year is left-censored because the policy was issued two-fifths of the way through the year, and the last period is right-censored because the policy terminated roughly seven-tenths of the way through the year.
exposed_cal <- toy_census |> expose(end_date = "2022-12-31", cal_expo = TRUE, target_status = "Surrender") exposed_cal |> filter(pol_num == 2)
The length of the exposure period can be decreased by passing "quarter"
, "month"
, or "week"
to the expo_length
argument. This can be used with policy or calendar-based exposures.
toy_census |> expose(end_date = "2022-12-31", cal_expo = TRUE, expo_length = "quarter", target_status = "Surrender") |> filter(pol_num == 2)
The following functions are convenience wrappers around expose()
that target a specific exposure type without specifying cal_expo
and expo_length
.
expose_py()
= exposures by policy yearexpose_pq()
= exposures by policy quarterexpose_pm()
= exposures by policy monthexpose_pw()
= exposures by policy weekexpose_cy()
= exposures by calendar yearexpose_cq()
= exposures by calendar quarterexpose_cm()
= exposures by calendar monthexpose_cw()
= exposures by calendar weekA common technique used in experience studies is to split calendar years into two records: a pre-anniversary record and a post-anniversary record. In actxps, this can be accomplished using the expose_split()
function.
Let's continue examining the second policy. exposed_cal
, which contains calendar year exposures, is passed into expose_split()
. The resulting data frame now contains 19 records instead of 10. There is one record for 2011 and 2 records for all other years. The year 2011 only has a single record because the policy was issued in this year, so there can only be a post-anniversary record.
split <- expose_split(exposed_cal) split |> filter(pol_num == 2) |> select(cal_yr, cal_yr_end, pol_yr, exposure_pol, exposure_cal)
The output of expose_split()
contains two exposure columns.
exposure_pol
contains policy year exposuresexposure_cal
contains calendar year exposuresThe two exposure bases will often not match for two reasons:
Calendar years and policy years have different start and end dates that may or may not include a leap day. In the first row, the calendar year exposure is 0.6 years of the year 2011, which does not include a leap day. In the second row, the policy year exposure is 0.5984 years of the policy year spanning 2011-05-27 to 2012-05-26, which does include a leap day.
Application of the annual exposure method. If the termination event of interest appears on a post-anniversary record, policy exposures will be 1 and calendar exposures will be the fraction of the year spanning the anniversary to December 31st. Conversely, if the termination event of interest appears on a pre-anniversary record, calendar exposures will be 1 and policy exposures will be the fraction of the policy year from January 1st to the last day of the current policy year. While it may sound confusing at first, these rules are important to ensure that the termination event of interest always has an exposure of 1 when the data is grouped on a calendar year or policy year basis.
Some downstream functions like exp_stats()
expect exposed_df
objects to have a single column for exposures. For split exposures, the exposure basis must be specified using the col_exposure
argument.
exp_stats(split)
tryCatch(exp_stats(split), error = function(e) cat(e$message))
exp_stats(split, col_exposure = "exposure_pol")
expose_split()
doesn't just work with calendar year exposures. Calendar quarters, months, or weeks can also be split. For periods shorter than a year, a record is only split into pre- and post-anniversary segments if a policy anniversary appears in the middle of the period.
expose_cq(toy_census, "2022-12-31", target_status = "Surrender") |> expose_split() |> filter(pol_num == 2) |> select(cal_qtr, cal_qtr_end, pol_yr, exposure_pol, exposure_cal)
Note, however, that calendar period exposures will always be expressed in the original units and policy exposures will always be expressed in years. Above, calendar exposures are quarters whereas policy exposures are years.
For machine learning feature engineering, the actxps package contains a function called step_expose()
that is compatible with the recipes package from tidymodels. This function applies the expose()
function within a recipe.
library(recipes) expo_rec <- recipe(status ~ ., toy_census) |> step_expose(end_date = "2022-12-31", target_status = "Surrender", options = list(expo_length = "month")) |> prep() expo_rec tidy(expo_rec, number = 1) bake(expo_rec, new_data = NULL)
As a default, the expose()
functions assume the census data frame uses the following naming conventions:
pol_num
status
issue_date
term_date
These default names can be overridden using the col_pol_num
, col_status
, col_issue_date
, and col_term_date
arguments.
For example, if the policy number column was called id
in our census-level data, we could write:
expose(toy_census, end_date = "2022-12-31", target_status = "Surrender", col_pol_num = "id")
If the census-level data contains other policy attributes like plan type or policy values, they will be broadcast across all exposure periods. Depending on the nature of the data, this may or may not be desirable. Constant policy attributes like plan type make sense to broadcast, but numeric values may or may not depending on the circumstances.
toy_census2 <- toy_census |> mutate(plan_type = c("X", "Y", "Z"), policy_value = c(100, 125, 90)) expose(toy_census2, end_date = "2022-12-31", target_status = "Surrender")
If your experience study requires a numeric feature that varies over time (ex: policy values, crediting rates, etc.), you can always attach it to an exposed_df
object using a join function.
# Illustrative example - assume `values` is a data frame containing the columns pol_num and pol_yr. exposed_data |> left_join(values, by = c("pol_num", "pol_yr"))
exposed_df
objectsIf you need to stack two exposed_df
objects, vctrs::vec_rbind()
is recommended over rbind()
or dplyr::bind_rows()
. The advantage of vctrs::vec_rbind()
is that it will combine attributes across all exposed_df
objects passed to the function. The study end date will be updated to maximum study end date. Similarly, the study start date will be set to the earliest study start date. Target statuses and transactions types will become a super set of all observed values. The other two functions will retain attributes from only the first object passed to them.
For example, below exposed_data2
contains study start and end dates that are before and after the study range in exposed_data
. In addition, this object contains a target status of "Surrender" whereas exposed_data
has none.
When vctrs::vec_rbind()
is used to combine exposed_data
and exposed_data2
, the result combines attributes across both objects.
exposed_data2 <- expose(toy_census, end_date = "2023-12-31", start_date = "1890-01-01", target_status = "Surrender") vctrs::vec_rbind(exposed_data, exposed_data2)
If dplyr::bind_rows()
were used instead, the attributes of exposed_data
only are shown, which is likely incorrect.
dplyr::bind_rows(exposed_data, exposed_data2)
In order to stack exposed_df
objects, the exposure period types and lengths must match. If they do not, an error will be thrown. For example, policy year exposure records cannot be combined with calendar month records.
Ordinary data frames can be stacked with exposed_df
objects using dplyr::bind_rows()
and rbind()
(assuming all column names match). If the exposed_df
object is the first argument, the exposed_df
class will be preserved with its original attributes.
exposed_df
class persistenceThe actxps package includes exposed_df
methods for the dplyr verbs listed below. These methods ensure that the functions below will always return an exposed_df
object.
dplyr::select()
dplyr::mutate()
dplyr::filter()
dplyr::arrange()
dplyr::group_by()
dplyr::ungroup()
dplyr::slice()
dplyr::rename()
dplyr::relocate()
dplyr::left_join()
dplyr::right_join()
dplyr::inner_join()
dplyr::full_join()
dplyr::semi_join()
dplyr::anti_join()
Generally speaking, any dplyr verbs that aren't listed that return data frames will preserve the exposed_df
class as long as the data is not grouped. If the data is grouped, the exposed_df
class may not persist. If this creates problems with your code, there are two options:
ungroup()
the data, call the function, and restore the groups with group_by()
.exposed_df
using as_exposed_df()
.The expose()
family does not support studies with multiple changes between an active status and an inactive status.
Any scripts or data that you put into this service are public.
Add the following code to your website.
For more information on customizing the embed code, read Embedding Snippets.