pp.testing: Testing the predictive power of PD rating model

In PDtoolkit: Collection of Tools for PD Rating Model Development and Validation

Testing the predictive power of PD rating model

Description

pp.testing performs testing of predictive power of the PD rating model. This procedure should be applied on the level of the rating scale. Four tests are implemented: the binomial, Jeffreys, z-score and Hosmer-Lemeshow test. Only the Hosmer-Lemeshow test refers to complete rating scale, while the remaining three are implemented on the rating grade level. The null hypothesis for the binomial, Jeffreys, and z-score tests is that the observed default rate \frac{n_b}{n_o} is less or equal to the calibrated PD (pdc) while for the Hosmer-Lemeshow test is that the calibrated PD (pdc) is the true one.

Usage

pp.testing(rating.label, pdc, no, nb, alpha = 0.05)

Arguments

rating.label	Vector of rating labels.
pdc	Vector of calibrated probabilities of default (PD).
no	Number of observations per rating grade.
nb	Number of defaults (bad cases) per rating grade.
alpha	Significance level of p-value for implemented tests. Default is 0.05.

Details

Due to the fact that test of predictive power is usually implemented on the application portfolio, certain prerequisites are needed to be fulfilled. In the first place model should be developed and rating scale should be formed. In order to reflect appropriate role and right moment of tests application, presented simplified example covers all steps before test implementation.

Value

The command pp.testing returns a data frame with input parameters along with p-value for each implemented test and the accepted hypothesis. Due to the fact that Hosmer-Lemeshow test is applied to complete rating scale, returned p-values are all equal between the rating grades as well as the test results.

References

Tasche, D. (2008). Validation of internal rating systems and PD estimates, The Analytics of Risk Model Validation, Quantitative Finance, Elsevier B.V., \Sexpr[results=rd]{tools:::Rd_expr_doi("10.1016/B978-075068158-2.50014-7")}.
Oesterreichische Nationalbank (2004). Rating Models and Validation, Oesterreichische Nationalbank (OeNB).

Examples

suppressMessages(library(PDtoolkit))
data(loans)
#estimate some dummy model
mod.frm <- `Creditability` ~ `Account Balance` + `Duration of Credit (month)` +
				`Age (years)`
lr.mod <- glm(mod.frm, family = "binomial", data = loans)
summary(lr.mod)$coefficients
#model predictions
loans$pred <-  unname(predict(lr.mod, type = "response", newdata = loans))
#scale probabilities
loans$score <- scaled.score(probs = loans$pred, score = 600, odd = 50/1, pdo = 20)
#group scores into rating
loans$rating <- sts.bin(x = round(loans$score), y = loans$Creditability, y.type = "bina")[[2]]
#create rating scale
rs <- loans %>%
group_by(rating) %>%
summarise(no = n(),
	    nb = sum(Creditability),
	    ng = sum(1 - Creditability)) %>%
mutate(dr = nb / no)
rs
#calcualte portfolio default rate
sum(rs$dr * rs$no / sum(rs$no))
#calibrate rating scale to central tendency of 27% with minimum PD of 5%
ct <- 0.33
min.pd <- 0.05
rs$pd <- rs.calibration(rs = rs, 
			dr = "dr", 
			w = "no", 
			ct = ct, 
			min.pd = min.pd,
			method = "log.odds.ab")[[1]]
#checks
rs
sum(rs$pd * rs$no / sum(rs$no))
#simulate some dummy application portfolio
set.seed(11)
app.port <- loans[sample(1:nrow(loans), 400), ]
#summarise application portfolio on rating level
ap.summary <- app.port %>%
	  group_by(rating) %>%
	  summarise(no = n(),
			nb = sum(Creditability),
			ng = sum(1 - Creditability)) %>%
	  mutate(dr = nb / no)
#bring calibrated pd as a based for predictive power testing
ap.summary <- merge(rs[, c("rating", "pd")], ap.summary, by = "rating", all.x = TRUE)
ap.summary
#perform predictive power testing
pp.res <- pp.testing(rating.label = ap.summary$rating,
		     pdc = ap.summary$pd,
		     no = ap.summary$no,
		     nb = ap.summary$nb, 
		     alpha = 0.05)
pp.res

PDtoolkit documentation built on Sept. 20, 2023, 9:06 a.m.