VALIDATION TOOLS FOR RATING AND SCORING MODELS

Abstract

According to the IRB approach under Basel II Accord, credit risk can be measured and managed as the probability of default (PD) with risk weights for different asset classes. For this purpose, Mercer Oliver Wyman provides the OCBC bank a Validation Master-Guide for the rating and scoring (PD) models and two supplementary excel PD tools. We focus on the two major criteria for the PD models i.e. discriminatory power (or rank-ordering performance) and calibration to examine the validation tools. We theoretically show that the mathematical foundations are sound, and use test cases in the independently built spreadsheet models to numerically validate that the PD tools are coded correctly. We analyse the accuracy ratio as the main rank-ordering performance measure (ROPM) and detail the test of ROPM during tool development and tool validation. At a factor that causes deterioration of performance, we further examine the detailed analysis for stability and concentration, relative risk level check for change of a risk profile, and model weight review for change of relative importance at a factor. We consider an alternative way besides matching matrices to compare the internal ratings with available external data from rating agencies, especially in the low-default scenario. We consider Spearman, Kendall™s Tau rank correlations and Mean Square Error, and propose appropriate benchmarks for the three measures. Results show that the validation tools are sound and the objectives are met.