A Macrofinancial Approach to Supervisory Standards Assessments

Standards assessments serve several important objectives but are not well integrated into Fund surveillance. Financial standards assessments, when undertaken in the context of FSAPs, are used to identify weaknesses in financial regulation and supervision, or other areas covered by international standards. However, those weaknesses are not specifically linked to the risks and vulnerabilities facing the financial sector. Conversely, the analysis of country-specific vulnerabilities in the FSAP does not contribute to targeting the standard assessment effort, since the assessment must be exhaustive and cover the entire standard.


Standards assessments serve several important objectives but are not well integrated into Fund surveillance. Financial standards assessments, when undertaken in the context of FSAPs, are used to identify weaknesses in financial regulation and supervision, or other areas covered by international standards. However, those weaknesses are not specifically linked to the risks and vulnerabilities facing the financial sector. Conversely, the analysis of country-specific vulnerabilities in the FSAP does not contribute to targeting the standard assessment effort, since the assessment must be exhaustive and cover the entire standard.

The Case for Mapping Macrofinancial Risks into Supervisory Standards

1. Standards assessments serve several important objectives for the Fund, as well as for countries and other stakeholders. For the Fund, financial standards assessments,1 when undertaken in the context of FSAPs, are used to identify areas of weakness in financial regulation and supervision (or other areas of financial infrastructure or safety nets covered by international standards). Outside the FSAP, they provide a comprehensive assessment that yields useful cross-country perspectives in regulatory and supervisory practices. For the countries, standards assessments provide an objective view of the quality of their regulation, supervision, infrastructure, or safety nets: in some cases, they reveal gaps that had not been previously identified; while in others, they help mobilize support for changes that are planned or under consideration. Assessments also often help identify technical assistance needs in individual countries. The international standard-setting bodies also rely on standards assessments for a comparative view of the level of compliance with their principles across countries. The assessment findings may also encourage standard-setting bodies to develop new policy in areas where the standards have been found to be lacking in granularity.

2. The three supervisory standards (for banking, insurance, and securities) are the most frequently assessed, and the average number of formal assessments undertaken in the context of FSAPs has fluctuated widely over time. Historically, the evaluation of the quality of supervision or other aspects of the financial system in most FSAPs was often done not through formal standards assessments (DARs and ROSCs) but through informal targeted assessments (summarized in FSAP Technical Notes). The average number of formal assessments of all financial sector standards (excluding AML/CFT) up to the financial crisis was less than one (Table 1).2 Following the financial crisis and widespread revisions in international standards by standard-setters, the average number of ROSCs per FSAP tripled to two-and-a-half, peaking in FY12 before starting to decline again. There are some differences in the frequency of formal standards assessments across groups of countries (systemic versus non-systemic)3 and regions, but they are relatively small. The most significant differences are in the frequency with which different standards are being formally ssessed (Figure 1): the three supervisory standards for banking (Basel Core Principles—BCP), insurance (Insurance Core Principles—ICP), and securities (Principles on Securities Regulation— IOSCO) are by far the most frequently assessed; and within these, BCP is the most prominent.

Figure 1.
Figure 1.

Evolution of Standards Assessments Conducted since the Last FSAP Review

Citation: Policy Papers 2014, 068; 10.5089/9781498342889.007.A001

Source: FSAP Tracking system.Note: S-25 indicates the 25 jurisdictions with systemically important financial systems.
Table 1.

Average Number of ROSCs Conducted During FSAP Missions by Standard

article image
Source: FSAP Tracking System.Note: Figures exclude stand-alone ROSCs and—prior to 2008—OFC assessments. CPSS and CPSS/IOSCO are now part of a new standard on Financial Market Infrastructures (FMI).

3. Their benefits notwithstanding, formal standards assessments are hard to integrate into the financial stability analysis, diminishing the usefulness of the exercise for FSAPs and, more broadly, for Fund surveillance. The methodology for formal assessments does not distinguish between individual principles based on their relevance for financial stability. Weaknesses in regulation and supervision identified in standards assessments are not linked to the risks and vulnerabilities they entail for the financial sector. Conversely, the analysis of country-specific vulnerabilities in the FSAP does not contribute to targeting the standard assessment work, which is supposed to cover the entire standard. Formal assessments are by design exhaustive and require intense expert interpretation to weigh the findings in terms of compliance. The IOSCO standard, in particular, places a much larger weight on market integrity and business conduct than on overall financial stability, even compared to the other two supervisory standards. DARs/ROSCs have become very long and detailed, and their recommendations are not prioritized based on the financial stability risks implied by the identified gaps. All this was noted in the 2011 Review of Standards and Codes Initiative, as well as the 2012 Financial Surveillance Strategy, which called for upgrading financial surveillance products to foster an integrated approach.

4. In addition, financial sector standards have become increasingly complex and assessments much more resource-intensive. In the aftermath of the crisis, revisions to the core principles and methodologies have extended the length of the assessment process and the final product (DARs). As a result, the human resource and dollar cost of producing assessments has increased considerably. Assessments now require the presence of additional assessors’ time (for up to three weeks) in the field and more extensive review. This creates concerns about the Fund’s ability to continue delivering high-quality assessments in the current resource environment. More importantly, it raises the question whether the amount of time and resources devoted to formal standards assessments is out of proportion with their net benefit for the FSAP and, more broadly, Fund surveillance.

5. A previous attempt to streamline standards assessments opened the door to more flexible, targeted assessments but has not yielded tangible results. The “Revised Approach to Regulation and Supervision Standards Assessments” in 2009 allowed for “targeted” (partial) assessments of a subset of principles in a standard, in order to focus the assessment effort and make the output more user-friendly and effective in the context of an FSAP. However, the criteria for selecting the subgroup of principles to be assessed were again compliance-based, unrelated to any macrofinancial considerations. And the conditions under which targeted assessments could be undertaken were set so narrowly that since 2009, only two such assessments have taken place.4 It should also be noted that the (relatively recent) standard on Financial Market Infrastructures (FMI) has a flexible approach already built in the assessment methodology: dependent on resources, the assessors (the Fund and/or the Bank) may identify, in cooperation with the country, a selection of FMIs and/or Principles for the assessment (Report to the Executive Boards of the IMF and the World Bank on the New CPSS-IOSCO Principles for Financial Market Infrastructure, July 2012).

6. A large body of work has analyzed what type of regulations and supervisory infrastructures are most effective in enhancing financial stability and improving financial sector efficiency (e.g., Barth, Caprio and Levine, 2004 and 2006). While this literature is not without shortcomings, the general finding is that a configuration of rules that encourage markets to discipline banks (including through regular and accurate public disclosure, supported by sound corporate governance and effective legal tools to seek redress), combined with judicious limits on the public safety net provided to banks, are most effective in fostering banking systems that are both stable and efficient. On the other hand, the connections between financial stability and compliance with international supervisory standards have, to date, been less clearly documented (the following section presents a more detailed review of the empirical literature on that score).

7. This paper proposes an approach aimed at (i) focusing standards assessments in FSAPs on macrofinancially-relevant principles, and (ii) incorporating their findings systematically into the FSAP’s overall risk assessment.

  • First, a subset of principles to be assessed in an FSAP—either formally in a DAR/ROSC or informally—should be selected based on the macrofinancial risks relevant for the stability of the particular financial system at a particular point in time. In essence, the proposed approach would build on the existing “targeted ROSC” model by replacing the compliance-based criteria for selecting principles to be assessed by macro-relevant and risk-based criteria.

  • Second, on the basis of the assessment of compliance with these principles, the FSAP team should judge whether the quality of regulation and supervision in these areas mitigates or aggravates the risks, and report this into an expanded version of the RAM. This would guarantee full integration of the supervisory findings into the risk assessment, make it easier to prioritize policy recommendations.

8. The proposed approach is limited to assessments of supervisory standards undertaken in the context of FSAPs or stability modules; it does not rule out full compliance assessments in other circumstances. At this stage, the analysis in this paper covers only the three supervisory standards, which are the most commonly assessed in FSAPs. It may be extended to other standards in the future. Even for these three supervisory standards, full assessments would continue, as at present, to be available to member countries. Full assessments would be relevant, for example, in cases where there has never been an assessment of the relevant standard; or there have been fundamental changes in the supervisory architecture since the last assessment; or the country has requested technical assistance that, in the staff’s and authorities’ view, would benefit from a prior full compliance assessment. These full assessments would be undertaken on a stand-alone basis (although they could also be combined with an FSAP if an FSAP happens to be planned at the time).

9. The proposed approach could be considered by the Board in the context of the next review of the Standards and Codes initiative. The intention is to discuss this approach with all relevant stakeholders, including standard-setters and the World Bank, with a view to arriving at a decision point at the next Board review of the Standards and Codes initiative planned for 2016.

The Empirical Methodology

The methodology for mapping macrofinancial risks into supervisory standards is a three-stage process. In the first stage, proxies for the macrofinancial risks relevant for the banking, insurance, and securities sectors are identified. In the second stage, the individual principles in each of the three standards are prepared to be used in the empirical exercise. In the third stage, explained in detail in Appendix II, both parametric and non-parametric econometric approaches are used to establish the links between macrofinancial risks and individual principles. Although the first two stages are preparatory for the empirical investigation in the third stage, they are essential parts of the proposed methodology.

A. Macrofinancial Risks

10. The point of departure is the identification of relevant macrofinancial risks for the banking, insurance, and securities sectors. Macrofinancial risks originate from or are transmitted to the financial sector in adverse macroeconomic scenarios, and can threaten the viability of individual institutions or the system as a whole.

  • Risks to the banking sector. The risks are classified into the four standard categories: credit, market, liquidity, and contagion risks.

  • Risks to the insurance sector. Insurance companies collect revenue in the form of insurance premiums and invest in cash assets, bonds, equity, real estate, and other assets. Their liabilities consist mainly of technical provisions—the estimated value of future claims associated with the underwritten insurance policies. These activities pose risks, as insurance companies could become insolvent if the value of their investments declines sufficiently, or if technical provisions turn out to be insufficient to cover insurance claims. The risks are accordingly classified into four categories: market/investment risks, insurance and underwriting risks, liquidity risks, and contagion risks.

  • Risks to the securities markets. The regulation of securities markets cover (i) the issuers of securities and the financial intermediaries that operate in those markets; (ii) the operation of collective investment schemes; and (iii) securities markets. Accordingly, risks have been classified into credit risk, liquidity risk, and market risk. Contagion risk across domestic markets and across borders has also been included in the analysis.

11. The empirical approach defines specific measures for each type of risk. One or more measures of individual risks for each of the three sectors are constructed based on available bank-, insurance-, and securities-specific information. In the case of banks and insurance companies, the data used to measure risk come from balance sheets and income statements of 1,426 individual banks in 83 countries and 848 insurance companies in 23 countries, respectively.5 For the securities markets, data come from market sources covering five market categories (money markets, debt markets, equity markets, financial intermediaries, and foreign exchange markets), as well as market estimates of default rates for securities issuers in the corporate, banking, and non-banking financial sectors for 77 countries. Global financial market variables and a broad range of estimated risk premia are also included in the analysis (all data used in this paper are available upon request).

B. The Supervisory Standards Principles Dataset

12. A quantitative analysis of compliance requires a scoring function that maps the “grades” given in standards assessments into quantitative measures. We use a linear scoring function assigning numerical values of 1 through 4 to the compliance grades.6 The proportionality principle7 establishes a moving hurdle rate to obtain a compliant grading. For instance, a jurisdiction that is home to many SIBs will naturally have to clear a higher hurdle in order to obtain a compliant grading compared to a jurisdiction that only has small, non-complex deposit-taking institutions. Under the assumption that the quality of supervision increases with the complexity of the financial system, the score function may not generate enough dispersion across jurisdictions as quality and hurdle rate move in the same direction. Figure 2 suggests that most of the grades are concentrated in the compliant and largely compliant categories in the BCP standards, and that more advanced countries have generally better scores in all standards.

Figure 2.
Figure 2.

Dispersion of Compliance Among Countries

Citation: Policy Papers 2014, 068; 10.5089/9781498342889.007.A001

Source: Staff estimates.Note: Vertical axes indicate number of FSAPs and horizontal axes the standards principles. BCP principles are expressed in terms of the 2006 methodology. Colors indicate principles’ grades, with higher numbers representing a lower degree of compliance. More details on the quantitative grading scale are presented in footnote 5.

13. For each standard, the number of principles assessed is large, and their ratings tend to be highly correlated. The core principles were established by the three main standard setters BCBS, IAIS, and IOSCO. The number of principles per standard is large: BCBS adopted 29 BCPs in 2012, IAIS issued 26 ICPs in 2011, and IOSCO 38 principles in 2010 (revised in 2011).8 Moreover, many pairwise correlations of principles’ grades are in excess of 50 percent,9 pointing at insufficient differentiation—countries that receive high ratings in one principle tend to receive high ratings in other principles. Unconditional average correlations among principles are 34 percent for BCP, 39 percent for ICP, and 26 percent for IOSCO. Correlations among principles’ grades are high even if principles do not belong to the same thematic grouping.10 High correlation among principles’ grades could introduce multicollinearity, posing technical challenges in the regression analysis.

14. The assessment methodologies have evolved over time and their application has become more conservative post-crisis. The principles and methodologies for standards assessments have changed over time to reflect developments in financial markets. Moreover, changes in the macrofinancial environment seem to have influenced the experts’ assessment of specific principles in practice. Table 2 shows the ranking of demeaned average compliance grades for the set of countries assessed under the 1999 BCP methodology and the merged 2006 and 2012 methodologies, with a lower score reflecting higher compliance. Countries that had received favorable grades in earlier FSAPs generally got lower grades in post-crisis assessments. These time-varying factors may not be fully captured in the empirical analysis, and hence the task of capturing the linkages between effectiveness of regulation and supervision, and financial risks may not be straightforward. Merging different versions of the standards could also introduce noise in the empirical analysis.11

Table 2.

Ranking of Compliance Across Time

article image
Note: Ranking based on average demeaned BCP compliance ratings.Source: Staff estimates.

C. The Methodology

15. The third stage of the analysis establishes the empirical link between macrofinancial risks and compliance with individual principles. It includes a factor analysis to reduce the dimensionality of the standards dataset and both parametric and non-parametric approaches to assure the robustness of the results. The technical aspects of the methodology are discussed in more detail in Appendix II.

16. Previous attempts to measure the effects of regulatory and supervisory quality on financial stability have had mixed results. Existing studies transform the qualitative four-grade scale into an overall index of compliance as the sum or average score across all the principles in a standard. They find little or no evidence of a connection between those compliance indexes and financial stability, although there is some evidence that compliance with BCPs reduces specific measures of risk in some economies and time periods. Evidence on the effects of compliance with IOSCO and ICP principles on financial stability is even more sparse

17. However, compliance measures typically used in the literature have several shortcomings. Most notably, the studies do not exploit the granularity of information from past assessments. Since they mostly rely only on summary statistics, such as the sum or the mean of the distribution grades across scored principles, the studies cannot differentiate among individual principles, identify those that are most relevant for financial stability, and take account of non-linear effects on financial stability (Box 1).

18. The proposed econometric methodology is superior to previous approaches. The paper proposes a more granular approach to evaluate the connection between compliance with subsets of principles and macro-financial risks:

  • It provides a rigorous analytical basis for selecting the macrofinancially-relevant principles by identifying the principles that are most relevant for each type of financial risk. The econometric technique also controls for different levels of development.

  • It addresses the multicollinearity problem using two approaches. First, it constructs a battery of regressions which allows carrying out a "horse-race" comparing the contribution of different principles to financial stability. Second, it uses factor analysis to reduce the number of explanatory factors while preserving as much information as possible of the original database. Therefore, the methodology uses both individual principles, as well as a reduced number of (unobserved) factors that span the individual principles, to mitigate the multicollinearity problem.

  • It captures non-linear effects of grades. Principles are evaluated and compared based on their relative performance in explaining financial stability with non-parametric as well as parametric econometric techniques. Both direct and indirect effects (through their interaction effect with macrofinancial risks) are considered in the analysis. The non-parametric approach allows endogenizing the principle-specific thresholds of compliance that are most relevant for mitigating financial risks. It also shows the compound effects on financial stability from weak compliance across principles.

  • It expands the sample coverage by merging countries assessed under the last two methodologies for each standard using an expert judgment mapping. Noise is reduced by focusing on the most recent methodological revisions (Appendix I).

Literature Review and Pitfalls

The existing literature finds at best mixed evidence on the correlation between compliance with international standards and financial stability:

  • The empirical evidence is mixed and inconsistent across groups of countries and time periods. For instance, the 2011 Review of Standards and Codes found that compliance with BCP standards is significant and with the expected sign in explaining financial stress only in 2009 (not in 2008), and only for selected advanced economies (not for emerging markets). Compliance with IOSCO standards is significant only for a sub-index of financial stress in securities markets of advanced economies. These results do not extend to emerging markets or to insurance or corporate governance standards.

  • The evidence on the effects of compliance with principles on bank stability is weak and depends on the individual risk or the specific stability measures analyzed. For example, Demirgüç-Kunt and Detragiache (2010) show that compliance with the overall BCP index is insignificant in explaining banks’ Z-score, whereas compliance with Chapter 2 BCP sub-index (on supervisors’ powers) is negative. On the other hand, Podpiera (2004) finds some evidence that higher BCP compliance leads to lower NPLs and lower net interest margins on an aggregate basis. Maechler et al. (2007) find that better BCP compliance reduces the positive effect of credit growth on banks’ Z-score whereas a higher score on CPSS compliance enhances the positive effect of liquidity on banks’ soundness. Das et al. (2005) show that the positive effect of macroeconomic stability on a country’s financial stress index is more pronounced the lower the index of BCP and IOSCO.

  • Existing studies ignore granular information and suffer from several shortcomings:

  • The distribution of grades is disregarded. The sum or the mean of the distribution of the scored principles are—in most cases—the only statistical measures used. Hence, it is not possible to identify or differentiate the individual principles that are most important for financial stability.

  • Deistances across grades are treated as identical. Qualitative categories (grades) are transformed into quantitative scores using a linear scale that assumes equal distance across categories. However, the mapping of categorical ranks to scores is non-linear. For instance, BCP grades 1 (compliant) and 2 (largely compliant) reveal an adequate degree of compliance with the principle, whereas grades 3 (materially non-compliant) and 4 (non-compliant) signal more serious problems: it might be argued that the ‘distance’ between 1 and 2 is smaller than that between 2 and 3. Also, distances between grades are not necessarily consistent across standards (for instance, the difference between a Largely Compliant and a Materially Non-Compliant in the BCP assessment appears in practice to be larger than that between a Broadly Implemented and a Partly Implemented grade in the IOSCO standard).

  • Asymmetric effects on financial stability and the effects of combined weaknesses across principles are ignored. The impact of the score on financial stability is assumed to be linear, overlooking the potential for asymmetric effects. Moreover, non-compliance against a single principle may not be sufficient to trigger financial instability; however, combined non-compliance against several principles could exacerbate financial vulnerabilities in adverse macro scenarios.

Factor Analysis

19. We use an innovative data reduction procedure to choose subsets of principles, while preserving as much information as possible from the original database. This way, we address the multidimensional nature of standards assessments. We use a combination of principal component analysis (PCA) and exploratory factor analysis (EFA)12 to identify which principles form coherent subsets or factors that are relatively independent of one another (Appendix II). PCA combines into factors the principles that are correlated with one another but largely independent of other subsets of principles. Then EFA allows factor rotation to make sharper distinctions in the contribution of each underlying core principle to the factors.

Parametric Approach

20. For each measure of risk, a “benchmark” parametric model is estimated. For BCP and ICP regressions we use bank and insurance level annual data respectively. For IOSCO principles we use aggregate monthly market data. The benchmark models, in which banks, insurance companies, or securities markets are nested within countries, is defined as follows:


where Yijtdenotes the specific risk analyzed, i denotes country, j denotes the individual bank or insurance company, or market, and t is time (year). The model includes bank-or insurance company-specific effects µj and time-varying controls Xijt. It also includes a set of country-specific macro-financial determinants MACRO it. Note that the benchmark model (1) does not incorporate individual principles or factors. In the empirical analysis, we consider whether the model is correctly specified as dynamic or static—in the latter case, the lagged dependent variable is excluded from the right-hand side of equation (1). When appropriate, we estimate dynamic versions of the model using the Arellano-Bond approach. Benchmark regression results are in Appendix II.

21. The benchmark parametric model is expanded to assess the effects of compliance with core principles or factors on risks. The analysis uses as input either scores on individual principles or factor scores that summarize information from individual principles. The benchmark regressions estimated for BCP, ICP, and IOSCO are expanded to include interaction terms between macro-fundamentals and individual principles or factors F :


Note that F is time-invariant and country-specific. Model (2) was estimated including either single or multiple interaction terms.

22. Once the models are constructed, a search approach is used to select the principles that are most relevant and robust. The number of BCP, ICP, and IOSCO principles is large and their interactions with macro variables cannot be simultaneously included in a regression. A multi-regression specification search approach is needed to select the individual principles.13 The relevant principles or factors are selected based on the following criteria:

  • Sign and significance of the interaction effects (parameter). A higher score in a principle or factor indicates a lower degree of compliance. Hence, in regressions including single macro-principle or macro-factor interactions, the risk analyzed should be exacerbated by a higher score in the principle or factor in “adverse” macroeconomic scenarios. In addition, the interaction of the principle or factor with macroeconomic variables must be statistically significant (at the 10 percent level).

  • Robustness to inclusion of additional interactions. The sign and significance of the interaction effect must be “robust” to the inclusion of one or more interaction terms between other principles and the same macro variable, and between other principles and other macroeconomic variables.

  • Ability to improve forecasts of looming financial risks for market data. For IOSCO, the panel regressions are estimated using rolling windows drawing on monthly market data.14 Principles are selected in a “horse race” that ranks principles according to their capacity to minimize forecasting errors.

Non-parametric Approach

23. A non-parametric approach is used to unravel the complex interactions between macroeconomic variables, risk factors, and regulatory compliance. The global financial crisis has highlighted that non-linear interactions among variables and perverse feedback loops can magnify financial instability. The appeal of non-parametric methods stems from the fact that they relax the parametric assumptions imposed on the model. Rather than specifying a “benchmark” regression, the non-parametric approach includes both macrofinancial determinants and principles’ grades in the panel specification and let the data reveal the most appropriate functional form.

24. We perform a binary tree analysis to identify the links among explanatory variables and principles’ compliance ratings that help predict financial risk levels.15 The binary tree analysis recognizes that it is the combination of economic factors and supervisory quality (as measured by compliance grades) that is crucial for financial stability. This methodology is particularly well suited to examine the propagation of risk under stressful conditions. It also allows identifying the specific thresholds of supervisory quality that are effective in mitigating risk in the financial system. At each node of the tree, this analysis identifies the variable and its particular threshold that best divides observations into a high-level and a low-level measure of financial risk— henceforth the “splitter”. We use an algorithm to construct a score for each variable as a function of its ability to perform as a “splitter” at every node of the tree, placing a greater weight in the results from nodes that represent adverse macroeconomic conditions. In this way, the non-parametric approach captures potential non-linearities in the relationship between compliance with individual principles and financial stability, which are not possible to capture in the parametric estimation. The principles that have the highest score based on this algorithm are the most important for financial stability, in the sense that better (worse) compliance with these principles has a substantial mitigating (aggravating) impact on financial stability variables, particularly in stressful macroeconomic conditions.

25. Figure 3 provides an illustration of the process of identifying “splitter” variables and their threshold. In this example, credit risk is proxied by the NPL ratio.16 Starting from the top, the analysis starts by distinguishing among countries showing low/high interest rates and low/high fiscal deficits. In the set of observations with high fiscal deficits, for example, when interest rates are low (below 6.5 percent), compliance with CP12 appears to be particularly relevant: on average, the NPL ratio is higher by some 5 percentage points when CP12 is graded as “Materially Non-Compliant” or “Non Compliant.” If interest rates are higher than 6.5 percent (though lower than 13.8 percent), compliance with CP13 becomes more relevant in explaining explain banks’ NPL ratios: the average NPL ratio goes from 5.3 percent to 8.7 percent when the grade is less than “Compliant.” By repeating this exercise for all principles across all macroeconomics variables, principles are assigned a score based on their significance as a “splitter”, with greater weight placed on observations involving values of macroeconomic variables indicating macroeconomic stress. The top-ranked principles are the ones that are consistently able to forecast the financial stability proxies across a range of macroeconomic conditions.

Figure 3.
Figure 3.

Binary Tree Including BCPs

Citation: Policy Papers 2014, 068; 10.5089/9781498342889.007.A001

Source: Staff estimatesNote: Binary classification tree for credit risk (NPL ratio) performed on the set of 25 advanced economies assessed against 2006 and 2012 BCP principles over 1995-2012 including a total of 319 observations. The grey shaded areas denote the main tree splits (indicating the relevant macroeconomic variables and core principles along with their endogenous threshold). The boxes show the average NPL ratio of the observations included in each split. The following notation is used: IRDEP (interest rate of deposits), FB (fiscal balance), and INFL (inflation). The figure shows the upper part of the optimal binary tree for credit risk (17 nodes). The complete binary tree contains 35 nodes. The set of regressors is consistent with the selection of

D. Empirical Results

26. The macrofinancially-relevant principles are chosen by the union of parametric and non-parametric results. The underlying idea is that the set of parametric results using factor analysis are complemented by non-parametric analysis, in which all individual principles are included. The latter approach addresses the concern that, as factor analysis is selects a limited number of factors to reduce the dimensionality of the data, some individual principles that do not co-move significantly with the rest may be excluded.17 Therefore, if a particular principle is significant in both the parametric and non-parametric analysis, it is included in the final list.

27. The analysis identifies 19 BCP principles, 15 ICP principles, and 21 IOSCO principles. The selected principles, listed in the first column of Table 3, are distinguished between those for which there is strong empirical evidence (statistically significant correlation in all or most parametric and non-parametric approaches) and weak empirical evidence (statistically significant in some parametric or non-parametric approaches). The Table also indicates principles that were not included in the empirical investigation due to lack of sufficient data (e.g., principles that were only recently added to the standard). The result suggests strongly that a large number of regulatory principles across all three standards are not macrofinancially-relevant.

Table 3.

Selected Principles by Standard

article image
Source: Staff estimates.Note: Dark green indicates strong econometric evidence (statistically significant in most parametric and non-parametric approaches), light green indicates weak empirical evidence (statistically significant in some parametric or non-parametric approaches), and blue indicates principle not assessed under the econometric approach.
Table 3.

Selected Principles by Standard (continued)

article image
Source: Staff estimates.Note: Dark green indicates strong econometric evidence (statistically significant in most parametric and non-parametric approaches), light green indicates weak empirical evidence (statistically significant in some parametric or non-parametric approaches), and blue indicates principle not assessed under the econometric approach.
Table 3.

Selected Principles by Standard (concluded)

article image
Source: Staff estimates.Note: Dark green indicates strong econometric evidence (statistically significant in most parametric and non-parametric approaches), light green indicates weak empirical evidence (statistically significant in some parametric or non-parametric approaches), and blue indicates principle not assessed under the econometric approach.

Principles 10 and 11 are included because their assessment is a necessary input to the assessment of Principle 13.

Although included in the expert selection, it is expected that this Principle would be relevant in only some jurisdictions, in which it may be included in the assessment on a case-by-case basis.

28. This empirical methodology also permits the association of selected principles with specific risks. Figure 4 shows the selected principles for each of the macrofinancial risks by standard. Not surprisingly, many principles are relevant for more than one risk.

Figure 4.
Figure 4.

Macrofinancially Relevant Principles by Financial Risk for each Standard

2012 BCP Principles

Citation: Policy Papers 2014, 068; 10.5089/9781498342889.007.A001

Source: Staff estimates

Combining Empirical Results and Expert Judgment in Selecting Macrofinancially-Relevant Principles

Empirical and expert approaches are combined to select a set of macrofinancially-relevant principles.

A. The Expert Judgment Approach

Expert judgment has a key role to play in identifying individual principles that are particularly important for financial stability. Given the frequent revisions of the standards and the nature of the assessment process—as well as the limitations of the empirical approaches discussed earlier—direct input from supervision and regulation experts can provide important perspectives in the selection of principles that are more relevant for financial stability. Experience gained through many years of compliance assessments in different countries can also provide important context that cannot be captured in the econometric exercise. Though inevitably judgmental and, to some extent, subjective, this input is crucial for the integrity of the exercise. This section presents the subset of principles in each of the three supervisory standards identified by MCM experts as relevant for financial stability, with a brief explanation. These principles are not associated with individual risks, as in the econometric approach, but are deemed to be the most relevant for financial stability writ large.

Basel Core Principles

30. BCPs include some important foundational principle (BCPs 1 and 2) and gateways (BCP5), which if missing or limited, undermine the supervisory function as a whole. Without adequacy of resources and legal powers, the supervisory authority is unable to act. Clarity of objectives, responsibilities, independence from interference, and legal protection are also essential to ensure the supervisor is willing to act. The use of corrective powers (BCP11) examines whether the supervisor acts to forestall emerging issues. Licensing (BCP5) can be seen as a “gateway” principle as it tests whether the authorities act to ensure that only sound enterprises are allowed to enter the banking system. The standards set in this principle must be maintained in ongoing supervision for all banks. Even in systems that have few or no new entrants, the principle captures whether the minimum expectations that banks must meet have been set at a suitable level or whether there is probable scope for vulnerabilities to emerge.

31. While there is no perfect supervisory approach, the lack of certain key features in a supervisory approach is likely to lead to macro-financial consequences. Principle BCP8 tests whether the authorities have a comprehensive, detailed, and forward looking understanding of their banks and the banking system as a whole. It also tests whether supervisors will intervene at an early stage and have the ability to take action to resolve banks in an orderly manner. If these elements are lacking, it is probable that supervisors will be slow to identify deterioration or emerging vulnerability and that they will be ill-placed to act effectively.

32. Although the crisis showed that banks with apparently strong capital could fail, the capital standard itself has been considerably strengthened post crisis. The principle addressing capital adequacy (BCP 16) examines one of the key inputs into any vulnerability analysis and stress testing. Moreover, it may be particularly relevant to advanced economies, where more banks are using internal ratings-based model approaches. Supervisory weaknesses in model review and approval could lead to highly unreliable assessments of capital adequacy.

33. The quality of risk governance within banks is critical to the stability of the individual institution and there are a number of principles which examine this aspect. The risk management process (BCP15) is an over-arching principle that establishes whether effective risk practices are in place. Principles addressing credit risk (BCP17), problem assets and provisions (BCP18), concentration risk (BCP19), and liquidity risk (BCP24) look more deeply at the effectiveness the management and control of these specific risks, which are central to sound banking business. While a strongly performing economy may mask weaknesses in identification and management of poor quality risk exposures, or of growing concentrations of risk, these principles—particularly in the revised 2012 format—provide the opportunity to discriminate between the banking systems that are more able and likely to identify and act upon weaknesses as they emerge. Another overarching principle, that of corporate governance (BCP14), provides insights into the understanding and quality of governance and management within an institution. Failures at board and management level have been responsible for a number of banking failures, but this is a new principle and therefore there is no data linked to it from past assessments.

34. Disclosure and transparency (BCP28) are fundamental tenets of a sound banking system. In particular, the provision of meaningful information about common key risk metrics to market participants reduces information asymmetry and helps promote comparability of banks’ risk profiles within and across jurisdictions. Regulatory disclosure requirements—such as those provided by Pillar 3 of the Basel framework—enable market participants to assess more effectively key information relating to a bank’s regulatory capital and risk exposures. It can also help them to promptly detect potential problems with a bank’s overall situation and, by this means, put market discipline in operation and allow it to contribute to financial stability.

Insurance Core Principles

35. For the insurance supervisory authorities, it is important to have adequate legal powers to conduct both prudential and business supervision to promote the maintenance of a fair, safe, and stable sector for the benefit and protection of policyholders. Similar to the BCP, adequacy of resources and independence from undue political and industry interference are essential to ensure that the supervisor has the will to act and use of its corrective powers (ICP2). The ICP on licensing (ICP4) is an important control mechanism to allow the entry of entities that will contribute to the financial stability and are able to fulfill their obligations to policyholders on an ongoing basis. ICP7 and ICP8 on governance and risk management are core to ensuring that the system of ‘checks and balances’ are working as intended. Lack of governance oversight, controls and management can lead to adverse macrofinancial consequences.

36. Weaknesses in supervisory intensity and effectiveness can lead to other macro-financial consequences. ICP9 tests whether the authorities have appropriate supervisory review and reporting mechanisms in place. The ICP takes a risk-based approach to evaluating the risk profile, conduct, and the quality of effectiveness of governance and compliance with supervisory requirements. It pays due attention to the evolving nature, scale, and complexity of risks posed by insurers and of the risks to which the insurers may be exposed. If the supervisory review and reporting is not intensive and effective, it is probable that the supervisors will not be able to respond to risks and vulnerabilities and enforce corrective actions (ICP11) in a timely manner. Similar to ICP9 being the basis for supervisory review, ICP24 forms the cornerstone of macroprudential surveillance for insurance. It requires supervisors to analyze, in a forward-looking manner, financial market developments and other exogenous factors which could have a negative impact on insurers’ solvency and profitability. This principle also demands a process for assessing the systemic importance of insurers and adequate supervisory responses.

37. An effective and timely resolution framework is critical for the stability of the insurance sector. Unlike in the case of banks, where a number of safety nets are in place—such as liquidity facility of central banks and deposit insurance schemes—those safety nets do not exist in many jurisdictions for the insurance sector. Therefore, effective resolution framework is the only tool to mitigate contagion risks to the wider financial sector from troubled insurance company failures. ICP12 (on winding-up and exit from the market) stipulates essential elements of insurance resolution, such as policyholder priority, timely provision of benefits to policyholders, and determination point of resolution.

38. Although there is no common global capital standard for insurers at the present time, the IAIS has been working on a basic capital requirement for G-SIIs and higher loss absorbency requirements for internationally active insurance groups. Until a global insurance capital standard is in place, the current solvency-related ICPs (ICP14 to 17) include important requirements for the valuation of assets and liabilities, investment activities, and capital standards for solvency purposes. These standards are the cornerstone of (i) risk identification and measurement, (ii) adequacy of technical provisions, and (iii) availability of capital to address all relevant and material risks underpinning robust insurance legal entity and group-wide solvency assessment.

39. Market conduct is an integral part of insurance supervision which aims to ensure consumers are treated fairly both before entering into a contract and through to the point at which all obligations have been satisfied (ICP19). Good conduct of business strengthens trust and consumer confidence in the insurance sector and contributes to overall financial stability. ICP19 supports a sound insurance sector by creating level playing field where insurers can compete while maintaining acceptable business practices with respect to fair treatment of customers.

38. With global interconnectedness and increasing presence of insurance groups and financial conglomerates, the importance of group-wide supervision is critical to the stability of the financial system. The importance of the insurance sector for financial stability matters has been increasing and any risk of contagion from one jurisdiction or sector to another could have serious stability implications. Group-wide supervision (ICP 23) applies to both the legal entity and insurance group or financial conglomerate to ensure that all relevant group-wide risks impacting the insurance entities are addressed appropriately. Failures of group-wide risk management and oversight have been responsible for many distressed insurers resulting in resolution and winding-up and exit from the markets.

IOSCO Principles

41. As with the BCP standard, the expert approach for the IOSCO Principles assessments builds on first ensuring that the regulator has sufficient independence, resources and powers and that its responsibilities are clearly defined. Without first assessing these basic Principles (P1-P3 and P10-P11), it is not possible to assess whether the regulator can effectively implement its supervisory and enforcement responsibilities (P12).

42. Collective investment schemes are important investment vehicles for both retail and institutional investors, including banks, insurers, and pension funds. Moreover, robust regulation and supervision (P24) and stringent requirements on client asset protection (P25), disclosure (P26), and asset valuation (P27) are essential to ensure that this interconnectedness does not lead to the transmission of financial shocks. The same applies to issuer disclosures and the quality of their financial reporting (P16 and P18). In some jurisdictions hedge funds are highly leveraged and interconnected, and assessment of this sector as potential source of systemic risk is important (P28). Overall, the securities regulator’s ability to monitor, mitigate and manage systemic risk is a key focus since the crisis (P6).

43. Market intermediaries are responsible for meeting the obligations arising from their own and their clients’ exposures in secondary markets. Robust supervision and capital and other prudential requirements is important to ensure that the related risks are controlled (P30-P31). In case of a failure, both the authorities and markets need to be able to effectively deal with the situation to avoid contagion (P32 and P37).

B. The Combined Set

44. There is remarkable overlap between the results of the two approaches. The two approaches are fundamentally different. The econometric approach is “context-free”: it is a purely data-driven, principle-by-principle empirical investigation, methodologically rigorous but agnostic as regards the content and evolution of individual principles. The expert approach, on the other hand, relies entirely on assessors’ a priori judgment and experience gained from a large number of compliance assessments over many years, without the benefit of any systematic quantitative evidence regarding the importance of individual principles. Nevertheless, the two approaches yield remarkably similar results: 70-90 percent of the principles identified as relevant by either approach in each of the three standards overlap with those identified by the other approach and, as a result, are part of the final set. This is a very reassuring result.

45. The final set of macrofinancially-relevant principles is the intersection of the empirical and expert approaches. Table 3 shows the list of principles selected by the empirical analysis in the first column and the expert judgment in the second column. The final list of principles in the last column of Table 3 and in Figure 4 includes principles that were both found to be macrofinancially-relevant in the econometric analysis and considered particularly important for financial stability by expert judgment. Also included are principles that were considered important for financial stability by the experts, but either were not included in the econometric analysis—due to lack of data—or needed to be assessed in conjunction with selected principles to provide an effective assessment. The final list includes 11 BCPs, 11 ICPs, and 17 IOSCO Principles.18

Integrating Assessment Results into the Financial Stability Analysis

Once the macrofinancially-relevant principles have been identified and the compliance assessment completed, the findings would be summarized in an expanded version of the RAM, thus becoming an integral part of the FSAP’s overall risk assessment.

46. The set of macrofinancially-relevant principles in each standard should be seen as the starting point for the assessment process in individual FSAPs. It is not meant as a hard rule but rather as a guide for focusing the assessment effort in areas of particular relevance. If the assessment of additional principles is seen as necessary from a financial stability point of view in the context of a particular country FSAP, these principles should also be included. The key point is that the selection be guided by financial stability considerations. This applies equally to cases where the assessment takes the form of a formal compliance assessment, summarized in a DAR/ROSC, or an informal assessment summarized in an FSAP Technical Note.

47. The results of the supervisory assessment conducted in the context of the FSAP will be integrated to the risk assessment through an “enhanced” RAM. On the basis of the (formal or informal) standards assessment, assessors and the rest of the FSAP team would arrive at a judgment of whether the overall quality of supervision and regulation in the relevant areas aggravates or mitigates the probability of realization and/or impact of the specific risks. This assessment will be explicitly included in the RAM, thus making it an integral part of the FSAP’s overall risk assessment.

Specifically, the structure of the RAM will be expanded to include:

  • How each macrofinancial shock identified in the RAM is likely to affect the financial sector (or relevant sub-sectors, like banking, insurance, or the securities market). This information is crucial to provide a link between the macro risks and the supervisory assessment. Although it is sometimes included in RAMs currently, this is not done systematically.

  • The team’s assessment on how the quality of supervision and regulation in the selected areas affects the “probability of realization” and the “expected impact.” The team’s judgment could be summarized as “Aggravating”, “Mitigating”, or “Neutral,” with a brief explanation.

Appendix III presents an illustrative example of how such an “enhanced” RAM would integrate the work of the assessment of compliance with the macrofinancially-relevant principles to the overall risk assessment of the FSAP.

49. There are risks associated with the proposed approach. Focusing the assessment effort on a subset of principles that are most relevant for the risks facing the country undergoing an FSAP would yield significant benefits in terms of focus, resource-efficiency, and effectiveness of the assessment. At the same time, it could create risks.

  • Incomplete coverage of the principles to be evaluated as a result of an incomplete identification of risks. Under this approach, the work of the experts will be framed by the views of the team on the financial risks faced by a particular jurisdiction. In practice, this means that the team will need to have a thorough understanding of the financial system of the country, the size and importance of different sectors and markets, the business model of different market participants and their level of interconnectedness. A substantial strengthening of the pre-mission preparation process—including scoping, data requests, and interaction between FSAP team and assessors—would be required. But even in that case, a residual risk that some important principles may be “missed” would still remain.

  • Fragmented view of the role of regulation and supervision in financial stability. While certain principles are more directly linked to specific risks, in practice many principles are interrelated. Therefore, a selective assessment of a subset of principles risks losing the holistic perspective. This risk was also noted in the targeted ROSC paper of 2009. One option to mitigate this risk would be to explore the extent to which certain principles (i.e., on independence, resources, powers) should be seen as essential and would need to be part of the mapping for any risk. However, while intuitively appealing, this option risks increasing the number of principles so much as to dilute the benefits of the targeted assessment.

  • Implementation risks: This approach would be a challenge for both economists and experts. The former would require a better understanding of financial sector issues in general, especially for non-bank sectors; the latter would need a more thorough understanding of the linkages between regulation and supervision and financial stability risks and vulnerabilities. This implementation risk could be mitigated via training, as well as more time provided for interaction between the FSAP team and the assessors on- and off-site. Particular attention would need to be paid to ensuring the efficiency of such interaction in FSAP teams that rely wholly or largely on external experts for the standards assessments.

Appendix I. Merging Supervisory Standards under Different Methodologies

50. We use an extended sample of ROSCs by merging core principles issued under the most recent methodologies by each standard-setter. Since they were originally issued, core principles for effective regulation and supervision have been reviewed and updated periodically to take account of significant developments in the financial sector, as well as in the regulatory landscape. BCP principles issued in 1997 were updated in 2006 and 2012. ICP principles issued in 1999 were reviewed in 2003 and 2011. IOSCO principles adopted in 2000 were revised in 2010. To expand the ROSC country coverage while minimizing biases from shortcomings in the mapping exercise, we focus on assessments conducted under the last two methodologies. We thus exclude ROSCs conducted under the 1997 BCP methodology as well as under the 1999 IAIS methodology, as major new insights and gaps in effective regulation have become apparent since then. The final sample includes 85 BCP assessments, 45 ICP assessments, and 77 IOSCO assessments. These assessments exclude offshore jurisdictions due to the lack of macroeconomic data.

51. The benchmark methodology is chosen to limit the noise created by merging different versions of the standards. Appendix Table I shows the mapping matrix broken down by standard. The conversion matrix has been informed by expert judgment. The choice of the benchmark methodology feeding into the econometric analysis has been guided by two considerations: (i) the relative country coverage of assessments under both methodologies; and (ii) the specifics of the conversion matrix. Whereas less than 10 percent of the assessments were conducted under the revised 2012 BCP methodology, over 30 percent of ICP assessments were carried out under the new 2011 ICP methodology, and close to 20 percent of IOSCO assessments were performed under the new 2010 IOSCO methodology. The first criterion thus favors the use of 2006 BCP, 2011 ICP, and 2010 IOSCO as the benchmark methodology. This choice is supported by the second criterion. In particular, noise is reduced when inputs feeding into a principle merge into a new principle rather than expanding into different principles. Appendix Table I confirms that this is achieved under the selected mapping.

Appendix I. Table 1.

Mapping of Standards under Different Methodologies

article image
Note: The mapping has been informed by expert judgment. The matrix is based solely on essential criteria laid out for each principle under the benchmark and the mapped methodology. Principles that are 100 percent new are excluded from the matrix. The only principle excluded from the 2012 BCP is the new CP14 (corporate governance).
Appendix I. Table 1.

Mapping of Standards under Different Methodologies (Continued)

article image
Note: The mapping has been informed by expert judgment. The matrix is based solely on essential criteria laid out for each principle under the benchmark and the mapped methodology. Principles that are 100 percent new are excluded from the matrix. These include: (i) ICP26 (cross-border cooperation and coordination on crisis management) for IAIS; and (ii) P6 (monitoring, mitigation, and management of systemic risk), P7 (review of the regulatory perimeter), P8 (conflicts of interest and misalignment of incentives), P19 (auditors' oversight), P20 (auditors' independence), P21 (auditing standards), P22 (credit rating agencies' oversight), P23 (regulation of other entities that offer investors evaluative services), and P28 (hedge funds' oversight) for IOSCO. P38 (regulation of securities settlement systems and CCPs) has also been excluded as it has never been assessed under the 2010 IOSCO methodology, and very infrequently under the 2000 IOSCO methodology.

Appendix II. The Methodology

This appendix contains technical background on the factor analysis used to reduce the dimensionality of the data, and provides the benchmark econometric specifications for the empirical analysis of the paper.

Factor Analysis

52. We use a formal variable reduction procedure to decrease the number of principles and address the correlation problem across grades. This is a key element for a robust econometric analysis. Factor analysis allows us to identify which principles form coherent subsets that are independent of one another. Principles with grades that are correlated with one another but independent of other subsets of principles are combined into factors.


53. We conduct factor analysis following a three-pronged approach:

  • First, we combine principal component analysis (PCA) and exploratory factor analysis (EFA) to reduce the dimensionality of the data and orthogonalize the dataset.

  • Second, we rotate the factors to make sharper distinctions in the meaning of the factors and the underlying contributing principles. The loadings of principles into the factors serve to inform the interpretation of the factors.

  • Third, we estimate the factor scores for each individual country from the rotated loadings using the Anderson method. Each factor score for an individual country is computed as a linear combination of the centered data for each principle.19 The estimated factor scores are used as regressors in the econometric analysis. We check that the factor analysis has served to orthogonalize the dataset.

54. We combine PCA and EFA to cluster principles into underlying factors. Both PCA and EFA are used as data dimensionality reduction techniques, because they allocate clustering principles into homogeneous sets allowing us to gain insight into which categories of principles move together in compliance ratings. Yet they exhibit different properties. PCA analyzes all of the variance across grades, including common variance among principles as well as variance unique to each principle, but does not allow rotation. EFA accounts only for the variance in grades which is shared with other principles but allows rotation. To lever up the benefits of each technique, we proceed in two stages: first, we use the PCA to generate a matrix that combines the commonality and the uniqueness variances of the grades to explore which patterns emerge in the data without any a priori clustering20; second, we use the generated matrix to generate a set of exploratory factors that allows rotation of the estimated factors to spread variability among the contributing principles.21

55. The optimal procedure selects four underlying factors. A central problem in the application of factor analysis is deciding how many factors to retain. The number of factors can be thought of as the number of unobservable uncorrelated constructs that are jointly measured by the observed principles. We use several criteria to determine the optimal number of factors, including the minimum average partial (MAP) method, the “eigenvalues greater than one” rule, and the scree test. Overall, the percentage of variance explained by the four factors is around 60 percent of total variability for the three standards.

56. The identification of significant loadings allows backing out the underlying principles that will be selected under a risk-based approach. Principles with the highest loadings show the highest correlations with the factors. We expect a significant loading for each principle on only one factor. To identify significant loadings we use a range of criteria including the condition that factor loadings should be greater than 0.50 in absolute variable, and that the rate of change in the relative magnitude of the loadings should reflect fast decline at the point at which the last principle is selected. On the other hand, as there are principles that fail to load significantly on any factor, we complement factor analysis with a principle-by-principle approach in the econometric analysis.


57. We select the principles with the highest factor loadings for BCP, ICP, and IOSCO Principles. After choosing the number of factors to retain, we rotate the factors so that principle loadings on various factors tend to be either very high or very low. This allows using the underlying principles to provide an intuitive characterization of the meaning of the factors: in general, the larger the absolute size of the loading of a principle, the more important that principle is in providing a characterization of the factor). It should be stressed that the names used to “label” each factor below are subjective; however, these “labels” are for convenience only, and have no bearing on the econometric results.

  • For BCP, the rotation of the loadings allows interpreting the four underlying factors as ‘Risk management’ (CP7, CP12, CP13, CP14, and CP15), ‘Legal’ (SP11, SP13, and SP14), ‘Information sharing’ (SP16, CP3, CP24, and CP25), and ‘Asset quality’ (SP15, CP8, CP10, and CP22).22

  • For ICP, the four underlying categories can be interpreted as ‘Governance’ (ICP5, ICP8, ICP16, and ICP19), ‘Information sharing’ (ICP3, ICP23, and ICP25), ‘Regulation’ (ICP4, ICP10, and ICP15), and ‘Prudential and Supervision’ (ICP9, ICP14, ICP17, and ICP24).

  • For IOSCO, based on the patterns of the factor loadings, the four categories can be named as ‘Information sharing’ (P1, P13, P14, P15, and P34), ‘Collective investment schemes’ (P24, P25, P26, and P27), ‘Market intermediaries’ (P30, P31, P32, and P36), and ‘Issuers’ (P5, P16, P17, and P18).

Empirical Framework

58. All models for BCP and ICP are estimated using annual data. For BCP we use firm-level data sourced from Bankscope and aggregate country data drawing on IMF FSI statistics. For ICP we use company-level data from the SNL database and OECD and EIOPA data for aggregate country variables. For IOSCO, all models are estimated using monthly data. We construct financial indicators drawing on Bloomberg, Datastream, Consensus Forecasts, WEO, and Haver Analytics.

59. For each standard, we estimate a benchmark model explaining the determinants of each measure of risk. The benchmark model includes a set of macro-financial determinants, fixed effects, and time-varying controls. When appropriate, we estimate dynamic versions of the model using the Arellano-Bond approach. The benchmark model is expanded to assess the effects of compliance with core principles or factors on risks. To select principles or factors, we include interaction terms between macro-fundamentals and individual principles or factors allowing for single or multiple interaction terms. Principles or factors showing a significant negative (positive) interaction coefficient in a specification where an increase in the dependent variable signals low (high) vulnerability were selected.23 The categories of risk and choice of dependent variables by standard and econometric approach is shown in Appendix II. Table 1. The remainder of this section shows the econometric results for all benchmark regressions conducted under the parametric approach. It also illustrates the non-parametric approach by reporting the benchmark binary tree for credit risk in the banking sector.

Appendix II. Table 1.

Appendix II. Table 1. Risk Definitions

article image
Note: The EDF measure is the implied expected default frequency from Moody's KMV database. The CMAX measure is calculated as x /max[x (x | j 0,1, T)] t where x denotes the MSCI index closing price and T = 24 for monthly data.

Econometric Results for BCP (Appendix II. Table 2)

  • Credit risk. The estimation shows that increases in lending rates, lower inflation, and lower growth rates exacerbate credit losses. Higher nominal lending rates and lower inflation increase ex-post real interest rates, while lower economic growth increases unemployment. Other macroeconomic determinants are also significant. Larger capital inflows are associated with more borrowing and a higher ratio of credit losses to total assets.

  • Market risk. Two measures of market risk are considered:

    • Interest rate risk in the banking book. Higher interest rates and faster past lending growth are associated with a higher ratio of net interest income to assets. Also, net interest income increases in response to higher real GDP growth rates.

    • Market risk from securities (including sovereign). Higher lending rates reduce net income from securities relative to assets. Higher long-term government bond yields are associated with higher net income from securities relative to assets. Also, the net income from securities to assets ratio tends to increase in periods of high economic growth and large current account balances.

  • Liquidity risk. Liquidity risk is analyzed through the loans to deposits ratio, which proxies the size of illiquid assets funded by non-deposit funding sources. As the loans to deposits ratio increases, liquidity decreases. Banks hold less liquid assets and increase lending when economic growth and the intermediation spread are higher. In regards to the latter, both higher lending rates and lower deposits rates deepen intermediation, increasing the loans to deposit ratio, and reducing liquidity.

  • Contagion risk. The estimation shows that changes in expected default frequencies (EDFs) corresponding to global banks are transmitted into EDFs of domestic banks, when other determinants of domestic EDFs are controlled for. Domestic banking sector EDFs also increase in response to changes in global stock market volatility (VIX), higher domestic corporate sector EDFs, and slower domestic economic growth. Contagion risk analysis was conducted based on country-specific (aggregate for each country) monthly data. Real GDP growth expectations are captured by monthly consensus forecasts.

  • Overall risk. Overall risk is inversely related to the z-score measure.24 The estimation indicates that higher economic growth increases the z-score, thus reducing overall risk. Also, an increase in bank funding costs (measured by the deposit rate) and a depreciation of the domestic currency exacerbate overall risk. This risk was also analyzed using country-aggregate data.

Appendix II. Table 2.

BCP Benchmark Estimation Results

article image
Notes: Estimation methods used under each risk are as follows: two-step Arellano-Bond for credit risk and liqudiity risk; fixed effects for interest rate risk in the banking book, market risk from securities, contagion risk and overall risk.




Econometric Results for ICP (Appendix II. Table 3)

Appendix II. Table 3.

ICP Benchmark Estimation Results

article image
Notes: Estimation methods used under each risk are as follows: Fixed effects for equity risk, overall market risk, issuer risk, sovereign risk, leverage ratio, retention ratio, and liquidity ratio; two-step Arellano-Bond for combined ratio; and recursive forecasts for contagion where the first rolling window spans 2000m1 to 2012m3.




Insurance companies provide insurance to customers and collect revenue in the form of insurance premiums. Revenue collection is allocated to different investments, including cash assets, bonds, equity, real estate, and other assets. Liabilities of insurance companies consist mainly of technical provisions— the estimated value of future claims associated with the underwritten insurance policies. These activities pose risks: insurance companies could become insolvent if the value of their investments declines sufficiently, or if technical provisions turn out to be insufficient to cover insurance claims.

  • Market/investment risk. A decline in the value of insurance companies’ assets is the main solvency risk. Assets typically held by insurance companies include: cash and money market instruments, long-term bonds, equity, real estate, and other assets. As bond holdings usually represent the largest investment, the main risks taken by insurance companies are associated with sudden changes in bond yields and credit risks embedded in securities issued by sovereigns, financial and non-financial corporations.25

    • Issuer risk. It captures the default risk embedded in corporate and financial bond exposures of insurance companies. According to the estimation, this risk is explained by the following factors. Default risk declines when either long term bond yields or equity returns increase—as both signal a stronger economy. For a given domestic equity return, and increase in international stock returns increase corporate/financial default risk. Higher domestic equity volatility is associated with a higher risk of corporate/financial default. An increase in VIX—holding domestic volatility constant— reduces domestic corporate/financial default risk. Finally, stronger domestic growth mitigates default risk.

    • Sovereign risk. It captures the default risk embedded in the sovereign bond holdings of insurance companies. An increase in domestic equity return signals a stronger economy and reduces the probability of sovereign default. Similarly, higher GDP growth and a strengthening of the fiscal balance mitigate sovereign default risks and are associated with lower CDS spreads. Higher domestic equity volatility and a lower VIX also increase sovereign risk.26

    • Equity risk. Lower price-to-book values of equity are associated with lower future earnings growth and higher risk—higher risk, in turn, is compensated by higher expected return. The results indicate that higher GDP growth and return on equity lead to higher market-to-book values, higher future earnings, and lower risk (and expected return).

    • Overall market risk. This risk encompasses both underwriting and insurance risks. Insurance companies benefit from an increase in interest rates—as they earn higher interest income on bonds held to maturity and the actuarial value of future claims associated with technical provisions declines. Overall profitability also increases in line with higher stock market returns and stronger GDP growth.

    • Insurance and underwriting risks. These risks are specific to the insurance business, and capture the effects of sudden increases in claims payment, insufficiency or inadequacy of technical provisions, declines in premium collection, and higher pass-through of premium written to re-insurers.

      • Leverage ratio. An increase in this ratio often signals higher risk appetite by an insurance company: revenue collected from policy underwriting outpaces the growth of technical provisions. Variations in the ratio over time are associated with the business cycle, with net premium written rising during economic expansions.

      • Combined ratio. An increase in this ratio reduces profitability, as it indicates that claims paid in the period are high relative to net revenue received from premium written. The estimation shows that higher growth is associated with a lower combined ratio—as net premium written appears to exhibit cyclicality. Other macroeconomic determinants include: size (measured by total assets of the insurance company), past profitability of the insurance company (net income ratio), and long term bond yields that affect the return on investments (bonds and stocks).

      • Retention ratio. Insurance companies experiencing capital shortages or other difficulties tend to transfer written premium to re-insurers; hence, a decline in the ratio (lower retention) can be interpreted as a signal of difficulties in the insurance company. The estimation indicates that declines in retention rates tend to occur during recessions. Also, larger and more capitalized insurance companies tend to retain a larger share of written premium.

    • Liquidity risk. Liquidity risk in insurance companies is considered less important than in banks as the former do not perform liquidity transformation. The estimation indicates that higher long term bond yield reduce the liquidity ratio, as the opportunity cost of holding cash increases. Similarly, during economic expansions, when GDP growth and equity returns are higher, the cost of holding the most liquid assets increases, and hence, the liquidity ratio declines.

    • Contagion risk. We examine the propagation of distress from key issuers of fixed income securities including the banking sector, the corporate sector, and its own sovereign.27 Our findings confirm the vulnerability of insurance firms to distress in the domestic banking sector as well as to their own sovereign reflecting their large holdings of fixed income instruments issued by these two entities. Whereas we find some evidence of contagion stemming from distress in global insurance firms, probably reflecting spillovers across internationally active conglomerates, this channel of contagion has subsided over time. On the other hand, our results suggest some flight-to-quality effects at the height of the financial crisis from distress in the global banking sector into the insurance sector.

Econometric Results for IOSCO (Appendix II. Table 4)

Appendix II. Table 4.

IOSCO Benchmark Estimation Results

article image
Notes: The estimation method is recursive 1-period ahead forecasts on monthly data. The first rolling window spans 2000m1 to the last observation in the recursive window. We capture monetary policy stance as the deviation of the policy rate from the Taylor rule. We use the weights and parameters specified in Taylor (2010) for both Aes, and EMs, namely 1 percent for real interest rate, 0.5 for gorwth, and 1.5 for inflation. A positive sign shows too lax MP, a negative sign shows too tight MP. The term premium is based on Cochrane-Piazzesi (2005) in which expected excess returns of US Treasury bonds are represented as a linear function of one-through five-year forward rates. The change in corporate credit yields is defined as monthly changes in the basis-point yield spread between BBB and AAA industrial bond indexes, extracted from the Bloomberg fair market curve. To compute the cross-border EDF banking sector measure at the 50th percentile, we construct for each country of the sample the average implied EDF of the banking sector for the other countries, weighted by GDP, excluding the specific country. The liquidity spread is defined as the difference between the US 3m repo rate and the US 3m T-bill rate to capture short-term counterparty liquidity risk. The world MSCI equity premium is defined as the monthly changes in the price-earnings ratio for the MSCI world index. The US equity excess return is the monthly value-weighted return on all NYSE, AMEX, and NASDAQ stocks minus the one-month Treasury-bill return.