Forecasting bank failures: timeliness versus number of failures

Abstract

Motivated by the observation that very few banks fail in normal years, we explore the impact of that pattern on the precision of a standard statistical failure model and discuss implications for regulation and risk management. Out-of-sample forecasting is found to be worse for a model fitted to recent data with few failures than for a model fitted to much older data with more failures.

Keywords:

• bank failure
• early warning
• rare events

JEL Classification:

• G21
• D80
• C53

Acknowledgements

The authors are grateful for helpful comments on earlier versions of this work from Frederique Bracoud, Abigail Brown, Fabrizio Carmignani, Erik Schlogl, Susan Thorpe and other seminar participants at the University of Queensland and the University of Technology, Sydney.

Notes

¹This point applies to both formal statistical models and subjective assessments of risk. According to the 1997 Economic Report of the President, the annual US business failure rate per 10 000 companies averaged 63.8 during 1955–1995. (More recent values for these figures appear to be unavailable.) By contrast, just 4.9 banks per 10 000 failed each year on average during 1995–2005. Not a single commercial bank failed in the Unites States during 2005–2006.

²Demirgüç-Kunt (1989) reviewed earlier studies in this literature and noted that statistical models to predict bank failure or high financial risk have been available to bank regulators since the mid-1970s.

³Although proportional hazard or survival-time models have also been used to forecast bank failures, logit models have proven quite accurate in such applications, have been used by the Federal Reserve in supervisory monitoring and early warning (Cole, 1995) and continue to be represented in the research literature (DeYoung, 2003; Arena, 2008).

⁴Newer banks exhibit atypical financial behaviour (DeYoung and Hasan, 1998; Shaffer, 1998; DeYoung, 2003). Some unknown fraction of voluntarily acquired banks would otherwise have failed (Thomson, 1991), and misclassifying them as surviving would bias the estimated coefficients. Retained banks satisfied all of the following: −1 ≤ equity/assets ≤ 0.5, 0 ≤ loans/assets ≤ 1, jumbo Certificates of Deposits (CDs) /assets ≤ 0.8, −0.25 ≤ net income/assets ≤ 0.25 and 0 < expenses/assets ≤ 0.3. Our ratio requirements removed fewer than 2% of the banks, none of which failed during the relevant years.

⁵King and Zeng (2001) suggested a method of stratified sampling for logit regressions with rare events, but acknowledged that using the full population is always preferable when available. We therefore base our sample on the full population, thereby also maintaining comparability to prior bank failure studies and regulatory models.

⁶Zero Type I error can always be achieved by predicting that no banks fail (yielding 100% Type II error), whereas zero Type II error can always be achieved by predicting that all banks fail (yielding 100% Type I error). Although a predicted probability of failure greater than 0.5 is usually interpreted as a prediction of failure, a more appropriate interpretation depends on the relative cost of each type of error and on the proportion of failed banks in the sample, which varies from year to year (King and Zeng, 2001; Greene, 2003, p. 685). The graph depicts outcomes for all possible interpretations, allowing the reader to focus on her preferred point along either axis.

⁷ Figure 1 also compares the accuracy of the 1989 coefficients in forecasting failures during 1992–1993 versus the accuracy of the 1989 coefficients in forecasting failures in 2002–2003. Unsurprisingly, the near-term holdout sample is more accurately predicted than the much later one, consistent with the significant difference between the 1989 and 1999 coefficients, and reflecting some depreciation over time of the information contained in the 1989 estimates.