comprehensive test of accounting, market and hybrid
TRANSCRIPT
Comprehensive Test of
Accounting, Market and Hybrid Bankruptcy Prediction Models
Julian Bauer
Cranfield School of Management
Vineet Agarwal*
Cranfield School of Management
This version: 15.12.2011
*Corresponding author:
Vineet Agarwal
Cranfield School of Management
Bedford MK43 0AL
United Kingdom
Tel: + 44 (0) 1234 751122
Comprehensive Test of
Accounting, Market and Hybrid Bankruptcy Prediction Models
Abstract
In recent years hybrid models, using both market and accounting information, have
become state of the art in predicting firm bankruptcies. However, a comprehensive test
comparing their performance against alternative approaches that use either accounting
or market data is missing in the literature. Using a complete database of UK Main listed
firms between 1979 and 2009, our ROC curve analysis shows that the hybrid approach
is superior to the alternatives. Further, our information content tests demonstrate that the
hybrid models subsume all bankruptcy related information in the Taffler (1983) z-score
model as well as in Bharath and Shumway (2008) contingent claims based market
model. Finally, using a mixed regime competitive loan market with different costs of
misclassification, the economic benefit of using the Shumway (2001) hybrid model is
clear.
JEL classification: C52; G33; M41
Keywords: Distress risk; credit risk; z-score; option pricing; hazard models
3
1 Introduction
The risk of going bankrupt is of major interest to shareholders, creditors, and employees
of a firm. There is a vast body of literature on assessing the risk that individual firms will go
bankrupt.1 The approaches in literature differ in their information basis. While traditional
models are predominantly based on either accounting information (e.g. Altman, 1968) or
market information (e.g. Vassalou and Xing, 2004), more recent literature provides hybrids
that assess bankruptcy risk using both data sources (e.g. Shumway, 2001). However, the
existing literature does not provide clean evidence on the usefulness of competing
approaches.
We fill this gap by providing a comprehensive comparison of the predictive ability of
existing hybrid, accounting and market models using a battery of tests. Specifically, we test
three hybrid models (Shumway, 2001; Campbell et al., 2008; and Christidis and Gregory,
2010), the Taffler (1983) z-score model, and a market based model using information content
tests, and receiver operating characteristics (ROC) curve. Finally, we use the framework of
Stein (2005) and Agarwal and Taffler (2008a) to test the economic impact of using different
bankruptcy prediction models in a competitive environment.
We find that while all three approaches possess bankruptcy prediction ability, the hybrid
models dominate the other two, both in our ROC curve analysis and information content tests.
We also show that the most-parsimonious hybrid model of Shumway (2001) has best risk
return characteristics in a competitive loan market.
Accounting data based bankruptcy prediction models filter the relevant information from
publicly available accounts to assess bankruptcy risk. In a way, traditional accounting models
1 Throughout the paper we use the terms bankruptcy, failure and financial distress interchangeably.
4
are a structured fundamental analysis using published financial statements. They are typically
developed by searching for the linear combination of ratios that best differentiates between
(matched) samples of non-failed and failed firms through discriminant or logit models.
Despite the widespread use of the accounting based bankruptcy prediction models in the
literature2, they are often criticised for their lack of theoretical grounding. Hillegeist et al.
(2004) argue that accounting data is by nature historical and prepared on a going concern
assumption, hence their use in predicting future, especially one that involves violating the
going concern assumption itself is fundamentally flawed. Similarly, Agarwal and Taffler
(2008a) acknowledge that (i) accounting numbers are subject to reporting standards (such as
conservatism and historical cost accounting) that might hinder a representation of the true
economic value of assets, and (ii) accounting numbers can, at least to some extent, be
manipulated by the management. In addition, there are methodological issues associated with
the development of accounting based bankruptcy prediction models. For instance, Zmijewski
(1984) argues that such models are biased as they typically oversample failed firms during
model development. Also, distressed firms are prone to filing their accounts late or not at all,
complicating a timely update of the bankruptcy risk assessment. Mensah (1984) argues that as
ratios change over time, a regular re-estimation of the models is necessary to maintain their
utility.
Market based bankruptcy prediction models overcome many of the fundamental
shortcomings of accounting based models. First, in efficient markets, prices reflect both
historical financial information (i.e. accounting data) as well as the individual and market-
wide outlook of a business. Moreover, market prices are less likely to be influenced by
accounting policies. Second, while accounting based models typically lack theoretical
2 See Agarwal and Taffler (2007) for examples.
5
underpinnings, market based models have impeccable theoretical grounding as they draw on
the Black and Scholes (1973) and Merton (1974) option pricing framework. In these models,
equity is viewed as a call option on the firm’s assets, and the probability of going bankrupt is
simply the probability that the call option is worthless at maturity (i.e., market value of total
assets is less than the face value of total liabilities).
However, implementation of market based models is far from straight forward. Firstly,
Saunders and Allen (2002) argue that structural models are unable to differentiate between the
different durations of debt since they assume a zero-coupon bond for all liabilities. Secondly,
Avramov et al. (2010) argue that distressed firms are prone to suffer from market micro
structure problems such as thin trading or limitations to short-selling which might result in
prices deviating from fair values for extended period. Perhaps more importantly though, some
key variables required for these models (e.g., asset volatility, expected asset returns, and
market value of assets) are unobservable and need to be approximated introducing potentially
large errors.
The competing arguments in accounting and market based bankruptcy prediction enforce
a trend in literature that argues for combining the two information sources. Sloan (1996) finds
that market prices do not accurately reflect the information from company accounts, hence,
accounting data can be used to complement market data. Pope (2010) argues for combining
the accounting and finance disciplines. In line with these arguments, latest hybrid models
dismantle the strict separation of accounting and market data while incorporating the
informational benefit of both.
The majority of recent hybrid models combine accounting and market data in simple
discrete time logit models following Shumway (2001). Chava and Jarrow (2004) use a
mixture of accounting and market based ratios consisting of profitability, liquidity as well as
6
market volatility or market price. Campbell et al. (2008) integrate accounting and market
information even further by using ratios that contain accounting variables (e.g. profit) in the
numerator and the market value of total assets in the denominator. Christidis and Gregory
(2010) argue that the model performs better in the UK market when it is adjusted for
additional macro-economic and accounting variables. However, given the critical importance
of the ability to identify potential failures early, the true worth of different approaches should
be measured by how good they are empirically rather than how sound they are theoretically.
The evidence in the existing literature that compares the performance of market based
and accounting based approaches shows that the theoretical superiority of the former does not
necessarily imply a higher explanatory power. Hillegeist et al. (2004) compare the o-score
and z-score with the contingent claims based measure using information content tests. While
they claim their market based model carries more information about future bankruptcy, they
also find that the two measures carry different information. Further, they do not provide tests
of classification accuracy of the different models. Reisz and Perlich (2007) use an alternative
market based model and ROC curve to compare it with the Altman (1968) z-score. They show
that for a one-year prediction horizon their down-and-out framework outperforms the Merton
(1974) framework but, in contrast to Hillegeist et al. (2004), underperforms the z-score
model. Likewise, Agarwal and Taffler (2008a) find that there is no significant difference in
the explanatory power between z-score and the market based approach. Similar to Hillegeist
et al. (2004) they demonstrate that the two measures carry different information about
bankruptcy risk. In addition, using the approach of Blöchlinger and Leippold (2006) they
prove that z-score is a better pricing tool for lenders.
The second strand of literature tests the performance of hybrid models against accounting
based models. Shumway (2001) compares a hybrid model to the accounting based alternatives
and finds that the majority of previously used accounting variables from Altman (1968) and
7
Zmijewski (1984) have little bankruptcy forecasting power. A combination of accounting and
market-driven variables such as past stock returns and idiosyncratic risk increases forecasting
accuracy. Charalambakis et al. (2009) find similar results for the UK market. Chava and
Jarrow (2004) extend the study of Shumway (2001) and provide further confirmatory
evidence that the accounting variables previously used in the literature add little predictive
power when market variables are also included.
Campbell et al. (2008) is the only study that compares hybrid and market based models.
They show that their hybrid model outperforms the market based model of Moody’s KMV in
information content tests. In addition, they also demonstrate that their hybrid model subsumes
the bankruptcy related information of the market based model.
In this paper, we provide the first comprehensive test of the three bankruptcy prediction
approaches. The hybrid models used in this study are the seminal models in Shumway (2001)
and Campbell et al. (2008) as well as the model in Christidis and Gregory (2010). In our tests,
accounting models are represented by Taffler’s (1983) z-score and the market based models
by the naïve distance to default model (naïve DD) of Bharath and Shumway (2008). We use
ROC curve and information content tests to compare the predictive ability of the models. We
further assess the economic performance in a hypothesised competitive loan market with
different misclassification costs.
The paper proceeds as follows. Section 2 introduces the basic form of the hybrid models
as well as the accounting and market based models used in this study. Section 3 describes our
sample, data sources and the variables used. Section 4 presents the approaches used to
compare the bankruptcy prediction models. Section 5 discusses the results, and section 6
concludes.
8
2 Bankruptcy Prediction Models
2.1 Hybrid Model
Hybrid models use both accounting and market data to assess the risk of a firm going
bankrupt. In the existing literature, these frequently take on the functional form of discrete
hazard models using logistic regression functions (e.g. Shumway, 2001). Discrete hazard
models use time varying variables to estimate a firm’s bankruptcy risk at each point in time.
Clearly, the probability that a firm will go bankrupt in t+1 is conditional on survival until t.
The dependent variable is of binary form i.e. survival or failure in t+1. We use annual data to
assess each year the risk that a firm will go bankrupt in the next twelve months (i.e. t+1).
Following Chava and Jarrow (2004) and Campbell et al. (2008) we specify the discrete
probability of failure at time t as
i,t i,t i,t
e t i,t
(1)
Where i,t is the probability at time t that firm i will go bankrupt in the next period, i,t
is coded 1 if the company failed in t+1 (0 if not) and i,t is the vector of the time varying
covariates known at time t and with its coefficients given by . We use the specifications in
Shumway (2001), Campbell et al. (2008), and Christidis and Gregory (2010).
2.2 Accounting Based Model
The seminal accounting based model of Altman (1968) is a widely used benchmark in
bankruptcy prediction literature. His model uses discriminant analysis to identify the linear
combination of financial ratios that best differentiates between failed and non-failed firms.
Taffler (1983) uses a similar approach to build a UK version of the model. The full UK z-
score model as first published in Agarwal and Taffler (2007) is
9
. . . . . (2)
Where is profit before tax over current liabilities, is current assets over total
liabilities, is current liabilities over total assets and is the no-credit interval that is
calculated as (quick assets – current liabilities) / ((sales – profit before tax – depreciation) /
365).3
2.3 Market Based Model
The third model we use in our analysis is a simple market based model that
operationalises the contingent claims approach of Black and Scholes (1973) and Merton
(1974). Market based models derive the distance to default and convert it to probability of
failure using a cumulative normal density function. The use of the option pricing formula
requires two assumptions: the total firm value follows a Brownian motion and total debt is a
discount bond maturing at time T. The equity value is given by the well-known option pricing
formula:
d e rfT d (3)
Where is the value of equity, the value of the total assets i.e. the value of the firm, rf
is the risk-free rate, the face value of total debt and describes the cumulative standard
normal distribution. d is given by
(4)
And d is given by
d d T. (5)
3 The model was constructed in 1977 and thus the z-scores are completely out of sample.
10
Where is the firm volatility and T the time to maturity of the assumed discount bond.
Firm volatility is unobservable but it is related to the observable equity volatility by:
d
(6)
Finally, distance to default is calculated and implemented in a cumulative normal density
function as
Where M is the probability of default using the distance to default score M in a
normal density function ; is the expected return on assets over the forecasting period.
The standard BSM model requires simultaneous estimation of Eq. (3) and Eq. (6).
However, Bharath and Shumway (2008) show that the value of traditional market based
models lie in their functional form rather than in solving of the BSM-model. As such, they
suggest a naïve version of the market model (naïve DD) that retains the functional form but
bypasses the simultaneous calculation of unobservable parameters. Therefore, Bharath and
Shumway (2008) estimate debt volatility as
,na ve . . . (8)
The firm volatility is interpreted as the weighted average of the equity and debt volatility:
,na ve
(9)
With these variables Bharath and Shumway (2008) define their naïve DD measure and
probability of default as
M M ln
.
T
T
(7)
11
na ve na ve ln
ri,t . ,na ve
T
,na ve T .
(10)
Where na ve is the probability of default for the na ve score. Instead of using forecasted
returns, the naïve model uses ri,t- which is the return over the previous year.4 The strike price
is D that is assumed to be a single discount bond maturing at . The prediction horizon is set
to one year and thus T equals one.
3 Sample Selection and Data
3.1 Sample and Sample Selection
Our sample consists of all UK non-financial firms listed in the Main market segment of
the London Stock Exchange (LSE) at any time between 1979 and 2009.5 Accounting data is
sourced from Datastream, Exstat and Company Analysis (in that order). For some failed firms
we add hand collected data from Fame (Bureau van Dijk) and the London Business School
Library.
Following Agarwal and Taffler (2008a), we choose the portfolio formation date to be at
the end of September each year. A portfolio year is defined as the twelve month period
starting with October each year from 1979 to 2009. To be included, a company must have
market data available one year before portfolio formation. Market data is as at the portfolio
formation date. The accounting data is lagged by five months e.g. at the end of September in
year t, the company must have accounting data with a fiscal year ending between May t-1 and
4 Bharath and Shumway (2008) as we all Agarwal and Taffler (2008a) show that the exact specification of
these variables has little impact on the forecasting ability of the model. 5 Other studies, including Christidis and Gregory (2010) use firms listed at Alternative Investment Market
(AIM). However, the two markets are distinct because, inter alia, the regulatory environment and the listing
requirements for Main are much stricter than for AIM, the AIM listed firms have a significant lower average
market capitalisation (£20.5m vs. £263.8m for Main), and the average failure rate for AIM listed firms is nearly
three times higher. Therefore studies should focus on either of the two segments.
12
April t. We relax this restriction in the portfolio year of failure and use the most recent
accounting data with fiscal years ending between May t-2 and April t.6
In contrast to Christidis and Gregory (2010) who include cancellations and suspensions,
we define failure as one of the following: liquidation, administration/receivership or valueless
company.7 First, we use LSPD to identify failures with death codes 7, 16, 20 and 21. Second,
we complement our sample with the failures provided by the Capital Gains Tax Book/HM
Revenue & Customs (companies in receivership and/or liquidation or companies of negligible
value). Third, we use Factiva (primary source is Regulatory News Service) to complement
and cross-check our list of failures (receivership or administration announcements). The
failure date is given by the last trading day of the failed company found in Regulatory News
Service, LSPD or Datastream (in that order).8 Return in the month of failure is set to
-100.0%.9
Our final sample consists of 28,804 firm years between portfolio years 1979 and 2009
consisting of 2,748 unique firms of which 273 failed. Our sample has an average annual
failure rate of 0.9% and 10.5 observations (i.e. years) per firm for portfolio years 1979 to
2009. Table 1 presents the distribution over the portfolio years 1979 to 2009.
TABLE 1 HERE
6 This approach is used since failed companies are unlikely to report their results timely or even fail to
report their latest accounts (Keasey and Watson, 1988). 7 Arguably, loan renegotiations/defaults, rescue rights issues, forced disposals etc. are all failures.
However, majority of bankruptcy prediction literatures uses formal insolvency proceedings to identify failure as
these are unambiguous. 8 In the majority of cases there is no difference in failure date.
9 Franks et al. (1996) argue that the UK bankruptcy regime is more creditors friendly and Kaiser (1996)
shows that stockholders are passed over in terminal payments. Similarly, Agarwal and Taffler (2008b) find only
one case where equity holders received a terminal payment.
13
3.2 Data
In this sub-section, we describe the different variables that are required for the alternative
hybrid model specifications. All firms in our sample have full coverage of the subsequently
described variables. We winsorise variables at the 5.0% level. 10
3.2.1 Accounting Variables
NITA and NIMTA are the profitability ratios defined as the net income (after minorities
and preference shares) over book value of total assets (TA) and market value of total assets
(MTA) respectively. MTA is the sum of market value of equity (MV) and the book value of
total liabilities (TL). TL is defined as the difference between TA and the book value of
shareholders’ equity less preference shares and minorities V . Likewise, TLTA and
TLMTA represent firm leverage from a shareholder perspective. CASHMTA and
NCASHMTA are cash and net cash respectively over MTA. Cash consists of cash and cash
equivalents and net cash is cash less bank overdrafts (i.e. short term debt). We find
NCASHMTA to be more meaningful than CASHMTA, especially for a study of this nature
since distressed firms frequently use bank overdrafts to finance their operations. BM is the
commonly used book-to-market equity ratio i.e. BV over MV. CFMTA is cash flow over
market value of total assets. For cash flow we take net income and add back depreciation and
amortisation and deduct (add) the change in current assets excluding cash (current liabilities
excluding short term debt).
10 Unlike Campbell et al. (2008) and Christidis and Gregory (2010), we do not winsorise share price. For
total assets, we do not adjust the book value of total assets by adding 10% of the difference between market and
book equity (other studies use a similar approach for book value of equity). The adjustment is argued to be valid
since it corrects book values that are probably mis-measured (Campbell et al., 2008: 2905). We purposely do not
choose to use such an adjustment. First, it is another data manipulation in addition to the winsorisation. It is hard
to argue on a general basis which values are actually mis-measured and which not. Second, the approach is only
used for total assets (or BV) but it fails to identify the individual balance sheet items that are most likely to be
mis-measured. As such, the MTA-ratios containing the total liabilities still carry the assumed ‘mis-
measurement’.
14
3.2.2 Market Variables
MV is the market value of common equity at portfolio formation date. RSIZE is a relative
si e measure. It is the log of the firm’s MV over the aggregate market value of the FT ll
Share Index. PRICE is the unadjusted or raw stock price and denominated in pence. EXRET
is the company’s log excess return over the FTSE All Share in the twelve months prior to the
portfolio formation date. SIGMA is the annualised standard deviation of daily returns over the
three months prior to portfolio formation. We follow Campbell et al. (2008) and use the cross-
sectional average for companies that have less than five non-zero observations in the three-
month window.11
3.2.3 Marco Economic Variables
Macro-economic variables indicate the state of the economy. Since literature shows that
failure rates are higher in recessions (Campbell et al., 2008), Christidis and Gregory (2010)
find several macro-economic variables to be significant in their bankruptcy prediction models.
Following their study, we incorporate the deflated Treasury bill rate (DEFLTBR) that affects
a firm’s financing situation. As such, a secure financing strengthens the solvency of a
company and reduces the risk of going bankrupt. Further, we include the term structure
premium (LONGSHT) which is the yield difference between the long-term government bonds
and TBR. A negative term structure, i.e. higher short than long term debt yields, is perceived
as a general recession indicator. For the same reasons, we include the change in the UK
industrial production index (INDPROD) as a general indicator of the economy.
11 There are on average 3.3% observations (with a maximum of 8.8% in 1995 and a minimum of 0.4% in
1986) and 30 failures with less than five non-zero observations in our study.
15
4 Model Evaluation
Our objective in this paper is to identify the best performing bankruptcy prediction model
in the UK. We evaluate the performance of different models by using ROC curve and
information content tests. We also use the mixed regime framework of Blöchlinger and
Leippold (2006) and Agarwal and Taffler (2008a) to evaluate the economic impact of using
the three hybrid models in a competitive environment and different misclassification costs for
type I and type II errors.
4.1 Receiver Operating Characteristics
ROC is a method to assess the appropriateness of prediction parameters. It is widely used
in the field of medicine (Hanley and McNeil, 1982) and is a well-established tool to assess
ratings and validate bankruptcy prediction models {{5108 Sobehart, J. 2001/psee e.g. ;3702
Vassalou, Maria 2004;4867 Agarwal,Vineet 2008;}}.
To construct the ROC curve, for each year we sort the sample firms from high to low
default probability. Let x be an integer between 0 and 100. For each integer x of the highest
default risk firms (x%), we calculate its percentage of failed firms (number of firms that failed
within the next year divided by the total number of failures in the sample). We cumulate the
figures for the total sample period. The plot of x% of highest default risk firms against its
percentage of failed firms results in the ROC curve.
Sobehart and Keenan (2001) argue that the area under the ROC curve (AUC) is the
decisive indicator of a model’s predictive ability. Following Hanley and McNeil (1982), we
calculate the AUC using the Wilcoxon statistic. Hanley and McNeil (1982) further show that
the standard error of the AUC is given by
16
se nF nF
n F
nFn F.
(11)
Where A is the area under the ROC curve, nF the number of failed firms and n F the
number of non-failed firms. is defined as
- and
is defined as
. The test statistic is
then
se
(12)
In order to compare the area under the curves of two different models (denominated by 1
and 2), Hanley and McNeil (1983) suggest the following normally distributed z-statistic:
se se
rse se
(13)
Engelmann et al. (2003) put the ROC curve into context with the cumulative accuracy
ratio (AR) and show that the area under the ROC curve contains the same information as the
AR. They find that AR is just a linear transformation of the area below the ROC curve:
. (14)
4.2 Information Content Test
In addition to ROC, we use the information content tests to examine the explanatory
power of the bankruptcy prediction models. Following Hillegeist et al. (2004) we use the
following discrete hazard model:
i,t e t i,t
e t i,t
e t i,t
(15)
17
Where i,t is 1 if the firm i fails within the next 12 months (0 otherwise), t is the baseline
hazard rate (proxied by the trailing one year failure rate), i,t is a matrix of independent
variables and is a column vector of estimated coefficients.
In order to test the information content of the hybrid, accounting, and market models, we
take the likelihood of failure from each bankruptcy model as independent variable ( i,t). To
be consistent with the underlying assumptions of the logit model, we follow Hillegeist et al.
(2004) and transform the default probabilities from the market and hybrid model into logit
scores:12
score ln p
p
(16)
For all models, we winsorise the probabilities to be between 0.00001 and 0.99999, i.e.,
the scores are within ±18.4207.
4.3 Economic Value when Misclassification Costs are different
Agarwal and Taffler (2008a) argue that while the ROC curve assumes equal costs for lending
to a firm that subsequently fails and not lending to a firm that does not fail, in practice, the
costs associated with the two misclassifications are vastly different. While refusal to lend to a
subsequently non-failed firm simply leads to the loss of extra revenue, lending to a firm that
subsequently fails can lead to substantial losses. We follow the approach of Agarwal and
Taffler (2008a) to assess the economic impact of using different models in a competitive
market. We use the loan pricing model of Stein (2005) and Blöchlinger and Leippold (2006)
to derive the credit spread as a function of the credit score (S) by:
12 We use the same formula to transform z-scores into default probabilities.
18
p t
p t k
(17)
Where R is the credit spread, p(Y=1|S=t) is the conditional probability of failure for a
score of t, p(Y=0|S=t) is the conditional probability of non-failure for a score of t, LGD is the
loss in loan value given default, and k is the credit spread for the highest quality loan.
We closely follow the method in Agarwal and Taffler (2008a) and assume a simple loan
market worth £100.0 billion with banks competing for business each using a different
bankruptcy prediction model. To keep the analysis tractable and objective, we assume all
loans are of same size and have same LGD. The banks reject customers that fall in the bottom
5.0% according to their respective models and quote a spread based on Eq. (17) for all the
other customers.13
The customer chooses the bank which quotes the lower spread. If the
quoted spreads are equal, the customer randomly chooses one of the banks (or equivalently,
the business is split equally between the banks).14
Each year, we independently sort our
sample firms on their probability of failure based on different bankruptcy prediction models
and group them into 100 categories for each of the models.
Due to the number of defaults and the 100 categories we recognise that some of the
categories are sparsely populated with failures. In extreme cases, this could lead to a lower
default probability for the next higher risk category and thus, to a lower credit spread although
failure risk is higher. In order to have a monotonic increase in credit spread with credit risk
we apply the method described in Burgt (2007: Eq. (7)) to smooth default probabilities of the
categories.
13 This is a mixed regime pricing as the banks first make a dichotomous decision of lending/not lending,
and then charge different rates to the customers they decide to lend based on their credit risk. 14
Since some customers may be refused credit by all banks, their market share may not sum to 1.
19
To assess the economic value of using different models for mixed regime loan pricing,
we use two measures to evaluate bank profitability, return on assets (ROA) and return on risk
weighted assets (RORWA):
rofit
ssets lent
(18)
and
rofit
risk eighted assets
(19)
Unlike ROA which ignores the inherent riskiness of profits, RORWA considers the risk
of the outstanding loans and hence is a more suitable performance measure. Similar to
Agarwal and Taffler (2008a), we use the Basel III Foundation Internal Ratings-Based
Approach to derive the value of risk-weighted assets (Basel Committee, 2011), the details are
in the appendix.
5 Results
5.1 Summary Statistics
Table 2 presents the summary statistics of the accounting and market variables for
portfolio years 1979 to 2009. It shows that, not surprisingly, failed firms are less profitable
(lower NITA and NIMTA), more highly geared (higher TLTA and TLMTA), and have
smaller cash holdings (lower CASHMTA and NCASHMTA). Also, failed firms have higher
BM ratios (1.5 vs. 0.9 for non-failed), are smaller (£42.6m market capitalization vs. £265.9m
for non-failed), have lower price (51p vs. 197p for non-failed), are past losers (EXRET of
-50.7% vs. -6.0% for non-failed), and have higher volatility (SIGMA = 88.6% for failed,
43.6% for non-failed). All the differences are significant at the 1% level except for
20
CASHMTA which is significant at the 5% level, and CFMTA that is not significantly
different for the two groups.
TABLE 2 HERE
5.2 Hybrid Models for the UK Main Market
Table 3 reports the estimated coefficients of the three hybrid models (Shumway, 2001;
Campbell et al., 2008; and Christidis and Gregory, 2010) using our full data set from portfolio
year 1979 to 2009 with 28,804 observations.
TABLE 3 HERE
Columns one and two of Table 3 present the coefficients of the Shumway (2001) model.
All the coefficients are statistically highly significant and have the expected sign. Lower
profitability, higher gearing, lower past year stock returns, higher volatility, and smaller
market capitalisation are all associated with higher bankruptcy risk.
Columns three and four present the coefficients for the Campbell et al. (2008) model. The
model differs from Shumway (2001) in that it (i) use the market value rather than book value
of total assets in the accounting ratios, and (ii) adds the cash ratio (CASHMTA), BM and
PRICE. The coefficients again have the intuitively correct sign. Higher profitability, lower
gearing, higher prior year return, lower volatility, and larger size are all associated with lower
bankruptcy risk. However, unlike Campbell et al. (2008), we find CASHMTA, BM and
PRICE not to be significant.
21
Finally, columns five and six show the coefficients for the hybrid model in Christidis and
Gregory (2010) see “accounting and market and economic model ” . We find that the
main firm specific variables enter the regression with the expected sign (i.e. NIMTA,
TLMTA, EXRET, SIGMA and RSIZE). However, in contrast to Christidis and Gregory
(2010), of the three macroeconomic variables, only LONGSHT is statistically significant.
The Pseudo R² for the three models are very similar and range from 24.1% for the
Shumway (2001) model to 25.2% for the Christidis and Gregory (2010) model. As such, there
is no clear evidence that any of these three models is superior to the other.
5.3 Test of Predictive Ability
In this sub-section we present the tests of predictive ability of the five bankruptcy
prediction models. We first present the results of ROC analysis and then those of information
content test. In order to avoid look-ahead bias, we re-estimate the coefficients for the three
hybrid models each year using only the information that is available at each portfolio
formation date.15
To allow a calibration period for the model, we start analysing with portfolio
formation year 1985. Thus, all our further analysis covers portfolio years 1985 to 2009 with
22,217 observations consisting of 2,428 unique firms of which 211 failed.
5.3.1 Receiver Operating Characteristics
We now turn to ROC analysis to assess the relative performance of the three hybrid
models as well as the accounting based z-score model and the market based model.
In order to have a clean test of explanatory power of the models, we need to ensure that
the models are properly calibrated. Table 4 presents the average default probabilities for the
15To calculate the default probabilities at portfolio formation date at the end of September 1985, we run a
regression for portfolio years 1979 to 1984, for the portfolio at the end of September 1986, we use data from
portfolio years 1979 to 1985 and so on.
22
failed, non-failed and all firms for each of the five models tested here. It clearly shows that all
five models generate much higher probabilities of failure for firms that actually failed all
differences are significant at the 1% level. However, both the z-score model and the market
based model produce average failure probabilities (26.3% and 10.8% respectively) that are
much higher than the observed failure rates (0.9% in our sample) suggesting miscalibration.
TABLE 4 HERE
Since Table 4 suggests model miscalibration, we use percentiles rather than failure
probabilities in our ROC tests. Figure 1 presents the ROC for Shumway (2001), z-score, and
the market based model.16
It shows that while all three perform much better than a random
prediction, the Shumway (2001) model clearly dominates the other two.
FIGURE 1 HERE
Panel A of Table 5 shows that all five bankruptcy prediction models significantly
outperform a random classification model (the lowest z-statistic is 17.5). It also shows that
while the three hybrid models have very similar area under the ROC curve (ranging from 0.89
for Shumway (2001) to 0.88 for the other two), the AUC for z-score and market based models
is much lower (0.81 and 0.86 respectively).
Panel B presents the test for differences in the AUC for different models. It shows that
while there is no statistically significant difference between the AUCs of the three hybrid
16 The ROC curves for the other two hybrid models are very similar to that of the Shumway (2001) model
and are excluded from the figure to reduce clutter.
23
models, both the z-score and the market based model have significantly lower AUCs (z for
difference between Shumway (2001) and z-score model is 5.0, and that for the difference
between Shumway (2001) and naïve DD is 3.7). It also shows that the z-score model is the
worst performing model of the five as its AUC is significantly lower than that of the market
based model as well (z = 2.6). 17
TABLE 5 HERE
Table 5 clearly demonstrates that while there is little to choose between the different
specifications of the hybrid approach, it produces probability of bankruptcy estimates that are
superior to the other two approaches.
5.3.2 Information Content Tests
In this sub-section we present the results of tests of information content of the five
bankruptcy prediction models.
Models 1 to 5 in Table 6 show that all three hybrid models as well as z-score and market
based models carry significant information about failure within the next 12 months even after
controlling for other market variables. Consistent with the evidence of Hillegeist et al. (2004)
and Agarwal and Taffler (2008a), Model 6 shows that both, z-score and naïve DD are
capturing distinct aspects of bankruptcy risk although the coefficient of naïve DD is
significant only at the 10% level.
17 The results are in sharp contrast to those in Agarwal and Taffler (2008a). However, using the same
sample period (1985 to 2001), we also do not find any significant difference between the AUCs of z-score and
market based model (z = 1.3). Agarwal and Taffler (2007) note that the performance of the z-score model has
deteriorated since the late 1990s.
24
Models 7, 8, and 9 show that the coefficients on both z-score and market based model
estimates become statistically insignificant (t is between 1.1 and 1.5 for z-score, and 0.5 and
0.8 for market based model) while those on hybrid models estimates remain statistically
highly significant (t is between 2.3 for Campbell et al. (2008) and 2.6 for Shumway (2001))
when all three are included in the same equation. The results show that the Shumway (2001)
model estimate subsumes all the bankruptcy related information in both z-score and market
based approach and clearly demonstrate the superiority of the hybrid approach.18
TABLE 6 HERE
5.3.3 Economic Value when Misclassification Costs are different
The results of the previous two sub-sections present strong evidence of superiority of the
hybrid models over the other two approaches in the UK. Hence, in this sub-section we restrict
the analysis to the three hybrid models that our previous tests are unable to distinguish
between. Agarwal and Taffler (2008a) note that the credit spread given by Eq. (17) is a
function of the probability of failure and non-failure, and is therefore influenced by both the
power as well as calibration of the model. In order to have clean measures of the economic
impact of model power uncontaminated by possible differences in calibration (see Table 4),
we use bankruptcy probability percentiles as in Agarwal and Taffler (2008a). Further, we
smooth the ROC curve using the method of Burgt (2008). Similar to Agarwal and Taffler
(2008a), we assume that the three banks follow the Basel III Foundation Internal Ratings-
Based approach, all loans are unsecured senior debt (i.e., LGD is 45.0%), and the risk
premium for the highest quality customer (k) to be 0.30%.
18 It is not possible to include the three hybrid model estimates in the same regression due to
multicollinearity.
25
Table 7 presents the revenue, profitability, and other statistics for the three banks under
the mixed regime competitive loan market described earlier.
TABLE 7 HERE
Table 7 shows that Bank 1 (Shumway, 2001) has the largest market share of 54.1% as
compared to 21.3% and 20.7% respectively for the other two banks that use Campbell et al.
(2008) and Christidis and Gregory (2010). The quality of loans granted by Bank 1 is also
better as only 0.39% of its customers default compared to 0.72% and 0.74% for the other two
banks. The better credit quality of Bank 1 loans is also reflected in the lower average spread it
earns (51bp against 55bp for Bank 2 and 67bp for Bank 3). The higher market share of Bank
1 also translates into much higher profits (£178.8m vs. £48.8m for Bank 2 and £70.4m for
Bank 3). While the ROA of Bank 2 is clearly lower than the other two banks, Banks 1 and 3
earn similar ROA. However, using RORWA, which measures profitability as a function of
risk, Bank 1 outperforms Bank 3 (1.23% vs. 0.32%). On this basis, considering the
differential misclassification costs, the Shumway (2001) model clearly outclasses the other
two hybrid models in economic terms.
6 Summary and Conclusions
Bankruptcy prediction literature provides various approaches to assess the risk that an
individual firm will go bankrupt. The three main approaches can be summarised by models
using either accounting data (e.g. Altman, 1968), market data (e.g. Hillegeist et al., 2004) or
both (e.g. Shumway, 2001) for credit risk assessment. The existing literature, however, fails
to provide clear guidance on the best approach empirically.
26
In this study, we use the hybrid models of Shumway (2001), Campbell et al. (2008) and
Christidis and Gregory (2010). We further implement the accounting based z-score (Taffler,
1983; Agarwal and Taffler, 2007) and the naïve version of the contingent claims market based
approach of Bharath and Shumway (2008).
We use a battery of tests to compare the performance of alternative approaches and show
that first, in terms of default probabilities, all models are able to differentiate between failed
and non-failed firms. However, there is clear evidence that the z-score and the market based
model are miscalibrated while hybrid models have average default probabilities that are closer
to observed default rates. Second, while the ROC analysis finds no significant difference
between the hybrid models, the other two approaches clearly underperform. Third,
information content tests reveal that all models carry significant distress related information.
Again, the alternative approaches have no distress related information that is not already
contained in the hybrid models. In contrast to ROC, the z-score carries more information
about subsequent failure than the market based approach.
We finally consider the economic value of the three hybrid models in a competitive
market where the cost of lending to a firm that fails is much higher than the opportunity cost
of not lending to a firm that does not default. Our results show that the Shumway (2001)
model leads to a much higher market share and profit, higher credit quality of the loan
portfolio as well as higher return on risk-weighted assets.
We conclude that the hybrid models are superior to the alternative models in bankruptcy
prediction and find no evidence that the more recent models in Campbell et al. (2008) and
Christidis and Gregory (2010) have any advantage over the existing Shumway (2001)
specification which is also simpler to estimate as it requires smaller number of variables. In
fact, Shumway (2001) model leads to economic outperformance once differential
27
misclassification costs are taken into account. We therefore suggest using the Shumway
(2001) specification for future UK research.
Appendix
According to the latest version of the International Convergence of Capital Measurement
and Capital Standards document prepared by the Basel Committee on Banking Supervision
(2011), for obligations not already in default, risk-weighted assets are computed as follows
(pp. 39):
Correlation (R) = [0.12 * (1 – e-50*PD
) / (1 – e-50
)] + [0.24 * (1 – ((1 – e-50*PD
) / (1 – e-50
)))]
Maturity adjustment (b) = [0.11852 – 0.05478 * ln(PD)]2
Capital requirement (K) =
)b*)5.2M(1(*)b*5.11(*LGD*PD)999.0(G*R1
R)PD(G*R1N*LGD 1
5.0
5.0
Risk-weighted assets (RWA) = K * 12.5 * EAD
Where:
PD = probability of default (at least 0.03%),
LGD = loss given default,
N(.) = cumulative normal density function,
G(.) = inverse cumulative normal density function,
M = effective maturity, and
EAD = exposure at default.
28
References
Agarwal, V. and Taffler, R. (2008a), "Comparing the performance of market-based and
accounting-based bankruptcy prediction models", Journal of Banking & Finance, vol. 32,
no. 8, pp. 1541-1551.
Agarwal, V. and Taffler, R. (2008b), "Does Financial Distress Risk Drive the Momentum
Anomaly?", Financial Management (Blackwell Publishing Limited), vol. 37, no. 3, pp.
461-484.
Agarwal, V. and Taffler, R. J. (2007), "Twenty-five years of the Taffler z-score model: does it
really have predictive ability?", Accounting & Business Research, vol. 37, no. 4, pp. 285-
300.
Altman, E. I. (1968), "Financial Ratios, Discriminant Analysis and the Prediction of
Corporate Bankruptcy", Journal of Finance, vol. 23, no. 4, pp. 589-609.
Avramov, D., Chordia, T., Jostova, G. and Philipov, A. (2010), "Anomalies and Financial
Distress", Working Paper, .
Basel Committee, ( 2011), Basel III: A global regulatory framework for more resilient banks
and banking systems, Basel Committee on Banking Supervision.
Bharath, S. T. and Shumway, T. (2008), "Forecasting Default with the Merton Distance to
Default Model", Review of Financial Studies, vol. 21, no. 3, pp. 1339-1369.
Black, F. and Scholes, M. (1973), "The Pricing of Options and Corporate Liabilities", Journal
of Political Economy, vol. 81, no. 3, pp. 637.
Blöchlinger, A. and Leippold, M. (2006), "Economic benefit of powerful credit scoring",
Journal of Banking & Finance, vol. 30, no. 3, pp. 851-873.
Campbell, J. Y., Hilscher, J. and Szilagyi, J. (2008), "In Search of Distress Risk", Journal of
Finance, vol. 63, no. 6, pp. 2899-2939.
Charalambakis, E. C., Espenlauby, S. K. and Garrett, I. (2009), "Assessing the probability of
financial distress of UK firms", Working Paper, .
Chava, S. and Jarrow, R. A. (2004), "Bankruptcy Prediction with Industry Effects", Review of
Finance, vol. 8, no. 4, pp. 537-569.
Christidis, A. C. and Gregory, A. (2010), "Some New Models for Financial Distress
Prediction in the UK", Working Paper, .
Engelmann, B., Hayden, E. and Tasche, D. (2003), "Testing Rating Accuraccy", Risk, , no.
January.
29
Franks, J. R. and Nyborg, K. G. (1996), "A Comparison of US, UK, and German Insolvency
Codes. (cover story)", FM: The Journal of the Financial Management Association, vol.
25, no. 3, pp. 86-101.
Hanley, J. A. and McNeil, B. J. (1982), "The Meaning and Use of the Area under a Receiver
Operating Characteristics (ROC) Curve", Radioligy, , no. 143, pp. 29-36.
Hanley, J. A. and McNeil, B. J. (1983), "A Method of Comparing the Areas under Receiver
Operating Characteristic Curves Derived from the Same Cases", Radioligy, vol. 148, no.
3, pp. 839-843.
Hillegeist, S. A., Keating, E. K., Cram, D. P. and Lundstedt, K. G. (2004), "Assessing the
Probability of Bankruptcy", Review of Accounting Studies, vol. 9, no. 1, pp. 5-34.
Kaiser, K. M. J. (1996), "European Bankruptcy Laws: Implications for Corporations Facing
Financial Distress. (cover story)", FM: The Journal of the Financial Management
Association, vol. 25, no. 3, pp. 67-85.
Keasey, K. and Watson, R. (1988), "The Non-submission of Accounts and Small Company
Financial Failure Prediction", Accounting & Business Research, vol. 19, no. 73, pp. 47-
54.
Mensah, Y. M. (1984), "An Examination of the Stationarity of Multivariate Bankruptcy
Prediction Models: A Methodological Study", Journal of Accounting Research, vol. 22,
no. 1, pp. 380-395.
Merton, R. C. (1974), "On the Pricing of Corporate Debt: the Risk Structure of Interest
Rates", Journal of Finance, vol. 29, no. 2, pp. 449-470.
Pope, P. F. (2010), "Bridging the gap between accounting and finance", The British
Accounting Review, vol. 42, no. 2, pp. 88-102.
Reisz, A. S. and Perlich, C. (2007), "A market-based framework for bankruptcy prediction",
Journal of Financial Stability, vol. 3, no. 2, pp. 85-131.
Shumway, T. (2001), "Forecasting Bankruptcy More Accurately: A Simple Hazard Model",
Journal of Business, vol. 74, no. 1, pp. 101.
Sloan, R. G. (1996), "Do Stock Prices Fully Reflect Information in Accruals and Cash Flows
About Future Earnings?", Accounting Review, vol. 71, no. 3, pp. 289-315.
Sobehart, J. and Keenan, S. (2001), "Measuring default accurately", Risk, , no. March, pp. 31-
33.
Stein, R. M. (2005), "The Relationship between Default Prediction and Lending Profits:
Integrating ROC Analysis and Loan Pricing", Journal of Banking & Finance, , no. 29,
pp. 1213-1236.
30
Taffler, R. J. (1983), "The Assessment of Company Solvency and Performance Using a
Statistical Model", Accounting & Business Research, vol. 13, no. 52, pp. 295-307.
van der Burgt, M. J. (2007), "Calibrating Low-Default Portfolios, Using the Cumulative
Accuracy Profile", Journal of Risk Validation, vol. 1, no. 4, pp. 1-17.
Vassalou, M. and Xing, Y. (2004), "Default Risk in Equity Returns", Journal of Finance, vol.
59, no. 2, pp. 831-868.
Zmijewski, M. E. (1984), "Methodological Issues Related to the Estimation of Financial
Distress Prediction Models", Journal of Accounting Research, vol. 22, pp. 59-82.
31
Figure 1 ROC Curves Hybrid, Accounting and Market Model: Correlations
The figure shows receiver operating characteristics (ROC) curves. At the end of each September between 1985
and 2009, we calculate the default probabilities from the model in Shumway (2001), Taffler (1983) z-score
model and the naïve distance to default model in Bharath and Shumway (2008). We transform the z-score into
probability by p=e^(z-score) / [1+e^(z-score)]. In order to avoid look-ahead bias we take the coefficients from
failure indicators and predictors variables that are known at the end of each September. For each year, we sort
the sample firms from high to low default probability and split them into 100 integers. For each integer x of the
highest default risk firms (x%), we calculate its percentage of failed firms (number of firms that failed within the
next year divided by the total number of failures in the sample). We cumulate the figures for the total sample
period. The plot of x% of highest default risk firms against its percentage of failed firms results in the ROC
curve.
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
% o
f fa
iled
fir
ms
% of firms
Shumway (2001) Z-Score
Bharath and Shumway (2008) Random Model
32
Table 1 Firm Years and Failures 1979 to 2009
The table lists the sample of UK non-financial firms listed in the Main market segment of the London Stock
Exchange (LSE) at any time between 1979 and 2009. The sample is derived from London Share Price Database
(LSPD). Failures are from LSPD (death codes 7, 16, 20 and 21), the Capital Gains Tax Book (companies in
receivership and/or liquidation or companies with negligible value) and the Regulatory News Service from
Factiva (receivership or administration announcements). We summarise below the characteristics for the Total
Sample (portfolio years 1979 to 2009) and for the Sub-Period for the portfolios years 1985 to 2009 used for
comparing the bankruptcy prediction models.
Portfolio Year No. Observations No. Failures Failure Rate
1979 1,067 8 0.75
1980 1,141 12 1.05
1981 1,132 19 1.68
1982 1,108 8 0.72
1983 1,089 9 0.83
1984 1,050 6 0.57
1985 1,018 3 0.29
1986 954 0 0.00
1987 907 2 0.22
1988 866 1 0.12
1989 841 9 1.07
1990 811 19 2.34
1991 824 18 2.18
1992 1,011 9 0.89
1993 1,015 4 0.39
1994 1,064 6 0.56
1995 1,213 8 0.66
1996 1,265 10 0.79
1997 1,280 14 1.09
1998 1,235 13 1.05
1999 1,111 10 0.90
2000 987 9 0.91
2001 916 20 2.18
2002 843 12 1.42
2003 747 7 0.94
2004 669 8 1.20
2005 606 2 0.33
2006 559 3 0.54
2007 510 10 1.96
2008 493 8 1.62
2009 472 6 1.27
Total 28,804 273 0.95
Firms 2,748
Final Sample 1985 to 2009
Total 22,217 211 0.95
Firms 2,428
Obs/Firm 9.15
33
Table 2 Hybrid Models: Summary Statistics Accounting and Market Variables
The table reports potential bankruptcy prediction variables. NITA (NIMTA) is net income available to common
shareholders over book value of total assets (MTA i.e. book value of total liabilities plus market value of
common equity). TLTA (TLMTA) is book value of total assets e cluding total common shareholders’ equity
over book value of total assets (MTA). NCASHMTA is cash and cash equivalents less short term debt over
MTA. BM is book value of common equity over the market value of common equity. CFMTA is operating cash
flow over MTA. MV is market value of common equity. RSIZE is log of MV over the market value of the FTSE
All Share Index. PRICE is unadjusted share price. EXRET is log excess return over the FTSE All Share Index
over the 12 months prior to portfolio formation. SIGMA is the annualised standard deviation daily returns for the
three months prior to portfolio formation. Variables are taken at the end of September (portfolio formation) each
year from 1979 to 2009. Accounting data must be available five months prior to portfolio formation. Mean,
Median and Standard Deviation are taken cross-sectionally over the entire observation period. All variables are
winsorised at the 5.0% level. Panel A contains the entire sample with 28,804 firm years and 2,748 firms. Panel B
contains non-failed firms (28,531 observations). Panel C reports the statistics of failed firms (273 observations)
and the differences to the non-failed group.
Mean
Failed
Mean
Non-Failed
Δ
NF-F t-Stat All Std Dev
NITA -5.98 4.10 10.07 18.01 4.00 7.36
NIMTA -5.06 2.75 7.82 16.50 2.68 5.54
TLTA 68.48 54.16 -14.32 13.28 54.30 17.42
TLMTA 68.13 44.66 -23.47 18.49 44.88 20.87
CASHMTA 4.79 5.71 0.92 2.29 5.70 6.42
NCASHMTA -11.16 -0.89 10.27 12.35 -0.99 11.18
BM 1.48 0.89 -0.59 9.08 0.90 0.74
CFMTA 5.04 4.87 -0.17 0.17 4.87 11.82
MV 42.6 265.9 223.3 22.58 263.8 528.3
PRICE 50.8 197.2 146.4 23.98 195.8 186.6
RSIZE -11.16 -9.32 1.84 23.11 -9.34 1.82
EXRET -50.66 -5.99 44.67 18.56 -6.41 36.04
SIGMA 88.58 43.63 -44.94 20.40 44.06 29.39
34
Table 3 Hybrid Bankruptcy Prediction Models
The table reports the results from binary logit regressions of the failure indicators on predictor variables. At the
end of each September (portfolio formation date) from 1979 to 2009 we take the predictor variables that are
known at the end of September i.e. current market data and accounting data with a lag of five months. The
failure indicator is 1 (0) if the firm failed (not failed) in the twelve months following the portfolio formation
date. The table contains the model presented in Shumway (2001), Campbell et al. (2008), Christidis and Gregory
(2010). We report coefficients and the z-value in parentheses below as well as regression statistics.
Shumway Campbell et al. Christidis and
(2001)
(2008)
Gregory (2010)
NITA -4.88
NIMTA -6.24
NIMTA -7.51
(6.41)
(6.79)
(7.86)
TLTA 2.46
TLMTA 2.27
TLMTA 2.35
(6.87)
(5.25)
(6.14)
EXRET -1.55
EXRET -1.31
EXRET -1.07
(8.56)
(6.89)
(5.61)
SIGMA 1.15
SIGMA 1.19
SIGMA 1.24
(5.45)
(5.58)
(5.71)
RSIZE -0.33
RSIZE -0.23
RSIZE -0.24
(5.96)
(3.85)
(4.01)
CASHMTA -1.88
DEFLTBR -52.48
(1.83)
(1.38)
BM -0.01
LONGSHT -0.18
(0.08)
(3.79)
PRICE 0.00
INDPROD -2.89
(1.40)
(1.63)
PRICE 0.00
(1.27)
CFMTA 0.81
(2.04)
Constant -10.72
Constant -9.21
Constant -9.47
(17.66)
(13.14)
(13.53)
Obs 28,804
Obs 28,804
Obs 28,804
Firms 2,748
Firms 2,748
Firms 2,748
Failures 273
Failures 273
Failures 273
χ 743.4
χ 750.5
χ 776.8
Pseudo R² 24.1 Pseudo R² 24.3 Pseudo R² 25.2
35
Table 4 Default Probabilities
The table reports default probabilities that are calculated at the end of each September between 1985 and 2009
for the model in Shumway (2001), Campbell et al. (2008), Christidis and Gregory (2010), Taffler (1983) z-score
model and the naïve distance to default model in Bharath and Shumway (2008). We transform the z-score into
probability: p=e^(z-score) / [1+e^(z-score)]. In order to avoid look-ahead bias we take the coefficients from
failure indicators and predictors variables that are known at the end of each September. We display failure rates
(in percentage) for our entire sample, for the non-failed firms and for failed firms. The t-statistic indicates the
significance of the difference between the non-failed and failed group.
All Non-Failed Failed ΔF-NF t-stat
Shumway (2001) 1.10 1.03 8.35 7.32 12.34
Campbell et al. (2008) 0.90 0.84 7.21 6.37 12.92
Christidis and Gregory (2010) 0.99 0.92 7.86 6.94 12.66
Z-score 26.31 25.85 74.14 48.29 18.27
Bharath and Shumway (2008) 10.81 10.38 56.35 45.97 18.92
36
Table 5 Area under ROC Curve
The table reports the area under receiver operating characteristics curve (AUC). At the end of each September
between 1985 and 2009, we calculate the default probabilities from the models of Shumway (2001), Campbell et
al. (2008), Christidis and Gregory (2010), Taffler (1983) z-score and the naïve market based distance to default
model in Bharath and Shumway (2008). In order to avoid look-ahead bias we take the coefficients from failure
indicators and predictors variables that are known at the end of each September. Panel A. contains the AUC and
standard errors. Column z reports the z-statistic for difference in AUC to a random model (AUC of 0.50). Panel
B. reports the significance for difference in AUCs.
Panel A. Area under ROC
Model AUC SE z
Shumway (2001) 0.885 0.0151 25.55
Campbell et al. (2008) 0.878 0.0154 24.47
Christidis and Gregory (2010) 0.878 0.0154 24.46
Z-Score 0.814 0.0180 17.48
Bharath and Shumway (2008) 0.855 0.0165 21.54
Panel B. Difference in Area under ROC
Shum Camp Greg Z BS
Shumway (2001)
Campbell et al. (2008) 0.90
Christidis and Gregory (2010) 0.90 0.01
Z-Score 5.03 4.49 4.48
Bharath and Shumway (2008) 3.68 2.78 2.77 2.57
37
Table 6 Information Content Test
The table reports different specifications of logit regressions. At the end of each September between 1985 and
2009, we take the scores from the model in Shumway (2001), Campbell et al. (2008), Christidis and Gregory
(2010), Taffler (1983) z-score model and the naïve distance to default model in Bharath and Shumway (2008).
Beta is the beta factor of each firm calculated for each firm on a monthly basis over the previous twelve months.
MV is the log of market capitalisation. BM is the book-to-market equity. PYR is prior year return. RATE is the
sample failure rate over the previous twelve months. In order to avoid look-ahead bias we take the coefficients
from failure indicators and predictors variables that are known at the end of each September. Scores are within
±18.4207. Other variables (except Rate) are winsorised at 5%.
Model 1 2 3 4 5 6 7 8 9
Shumway 1.01
0.79
(2001) (4.53)
(2.61)
Campbell et al.
0.86
0.64
(2008)
(4.10)
(2.29)
Christidis and
0.78
0.56
Gregory (2010)
(4.02)
(2.41)
Z-Score
0.08
0.07 0.04 0.04 0.05
(3.16)
(2.64) (1.13) (1.30) (1.51)
Bharath and
0.19 0.14 0.04 0.04 0.06
Shumway (2008) (2.28) (1.81) (0.48) (0.46) (0.77)
Beta 0.15 0.17 0.14 0.21 0.22 0.20 0.15 0.17 0.14
(0.88) (1.00) (0.82) (1.21) (1.25) (1.15) (0.88) (0.99) (0.83)
Size 0.06 -0.04 -0.05 -0.40 -0.35 -0.28 0.02 -0.06 -0.06
(0.30) (0.20) (0.25) (2.70) (2.30) (1.83) (0.10) (0.35) (0.36)
BM 0.25 -0.02 -0.15 0.11 -0.13 0.06 0.26 0.06 -0.02
(1.23) (0.13) (0.80) (0.53) (0.69) (0.30) (1.22) (0.29) (0.12)
PYR -0.03 -0.24 -0.45 -1.47 -0.66 -0.59 -0.04 -0.23 -0.27
(0.06) (0.40) (0.73) (2.46) (0.90) (0.86) (0.08) (0.36) (0.42)
Rate -0.10 -0.19 -0.05 0.09 0.04 0.00 -0.09 -0.15 -0.09
(0.29) (0.52) (0.13) (0.26) (0.12) (0.00) (0.26) (0.43) (0.25)
Constant -0.24 -0.41 -0.96 -3.74 -3.14 -3.38 -0.98 -1.22 -1.61
(0.28) (0.46) (1.21) (6.12) (5.15) (5.43) (0.93) (1.13) (1.79)
Obs 22,217 22,217 22,217 22,217 22,217 22,217 22,217 22,217 22,217
Log-likelihood 583.8 567.3 561.1 494.6 470.3 533.0 597.0 583.7 588.2
Pseudo R² 24.5 23.8 23.5 20.7 19.7 22.4 25.0 24.5 24.7
38
Table 7 Comparative Economic Value of Hybrid Models
The table shows the results of an illustrative example of a competitive credit market. We hypothesis three banks
using default probabilities from either the bankruptcy prediction model in Shumway (2001), Campbell et al.
(2008) or Christidis and Gregory (2010). At the end of each September between 1985 and 2009 we calculate
default probabilities. The banks reject all firms with probabilities that fall in the bottom 5.0% based on their
respective models while offering credit to all others at a credit spread derived using the centred default
probability of Burgt (2007) and Eq. (17). The bank with the lowest credit spread is assumed to grant the loan.
Market share is the total number of loans granted as a percentage of total number of firm years, defaults is the
number of firms to whom a loan is granted that went bankrupt, share of defaulters is the percentage of total
number of bankruptcies. Revenue is market size * market share * average credit spread, and Loss is market size
* prior probability of failure * share of defaulters * loss given default. Profit is Revenue - Loss. Return on assets
is profit divided by market size * market share. We assume the market size to be £100.0 billion, equal size loans,
loss given default to be 45.0%, and credit spread for the highest quality customers to be 0.30%. The prior
probability of failure is taken to be the same as the ex-post failure rate of 0.95% during the sample period.
Bank 1 Bank 2 Bank 3
Model
Shumway
(2001)
Campbell et al.
(2008)
Christidis and
Gregory (2010)
Credits 12,021 4,734 4,597
Market Share (%) 54.1 21.3 20.7
Defaults 47 34 34
Defaults/credits (%) 0.39 0.72 0.74
Avg. Credit Spread (%) 0.51 0.55 0.67
Revenue (£m) 273.8 117.5 139.1
Loss (£m) 95.0 68.7 68.7
Profit (£m) 178.8 48.8 70.4
Return on asset (%) 0.33 0.23 0.34
Return on RWA (%) 1.23 0.69 0.82