Suspecting the Rating Agencies

Dan diBartolomeoNorthfield Information ServicesJanuary 2012

The Matter At Hand

One of the largest contributing factors to the Global Financial Crisis of 2008-2009 was the huge number of fixed income instruments with very high ratings (e.g. AAA) that were either severely downgraded or went into actual default.

All three of the major rating agencies, Moody’s, Standard and Poor’s, and Fitch were shown to be seriously deficient in their ratings of a variety of debt instruments, particularly complex instruments (e.g. Credit Default Obligations or CDOs).

The rating agencies failed again with the obligations of major financial institutions such demonstrated by the spectacular collapses of firms such as AIG, Bear Stearns and Lehman Brothers.

Some Unpleasant Outcomes

From the first quarter 2005 through the third quarter of 2007, two thirds of CDOs S&P had rated were downgraded. 44% of all CDOs were

downgraded to “speculative” or “in default”

Over the same time, 17% of sub-prime residential mortgage securities were downgraded

9.8% were downgraded to “speculative” or “in default”

$250 Billion in major bank write offs from January 1, 2007 through April 2008, another $1.3 Trillion estimated from April 2008 to date

When Money Is At Stake, Cheat!

S&P has admitted to “notching” ratings on many securitized instruments

If an issuer wanted a rating done in a hurry, S&P would simply assume a rating of one grade lower than whatever S&P or Moody’s had most recently rated the debt of the same issuer

To my mind, this is fraud The expectations of financial market participants is that

rating agencies actually conduct some form of credit analysis before issuing a rating

At a cocktail party, a securitization lawyer for S&P rebuked me, saying they had done nothing illegal, as there are no actual requirements for any analysis

Beating the Agencies at their Own Game

At previous QWAFAFEW events we have described an approach to combine equity factor risk models and structural models of credit risk to provide consistent measures of equity risk, default risk and default correlation. The most important metric arising from this process is a

“market implied” expected life of a firm. We employ this as a quantitative measure of the “sustainability” of firms

In this presentation we will show how the a simple model using the expected life metric captured bankruptcy risk during the Global Financial Crisis and was predictive of subsequent credit rating changes and defaults We will also show that a related simple portfolio strategy

would resulted in significant alpha for US fixed income portfolios during the most stressful portion of the GFC

Basic Contingent Claims Literature Merton (1974) poses the equity of a firm as a

European call option on the firm’s assets, with a strike price equal to the face value of the firm’s debt Alternatively, lenders are short a put on the firm assets Default can occur only at debt maturity

Black and Cox (1976) provide a “first passage” model Default can occur before debt maturity Firm extinction is assumed if asset values hit a boundary

value (i.e. specified by bond covenants) Leland (1994) and Leland and Toft (1996)

Account for the tax deductibility of interest payments and costs of bankruptcy

Estimate boundary value as where equity value is maximized subject to bankruptcy

Reverse the Concept: Sustainability

Instead of trying to estimate how likely it is that firm goes bankrupt, let’s reverse the logic

We will estimate the “market implied expected life” of firms using contingent claims analysis Formally, our measure is the median of the expectation of

the distribution of the life of the firm Makes different default probabilities for different bond

issues very natural as each maturity will lie at a different point in the survival time distribution

Firms with no debt can now be included since it is possible that they get some debt in the future and default on that

A quantitative measure of the fundamental and “social” concept of sustainability

Our Basic Option Pricing Exercise

Underlying is the firm’s assets with asset volatility determined from the equity factor model How volatile would a firm’s stock be if the firm had no debt? This is the volatility of the assets

Solve numerically for the “implied expiration date” of the option that equates the option values to the stock price Market implied expected life of the firm See Yaksick (1998) for numerical methods for evaluating a

perpetual American option

Include a term structure of interest rates so that as the implied expiration date moves around, the interest rate changes appropriately

Our Previously Published Research diBartolomeo, Journal of Investing, December 2010

Used equity volatilities from Northfield US Fundamental Model One year horizon for risk forecast Near horizon” model are more suitable but less history

available

Estimate monthly for all firms in Northfield US equity universe from December 31, 1991 to March 31, 2010

Study three samples: All Financial firms Non-financial firms

Sources of Time series variation Stock prices, debt levels, Northfield risk forecasts Mix of large and small firms, 4660 <= N <= 8309

A Digression on “Too Big to Fail”

For the full sample period of 1992 through March 31, 2010

Non Financials: Median 14.74, Cap Weighted 18.42 Revenue Weight 17.60

Financials: Median 22.28, Cap Weighted 17.06 Revenue Weighted, 7.86

“Too Big to Fail” is really real Risk taking is heavily concentrated in the largest

financial firms Risk taking has been concentrated in the largest

financial firms for at least 20 years

Quantifying “Sustainability”

MSCI KLD DSI 400 index of US large cap firms considered socially responsible, 20 year history Typically about 200 firms in common with the S&P 500

July 31, 1995 DSI 400, Median 17, Average 17.91, Standard Deviation

9.93 S&P 500, Median 14, Average 15.40, Standard Deviation

9.28 Difference in Means is statistically significant at 95% level

March 31, 2010 DSI 400, Median 30, Average 26.39, Standard Deviation

11.45 S&P 500, Median 30, Average 24.93, Standard Deviation,

10.92 Difference in Means is statistically significant at 90% but not

95%

Testing on Disjoint Sets (DSI NOT S&P, S&P NOT DSI) Statistically significant difference in means for every time

period tested

Results to “Sustainability” Equity Investing (1992 through March 2010)

Table 11

Mean Annual Leveraged

Monthly Cumulative Monthly Compound S&P Risk

Return ReturnStandard Deviation Return

Equivalent Return

Q5 Equal 1.33 713.77 9.15 10.90 7.45

Q1 Equal 1.03 790.86 3.64 11.50 12.83

Q5 Cap 0.77 251.60 6.62 4.98 4.76

Q1 Cap 0.79 414.32 3.78 7.77 8.26

S&P 5002 0.75 347.74 4.32 6.78 6.78

Combining Sustainability and MV (1992 through March 2010, 200 Max Positions)

Mean Monthly

Cumulative Monthly Standard Deviation

Annual Compound Return

Annual Sharpe Ratio

Q1MV

1.07 840.43 2.96 12.34 .81

Q5MV

1.77 2901.15 6.80 19.33 .71

A Simple Start on Credit Ratings

We combined rating levels from S&P, Moody’s and Fitch into a unified letter scheme

Each rating level was assigned a numerical value A rating of “AAA” was 10 on the numeric scale A rating of “D” (default) was 1 on the scale Intermediate levels of AA,A,BBB,BB,B,CCC,CC,C A “+” added .333 A “-” subtracted .333

The scale is convenient but does not reflect any actual differences in probability of default (PD) or economic “loss given default” (LGD)

Numeric Rating Values Based on Spreads As part of our normal fixed income analysis we

estimate “option-adjusted spreads” for about 6 Million fixed income instruments on a monthly basis

Measures the portion of bond yield not associated with time value of money. This is premium for credit risk and illiquidity The median of the OAS values for set of the category

members is used Monthly history available to 2001 Bonds are broken into about 800 categories based on rating,

geographic region of issue and sector Computational model assumes lognormal interest rates and

combines features from Fabozzi and Dattatreya (1989) and Black, Derman and Toy (1990).

We can keep “AAA” ratings at 10, and “D” at 1 but rescale intermediate levels inversely proportional to OAS

A Revised Numeric Scale From Spreads60 Months Ending June 2011

Rating Simple Numeric

OAS Spread BP

Spread Numeric

AAA 10 86 10

AA 9 89 9.96

A 8 107 9.73

BBB 7 149 9.20

BB 6 287 7.47

B 5 364 6.50

CCC 4 428 5.69

CC 3 455 5.35

C 2 494 4.86

D 1 801 1

Criminal Abuse of a Temp

We wanted a really clean history of US bond rating changes across all rating agencies

We sent a temp (a classmate of mine) to the Boston Public Library to hand collect every rating change published in Barrons from 1992 to 2008 Information was hand entered into spreadsheets and then

matched to issuers, in a partly automated , partly manual way

It took roughly four months of tough full time effort There are roughly 8500 events in the data set and

we have been able match about 6500 of those to entities with publicly traded equity

The good news is that “Steve” was so good that he is now a very valued member of our Boston tech support staff

Criminal Abuse Of An Intern Our summer intern was then left to do an “event

study” type model of the rating changes for the subset of US corporate bonds The big job is merging the “expected life” data derived from

equities to issue level bond data It sounds easy but it is really a mess to track the related

equity across mergers, acquisitions and spin-offs All data was standardized to make pooling across time

easier Dependent variable was the percentage change in

the “simple” numerical value of the credit rating Independent variables:

12 month percentage change in expected life as of prior month end

12 month change in the cross-sectional Z-score of expected life within the US equity universe

“Ethan” survived too

A Modest But Encouraging Result We converted all data to rank values within the

pooled sample

In-sample our model had a correlation of about 40% (R-squared = .16) A very high degree of statistical significance on coefficients

(T > 4) R-squared was higher for subsets of lower grade bonds (i.e.

NOT “A”)

Even with our simple model we could meaningfully predict subsequent changes in bond ratings These results are all conditional that a change in rating

would eventually take place since only such events existed in our data

Non-events (no rating change) were excluded from the sample by design

Perhaps our model would predict 14 of every 5 downgrades

Expanding Study to Full Data

Universe is all US corporate bonds in the Northfield “Everything Everywhere” model Typical size around 18,000 bond issues

Study period from December 31, 2005 to June 30, 2011

Minimum maturity one year Each bond is matched to contemporaneous expected

life of issuer Assignments are updated annually for mergers, acquisitions

Return performance calculations exclude bonds with price outliers at the start of the period

Things Go “Pear Shaped” In The GFC

It should be intuitive that bonds with higher ratings should be associated with issuers with longer expected lives Break all bonds into 20 rating categories (including “+” and

“-”) Calculate average expected life for all bonds in each rating

category Correlate the average expected life and our simple numeric

rating

At 12/31/2005, the correlation across categories was +.68 Sample size of 17445 issues

At 12/31/2007 (pre bailouts), the correlation was -.35 Sample size of 22069 issues

By 12/31/2008, (post bailouts) the correlation was +.27 Sample size of 20043 issues

A Simple Metric : Z Score of Expected Life Within Rating Category

At each year end starting at 2005 we convert the expected life of issuer for each bond issue to a Z score within rating category

A negative Z score indicates that our metric suggests that the firm is less creditworthy than the published rating

Sort universe of 22000 bond issues into quintiles by Z score

At 12/31/2006: Of the bottom quintile of 4400 bond issues, 2940 were from Wall Street firms that either went bankrupt, were acquired or needed major government assistance

The rogues gallery included: Bear Stearns (534 issues), Merrill Lynch (868), Lehman

Brothers (657), Morgan Stanley (257), CIT Financial (338), Countrywide (136) and Washington Mutual (24)

Nearly identical result for 12/31/2007

Z-score Within Rating January 2006 Through June 2011

US government intentions to mount the TARP bailout were announced on October 3, 2008 with most of the details filled in a couple weeks later.

At October 31, 2008, the cumulative Q1/Q5 return spread was more than 1200 basis points in less than three years on widely diverse portfolios (equal weight across issues).

The cumulative return spread peaked in December 2008 and declined back to almost exactly zero by June 2011.

The implicit and explicit guarantees by the US Treasury and Federal Reserve had essentially driven the perceived creditworthiness of corporate bonds back to pre-GFC levels

-4

-2

0

2

4

6

8

10

12

14Cumulative Q1/Q5 Return Spread

Other Agencies: The Lace Case

Lace Financial is a small US rating agency specializing in community banks, credit unions and insurance companies

Santoni and Arbia (2010) studied the effectiveness of the Lace ratings on small US banks

116 small banks rated by Lace went under in 2009 72% of the failed banks were rated “E” (lowest non-

investment grade) at least a year before actual failure 94% of the failed banks were rated “E” six months before

the actual failure Clearly, the Lace ratings were effective but maybe

it’s just an issue of answering an easy question. Analyzing a small community bank may just be that much

easier than a complex, global entity like Citigroup or Lehman Brothers.

Let’s Start at the Beginning In the late 1990s, Moody’s began to provide

credit ratings of securitized corporate loans (CLO) based on a method known as the Binomial Expansion Technique

Once a default probability had been estimated for loans in a pool, the probabilities of potential multiple defaults within the pool was based on simple binomial probability formulas that assumed that defaults would be fully independent across borrowers

To account for the obvious flaw that defaults are likely to be correlated, Moody’s introduced an adjustment called Diversity Scoring

In 1999, Northfield published a research paper criticizing BET and Diversity Scoring based on a client request

Moody’s Diversity Scoring To account for default correlation, Diversity

Scoring reduced the assumed number of issues in a loan pool

For example, if you had 70 loans in a pool, you might count that as 40 independent loans

Moody’s broke firms into 32 industries 2 credits in the same industry counted as 1.5 10 credits in the same industry counted as 4

Implicitly there is an assumption that defaults are correlated within industries, but never across industries Completely ignores potential for pervasive effects of

recession, war or other systemic influences Although Moody’s largely abandoned BET in

2005, many institutions such as AIG continued to use the method

“The Secret Formula That Destroyed Wall Street” Cover story in WIRED magazine, March 2009 Refers to the “Gaussian Copula” approach for

estimating default risk across a pool of loans, from Li (2000) Allows for closed form calculation of marginal risks as more

and more loans are added to the pool to be securities Requires an assumption of the expected correlation of

defaults The problem is credit risky instruments have high

skew in the payoff distribution, so the joint distribution will be Gaussian under the Central Limit Theorem only for very large numbers of independent events Even a small degree of correlation calls the method into

question

The Problem of Higher Order Dependence Imagine you want to allocate money to two hedge

funds based on traditional MPT The two portfolio managers happen to have

offices in the same building and meet for coffee every morning During conversation, they flip a coin. If the coin comes

up “heads” they hold the identical portfolio for that day. If it comes up “tails” they go long/short against each other

If the coin is fair, the time series of their portfolio returns will be zero It varies daily from +1 to -1. averaging zero

Being independent implies zero correlation, but zero correlation does not imply independence

We Go Blindly Forward with the Gaussian Copula Investment banks love the Gaussian copula because

now you can get joint default probabilities over any number of loans with the estimation of a single number, the default correlation The rating agencies such as Moody’s and S&P go along and

rate based on this method The problem is where to get the default correlation

assumption, since actual defaults were rare events, so statistical estimation from historical data is questionable

Bankers used observable correlations from changes in spreads in the credit default swap market

CDO/CDS Default Correlations

Assessing default correlations from credit swap curves and CDO trading was horrendously faulty

Once CDOs and CDO2 could be written on “generic ABS” index results instead of specific pools, they were a form of legalized gambling

The volumes in the CDO market were many times the volume of actual underlying loans against which to hedge credit risk, creating severe pricing distortions

The CDO/CDS markets were dominated by a few large players such as AIG, further distorting economic pricing relationships

Summary

The global financial crisis has many contributing factors, but the largest single contributor was the horrendous performance of fixed income credit ratings from the major rating agencies

In some cases, the quality of ratings was severely upward biased by business in considerations

In many cases the contributing factors were simply poor quantitative analysis in which mathematical convenience was allowed to take precedence over the conceptual rigor of the models

The shortcomings in the analytical methods applied to the RMBS, CDO and CDS markets were easily observed by those who were relatively free of conflicts of interest