Deposing an Econometrics Expert

Presentation to Boston Bar Association Business Litigation Committee

by

Roy J. Epstein, PhDExpert economic analysis for complex litigation

Adjunct Professor of Finance, Boston CollegeApril 9, 2008

Roy J. Epstein PhD

2

What is Econometrics? Combines economic theory, data, and

statistical methods

Mainstream tool in legal proceedings

Generates formulas to show causation (liability) and to estimate damages

E.g., did release of a pollutant lower property values and, if so, by how much

Roy J. Epstein PhD

3

Most Common Econometric Model—Linear Regression

Predicts “dependent” variable in terms of one or more “explanatory” variables, e.g.:

Crop Yield = 5*Rain + 2*Fertilizer

Coefficients of 5 and 2 “best fit” the rain and fertilizer data to crop yield

Sorts out individual effects of multiple causal factors, e.g.,: 5 bushels per additional inch of rain 2 bushels per additional ton of fertilizer

Roy J. Epstein PhD

4

Principal Outputs from Linear Regression

Estimated value of each coefficient in the regression equation

Test of “statistical significance” of each estimated coefficient

Not significant means a coefficient is statistically indistinguishable from zero, regardless of value actually obtained

Roy J. Epstein PhD

5

Clash of Models For same alleged conduct and facts:

Expert for one side typically finds large and statistically significant coefficients

Expert for other side typically finds small and/or statistically insignificant effects

Roy J. Epstein PhD

6

How Econometric Experts Reach Opposite Conclusions

Different results usually due to combination of:

Using different explanatory variables

Using different data

Using different statistical procedures

Deposition must explore each area

Roy J. Epstein PhD

7

If You Could Ask Only a Single Question at the Deposition

“What did you do to establish the reliability of your results?”

Roy J. Epstein PhD

8

Deposition Step 1—Discovery Opposing expert’s backup materials

Raw data and/or identification of exact sources Details of all data manipulations All regression runs, graphs, and other data analyses

considered Allow adequate time for your expert to

replicate/review

Roy J. Epstein PhD

9

Deposition Step 2—Planning Your Questions Opposing expert’s results usually sensitive to

assumptions involving choice of variables, data, and estimation procedures

Work with your expert in advance Identify key assumptions Know effect of adopting alternative assumptions

Questions should probe basis for opposing expert’s choices

Deposition Step 3—General Topics to Cover

Roy J. Epstein PhD

11

Estimated Coefficients Algebraic sign

Effect of explanatory variable in “right” direction?

Magnitude Implausibly large or small?

Statistical significance Did expert use 95% confidence interval?

Roy J. Epstein PhD

12

Variables Selection of explanatory variables

How many different models were estimated? How were they different? Did any yield contrary results?

What did expert do to establish chosen model was more reliable than alternatives considered?

Roy J. Epstein PhD

13

Data Reliability of data sources

Procedures used to construct data

Rationale for grouping of transactions (transaction, plaintiff, all customers, product, industry)

Rationale for time period chosen

Checks/controls for outliers (atypical data points)

Roy J. Epstein PhD

14

Estimation Procedures Ordinary Least Squares (“OLS”) most widely

used procedure but inappropriate in certain situations Adjustments may be needed for reliable

coefficient estimates

Tests exist to assess whether alternative procedures should be used Did the expert use them?

Case Studies

1) General Use of Regression: Ivy League Financial Aid Antitrust Litigation

Roy J. Epstein PhD

17

Assessing Market Impact of Alleged Conduct

DOJ sued MIT and Ivy League schools for colluding on financial aid awards Key issue: did challenged practices have

anticompetitive effect?

MIT used econometric model to analyze prices charged by national sample of schools

No evidence that alleged conduct raised prices

Roy J. Epstein PhD

18

Roy J. Epstein PhD

19

The Model Dependent variable: average price (tuition +

room and board) by school 14 explanatory variables to account for

different school characteristics No price effect of alleged collusion:

Controlling for other factors, MIT and Ivys charged $322 less than other schools

But effect not statistically significant, therefore indistinguishable from zero

2) Assumptions about Explanatory Variables: Estimating Profits in a Damages Claim

[a case last year in which Dr. Epstein was involved]

Roy J. Epstein PhD

21

Different Models for Profit Analysis Defendant produced two products, A and B

Defendant: overhead expenses caused by total sales (1 explanatory variable)

Plaintiff: separate effects on overhead from products A and B (2 explanatory variables)

Roy J. Epstein PhD

22

Importance of Choice of Explanatory Variables Defendant: each $1 increase in total sales

adds $0.40 in overhead (and statistically significant)

Plaintiff: sales of B have no statistically significant effect on overhead

Profitability of product B: Zero under defendant theory Substantial under plaintiff theory

3) Data Reliability (or Lack Thereof): the Conwood Case

Roy J. Epstein PhD

24

Conwood v. US Tobacco Plaintiff analysis relies on extreme data

outlier

$1 billion claimed damages, after trebling

Sustained after review by Supreme Court

Roy J. Epstein PhD

25

Data Outlier Skews Regression Result

Washington, DC

Informative Legal Decisions

Roy J. Epstein PhD

27

Selected Cases that Discuss Quality of Econometric Evidence Freeland v. AT&T Corp., 238 F.R.D. 130 (S.D.N.Y. 2006)

Issues: omitted explanatory variables, misuse of average prices In Re Methionine Antitrust Litigation (West Bend Elevator,

Inc. v. Rhone-Poulenc), 2003 U.S. Dist. LEXIS 14828 (N.D. Cal., August 26, 2003) Issues: omitted explanatory variables, irrelevant data,

improper/insufficient time period, improper estimation procedure Johnson Electric v. Mabuchi Motor America, 103 F. Supp. 2d

268 (S.D.N.Y 2000) Issues: unreliable data, implausible magnitudes of coefficients

Roy J. Epstein PhD

28

Summary Most econometric models sensitive to one or more

assumptions regarding: Choice of explanatory variables Appropriate data Estimation procedure

Regression results not reliable until sensitivities identified and explained

Deposition must address basis for opposing expert’s assumptions

For Further Information…

Roy J. Epstein, PhDExpert economic analysis for complex litigation

1280 Massachusetts Ave., 2nd Fl.Cambridge, MA 02138rje@royepstein.com(617) 489-3818Adjunct Professor of Finance, Boston College