Michael R. Baye, Indiana University

Joshua D. Wright, George Mason University

Georgetown University School of Law

Law and Economics Workshop

October 2, 2009

The Impact of Economic Complexity &

Judicial Training on Appeals

Is Antitrust Too Complicated

for Generalist Judges?

Introduction

Antitrust analysis is becoming increasingly complex

Increasingly relies on economic experts

More mathematically rigorous and technically

demanding analysis

Shift from per se rules to a case by case, “effects-

based” approach

Sherman Act delegates to the judiciary identification of

“unreasonable restraints of trade”

Over the past few decades, merger analysis has

“come of age”

So, what impact does this complexity have on the

quality of judicial decision-making?

Economic Complexity and Judicial

Decisions

ABA Task Force published a report in 2006 on the role of economic evidence in federal court: “It is critical that judges and juries understand

economic issues and economic testimony in order to reach sound decisions”

Survey of antitrust economists found that only 24% believe that judges “usually” understand the economic issues

A number of potential solutions have been suggested: Expanding use of court appointed experts Increasing use of Daubert Creating specialized antitrust courts Providing basic economic training to judges

Economic Training for Judges George Mason’s Law and Economics Center

(LEC) began training judges in 1976 At one point, it was responsible for providing

economic training to about 40% of the federal judiciary

Criticism eventually arose that the program was designed to indoctrinate judges with a conservative, free-market oriented set of economic beliefs

Provides judges with economic knowledge they would otherwise not get

Many judges opt for such training

Rationale: it reduces time spent on cases and improves reputations (reduced appeals or reversals)

Our Paper Represents a first attempt to examine the effect of

economic complexity and assess training on the quality of judicial decisions in antitrust

Our findings: Economic complexity increases the likelihood that a

judge’s opinion is appealed (and reversed)

Judges with basic economic training have decisions appealed (and reversed) less frequently

Experience is a poor substitute for economic training for judges

Lends some support to the claim that antitrust analysis has become too complex for generalist judges

Four Categories of Data

Information extracted from judicial opinions

Universe of all rulings on substantive antitrust

claims by federal district court judges (641

decisions) and administrative law judges (73

decisions) from 1996-2006

Includes

Type of antitrust claim (merger, monopolization, price

fixing, etc.)

Plaintiff (FTC, DOJ, private party, State AG)

Date of decision

Whether the decision was appealed

Whether an appeal resulted in a reversal

Four Categories of Data (cont.)

Judge and court characteristics

Judges’ political ideology (political party of the

appointing President)

Judges’ post-graduate education

Judges’ prior antitrust experience (number of

prior AT decisions)

Federal appellate circuit to which the judge

belongs

Four Categories of Data (cont.)

Economic complexity of the case

Searched decisions and generated counts of 14

key terms one would expect to arise in complex

antitrust cases

Terms such as “Regression”, “Economic Expert”,

“Statistics”, “Econometrics” , “Economist”,

“Economic Report”, etc.

Whether judge received basic economic training

128 judges attended LEC economics training

seminars before the particular decision was

issued

Majority of Cases Involve Judges with

Little Antitrust Experience and are

“Simple” Cases

0.1

.2.3

Fra

cti

on

0 10 20 30 40 50Experience

0

.2

.4

.6

.8

Fraction

0 5 10 15 20+ Complexity

Distribution of Judges’ Prior Antitrust

ExperienceDistribution of Economic Complexity

Measures of the Quality of Judicial

Decisions Primary measure of the quality of an initial court’s

decision is a party’s decision to appeal Since a party incurs costs by appealing, such

appeals constitute a revealed preference – i.e. the party decides that the opportunity to overturn the decision is great enough to warrant the cost

All things equal, an appeal indicates the party’s belief, often formed with input from economic experts, that it can convince the court that the initial decision contained reversible error

Though appeals may be based in legal issues, these are inextricably linked with economic issues in the realm of antitrust cases

Measures of the Quality of Judicial

Decisions Secondary measure of the quality is the reversal

of decisions by appellate. Disadvantages of this measure: Reversals made by panels of decision-makers, so

difficult to control for training, ideology, and interaction among these decision-makers

Occurs conditional on an appeal, so greatly reduces sample size

Caveats Number of “complex” cases is limited

Not sufficient thickness in the data to separately control for each key term

Split decisions into “simple” (no key terms) and “complex” (any key term) cases

Some potentially important predictors of appeals not observed Stakes of the litigation may correlate with case

type and complexity, which are in the model Quality of legal representation and judge-specific

effects may also affect the appeal rate

The sample excludes cases that are settled and focuses on “close calls”

Potential endogeneity

Comparison of Means

Variable N Mean Std. Err

Complex Cases 222 0.505 0.034

Simple Cases 492 0.262 0.02

Combined 714 0.338 0.018

Difference 0.242 0.037

t -Statistic 6.51

Trained Judges 97 0.227 0.043

Untrained Judges 617 0.355 0.019

Combined 714 0.338 0.018

Difference -0.128

t -Statistic 2.49

Appeals: Impact of Economic Complexity and Basic Economic Training

Two-sample t-test with equal variances

Summary of Means Comparisons

Appeal rates differ greatly based on complexity

and training

Economically complex cases are 24.2% more

likely to be appealed than simple cases

Decisions authored by judges with basic

economic training are 12.8% less likely to be

appealed than those by untrained judges

Results are significant at the 1% level

Similar results when reversals are used as

indicator of judicial quality

Complex cases are reversed 9.1% more often

Untrained judges’ decisions are reversed 10.1%

more often

Appeal Rates and Training Levels Vary Greatly Across

Circuits, Case Types, and Plaintiffs

Identifier

Number

of Cases

Percent

Appealed

Percent

Complex

Percent

with

Trained

Judge

Percent

with Judge

Trained at

Time of

Decision

By Circuit

1 First Circuit 48 27.08% 18.75% 2.08% 0.00%

2 Second Circuit 131 23.66% 16.03% 16.79% 12.21%

3 Third Circuit 75 22.67% 20.00% 16.00% 14.67%

4 Fourth Circuit 46 36.96% 36.96% 32.61% 30.43%

5 Fifth Circuit 30 33.33% 20.00% 13.33% 3.33%

6 Sixth Circuit 47 23.40% 27.66% 34.04% 23.40%

7 Seventh Circuit 47 17.02% 27.66% 34.04% 25.53%

8 Eighth Circuit 22 36.36% 31.82% 18.18% 18.18%

9 Ninth Circuit 60 35.00% 28.33% 20.00% 16.67%

10 Tenth Circuit 42 28.57% 30.95% 30.95% 26.19%

11 Eleventh Circuit 54 25.93% 27.78% 22.22% 12.96%

13 Federal Circuit 39 30.77% 48.72% 2.56% 0.00%

14 FTC Admin Litigation 73 91.78% 78.08% 0.00% 0.00%

1 Merger 78 61.54% 73.08% 7.69% 2.56%

2 Monopolization 235 24.26% 27.23% 19.57% 15.74%

3 Robinson-Patman 33 18.18% 33.33% 12.12% 9.09%

4 Multiple Claims 146 34.93% 25.34% 16.44% 10.96%

5 Price Fixing/Conspiracy 222 35.59% 23.87% 21.62% 17.57%

1 Private 571 26.44% 21.89% 20.84% 16.29%

2 FTC 112 72.32% 74.11% 3.57% 0.00%

3 US DOJ 12 41.67% 58.33% 8.33% 8.33%

4 State Attorney General 19 21.05% 36.84% 21.05% 15.79%

ALL DATA 714 33.75% 31.09% 17.93% 13.59%

By Type of Case

By Plaintiff

Regression Analysis

Probit regressions

Dependent variable is an indicator for whether the case was appealed

Independent variables: Dummy for whether the case was “complex”

Whether the judge was trained

Interactions

Year of decision

Type of case fixed effects

Plaintiff fixed effects

Circuit fixed effects

Baseline Probit Regressions

(1) (2) (3) (4) (5) (6)

COMPLEX 0.236*** 0.227*** 0.152*** 0.166*** 0.131*** 0.107**

-6.05 -5.54 -3.52 -3.72 -2.79 -2.17

TRAINED -0.107**

-2.06

COMPLEX_TRAINED -0.053 0.072 0.06 0.093 0.087

-0.51 -0.64 -0.55 -0.83 -0.73

SIMPLE_TRAINED -0.125** -0.105* -0.109* -0.097 -0.108*

-2.06 -1.69 -1.76 -1.54 -1.68

YEAR -0.021*** -0.021*** -0.015*** -0.012***

-7.13 -6.43 -3.56 -2.79

FIXED EFFECTS:

Type of Case No No No Yes Yes Yes

Plaintiff No No No No Yes Yes

Circuit No No No No No Yes

Robust z statistics in parentheses

* significant at 10%; ** significant at 5%; *** significant at 1%

Baseline Probit Regressions Reporting Marginal Effect on Appeal Rate

(714 Antitrust Cases)

Summary of Baseline Results

Results similar to means comparisons

Complexity increases the appeal rate by 23.6% while economic training reduces appeals by 10.7%

Including interaction terms: Complexity increases appeal rate by 10%

Basic economic training decreases appeals by 10% in simple cases

Economic training has no effect in complex cases

Results robust to the addition of a time trend and fixed effects for case type, plaintiff, and circuit

Economic Training vs.

Prior Antitrust Experience

These results lend some support to reforms to

provide judges with greater economic training

What about proposals to create antitrust tribunals

to give judges repeated exposure to complex

antitrust issues? Does experience in antitrust

cases have an effect on the appeal rate?

Add “EXPERIENCE,” which measures the

number of previous antitrust decisions issued by

the judge, which has a small negative effect on

appeals (suggesting experience not a substitute

for training)

Economic Training vs. Prior Antitrust

Experience

(1) (2) (3) (4) (5) (6)

COMPLEX 0.235*** 0.227*** 0.152*** 0.166*** 0.130*** 0.107**

-6.03 -5.52 -3.52 -3.71 -2.78 -2.17

TRAINED -0.103*

-1.96

EXPERIENCE -0.002 -0.002 -0.001 -0.002 -0.002 -0.001

-0.78 -0.79 -0.44 -0.61 -0.66 -0.23

COMPLEX_TRAINED -0.047 0.075 0.065 0.099 0.09

-0.46 -0.66 -0.59 -0.88 -0.75

SIMPLE_TRAINED -0.121** -0.103 -0.107* -0.094 -0.107*

-1.98 -1.64 -1.7 -1.48 -1.65

YEAR -0.021*** -0.021*** -0.015*** -0.012***

-7.1 -6.4 -3.54 -2.78

FIXED EFFECTS:

Type of Case No No No Yes Yes Yes

Plaintiff No No No No Yes Yes

Circuit No No No No No Yes

(714 Antitrust Cases )

Robust z statistics in parentheses

* significant at 10%; ** significant at 5%; *** significant at 1%

Probit Regressions Reporting Marginal Effect on Appeal Rate

Controls for Antitrust Experience of Judges

Additional Robustness Checks

Alternative dataset with Federal District Court judges only FTC administrative litigation has higher appeal rate

(91.2%), greater levels of economic complexity (78.1%), and ALJs had no economic training (0%)

Judicial training vs. political ideology Perhaps decision to attend training captures

conservative or pro-business leanings. Control for PARTY

Judicial training vs. judge quality Perhaps propensity to get training reflects fact that

better judges seek to improve themselves. Control for QUALITY (Masters or Ph. D)

Results Robust For These Data and

Additional Controls

(1) (2) (3) (4)

COMPLEX 0.096** 0.096** 0.096** 0.096**

-2.06 -2.06 -2.06 -2.06

COMPLEX_TRAINED 0.08 0.082 0.08 0.08

-0.73 -0.75 -0.73 -0.7

SIMPLE_TRAINED -0.095* -0.094* -0.094* -0.095*

-1.69 -1.66 -1.66 -1.67

YEAR -0.010** -0.010** -0.010** -0.010**

-2.11 -2.11 -2.11 -2.11

EXPERIENCE -0.001 -0.001 -0.001

-0.2 -0.22 -0.17

PARTY 0.005 0.001

-0.14 -0.02

QUALITY -0.061

-0.78

FIXED EFFECTS:

Type of Case Yes Yes Yes Yes

Plaintiff Yes Yes Yes Yes

Circuit Yes Yes Yes Yes

Robust z statistics in parentheses

* significant at 10%; ** significant at 5%; *** significant at 1%

Probit Regressions Reporting Marginal Effect on Appeal Rate, 641 Antitrust Cases

(Sample of Federal District Court Judges)

Summary and Concluding

Remarks Decisions involving the evaluation of complex

economic evidence are appealed at a 10% higher rate

Decisions of judges with basic economic training have a 10% lower appeal rate in “simple” cases

Basic economic training does not reduce appeals in “complex” cases

Court appointed experts?

More advanced economic training?

Antitrust experience (repeated exposure to antitrust cases) does not appear to be a good substitute for economic training

Specialized tribunals?

Out of sample predictions?