cfpb proxy methods: implications on fair lending … use of proxies: implications on fair lending...

CFPB Proxy Methods:

Implications on Fair Lending Testing

October 21, 2014

Arthur Baines

2

Use of Proxies: Implications on Fair Lending Testing

Agenda

• When to Use Proxies?

• Which Proxy methods do Regulators use?

• How to calculate BISG Proxies?

• Are they accurate?

• How do Regulators apply proxies in fair lending analysis?

• Key challenges in analyzing disparities in dealer reserve or ‘mark-

up’1

1As defined in CFPB Bulletin 2013-02

3


When to Use Proxies?

• Relevant Products: Auto, Credit Card, non-HMDA mortgage

• Proxies may be effective in the context of monitoring for fair

lending compliance

• Underwriting outcomes

• Pricing outcomes

• Proxies have some significant limitations

1As defined in CFPB Bulletin 2013-02

4


Which Proxy methods do Regulators use?

• All agencies estimate race/ethnicity using proxies or neighborhood characteristics

based on publically available Census Bureau data

(http://www.visualwebcaster.com/FederalReserveBankSF/94628/event.html)

• CFPB uses Bayesian Improved Surname Geocoding (BISG) to estimate

race/ethnicity

• Federal Reserve Board uses Majority Minority approach (neighborhood) and

surname (Hispanic) or first name (gender)

• OCC and FDIC use traditional geographic, surname proxies

• BISG -- tested by researchers at Rand on health care claims data with known

race/ethnicity1

1Elliott, Marc N. et al, “Using the Census Bureau’s Surname List to Improve Estimates of Race Ethnicity and Associated

Disparities,” Health Serv Outcomes Res Method (2009) 9:69–83.

5


http://www.census.gov/genealogy/www/data/2000surnames/index.html

How to calculate BISG Proxies?

Step 1: Surname

Race/Ethnicity Share

Hispanic 1.5%

African American 33.8%

Asian/PI 0.4%

American Indian 0.9%

White 61.6%

2+ Races 1.8%

Total 100.0%Source: Census Bureau

Probabilities for Surname "Johnson"

6


*Important – BISG does not use the Intra-tract shares commonly used in other geography-based proxies.


Step 2: Geography

Race/Ethnicity

Tract

Counts

Intra-Tract

Shares*

U.S. 18+

Population

Count

Share of

U.S.

Hispanic 1,340 24.5% 36,138,485 0.0037%

African American 1,008 18.4% 27,327,470 0.0037%

Asian/PI 307 5.6% 11,637,514 0.0026%

American Indian 15 0.3% 1,600,043 0.0009%

White 2,693 49.2% 157,123,289 0.0017%

2+ Races 109 2.0% 3,177,961 0.0034%

Total 5,472 100.0% 237,004,762 0.0023%Source: Census Bureau

18+ Population of Tract 0050.02 - Washington, DC

7


1Elliott, Marc N. et al, “Using the Census Bureau’s Surname List to Improve Estimates of Race Ethnicity and Associated

Disparities,” Health Serv Outcomes Res Method (2009) 9:69–83.


Step 3: BISG Probabilities

Race/Ethnicity

Surname

"Johnson"

Tract

0050.02

Wash, DC

BISG

Probability

Hispanic 1.5% 0.0037% 2.3%

African American 33.8% 0.0037% 51.1%

Asian/PI 0.4% 0.0026% 0.5%

American Indian 0.9% 0.0009% 0.4%

White 61.6% 0.0017% 43.3%

2+ Races 1.8% 0.0034% 2.6%

Total 100.0% 0.0023% 100.0%Source: Census Bureau & CRA computations

8


Are race and ethnicity proxies accurate?

• Proxies built upon Census Bureau geography and surname share some

challenges.

• Overestimate the share of African Americans and Hispanics in the

portfolio,

• Fail to identify significant numbers of minority contracts,

• Errors are correlated with geography, FICO, income and relative

income,

• BISG subject to high error rates, but relatively lower than geography or

name alone.

9


Classifier or

Proxy Hispanic White Black

Asian/Pacific

Islander

American

Indian/Alask

a Native Multiracial

Reported 5.8% 82.9% 6.2% 4.5% 0.1% 0.4%

BISG 6.1% 79.7% 7.5% 5.0% 0.2% 1.4%

Surname Only 7.4% 75.4% 10.0% 4.9% 0.6% 1.7%

Geography Only 7.2% 78.6% 8.1% 4.8% 0.3% 1.0%

Souce: CFPB "Using Publically Available Information to Proxy for Unidentified Race and Ethnicity," September 2014

Table 2: Distribution of loans by race and ethnicity

• The overstatement of minorities populations is high under all three

scenarios, but relatively lower with BISG.

Are race/ethnicity proxies accurate?

Table 10: Classification Over Ranges of BISG Proxy For Non-Hispanic Black

Black BISG

Proxy Probability

Range

Total

Applications

(1)

Estimated

Black (BISG)

(2)

Reported

Black

(3)

Reported

White

(4)

Reported

Other

Minority

(5)

0-10 160,733 1,859 1,466 139,684 19,583

10-20 9,742 1,387 941 8,403 398

20-30 4,916 1,207 906 3,814 196

30-40 3,101 1,072 726 2,242 133

40-50 2,229 997 738 1,408 83

50-60 1,680 922 736 877 67

60-70 1,417 920 765 596 56

70-80 1,407 1,057 963 391 53

80-90 1,517 1,293 1,222 241 54

90-100 3,693 3,548 3,408 200 85

Total 190,435 14,262 11,871 157,856 20,708

Souce: CFPB "Using Publically Available Information to Proxy for Unidentif ied Race and Ethnicity," September 2014

10



Reproduced from CFPB White Paper, Summer 2014, “Using publically available information to proxy for unidentified race and ethnicity”

21% overestimation

BISG probabilities

highly inaccurate

11


Identified by

Proxy

Correctly

Identified by

Proxy

Not Identified

by Proxy

(false

negatives)

Percent

Wrongly

Included

(false

positives)

African American 12,874 3,379 2,547 832 10,327 19.8% 80.2% 24.6%

Hispanic 13,623 9,561 8,261 1,300 5,362 60.6% 39.4% 13.6%

Asian 7,341 4,072 3,524 548 3,817 48.0% 52.0% 13.5%

non-Hispanic White 157,834 129,793 123,447 6,346 34,387 78.2% 21.8% 4.9%

Proxy

Method Race/Ethnicity

Count of

Borrowers

in Group

Proxy = Yes

Actual = Yes

Proxy = Yes

Actual = No

Proxy = No

Actual = Yes

Percent of Actual Group

Comparison of Proxy Approaches at Identifying Race/Ethnicity

BISG-80%

African American 12,874 1,206 912 294 11,962 7.1% 92.9% 24.4%

Hispanic 13,623 1,349 1,005 344 12,618 7.4% 92.6% 25.5%

Asian 7,341 12 8 4 7,333 0.1% 99.9% 33.3%


Tract-80%

African American 12,874 569 462 107 12,412 3.6% 96.4% 18.8%

Hispanic 13,623 11,397 9,423 1,974 4,200 69.2% 30.8% 17.3%

Asian 7,341 3,946 3,405 541 3,936 46.4% 53.6% 13.7%


Name-80%

African American 12,874 5,001 2,544 2,457 10,330 19.8% 80.2% 49.1%

Hispanic 13,623 5,555 2,902 2,653 10,721 21.3% 78.7% 47.8%

Asian 7,341 458 251 207 7,090 3.4% 96.6% 45.2%


Tract-50%


12


Hispanic Black Asian White

2011 National 11.5% 19.2% 10.1% 6.9%

Year State

Percent of Household with No Vehicle

Comparison of Vehicle Ownership by Race/ethnicity and geography

2011 AL 4.0% 13.6% 2.8% 4.0%

2011 AK 9.8% 15.8% 3.2% 10.7%

2011 AZ 8.3% 13.8% 6.7% 6.3%

2011 AR 5.2% 14.3% 5.2% 4.8%

2011 CA 8.0% 14.9% 7.5% 7.0%

2011 CO 6.4% 13.7% 6.1% 4.9%

2011 CT 18.7% 22.6% 6.8% 6.1%

2011 DE 2.3% 10.7% 6.6% 4.1%

2011 DC 44.1% 40.7% 43.5% 33.1%

Source: Census Bureau, American Community Survey

Why are race and ethnicity proxies are not accurate?

13


BISG Probabilities for a Hypothetical Portfolio

0% 100%

There are two methods for using the BISG probabilities

1) Continuous or Proportional estimation: measures the change as we

move from lower probabilities to higher probabilities

• Requires the use of regression analysis

2) Threshold-based estimation:

- identify contracts with > 80% probability of being African American

- identify contracts with > 80% probability of being Non-Hispanic White

- compare the outcomes between groups

• Can be calculated with or without use of regression analysis

If no controls are used, the observed differences are referred to as “raw”

14


Key challenges in analyzing disparities in dealer reserve

• Portfolio level results

• Are they meaningful, or do they simply reflect the different pricing strategies

across dealerships?

• What controls will the CFPB allow?

• Is the information/data available?

• Dealership level results

• Low contact volume from most dealerships

• No identifiable minority contracts from most dealerships

• What do you do with the monitoring results?

• When and what do you communicate to the dealership?

• Remuneration?

• Do you ‘participate’ in the dealer reserve?

15


Where to start?

• Establish a base line

• Analyzing underwriting and buy rate may be helpful

• Understand what the Regulator/CFPB is going to see.

• Measure dealer reserve in bps (test sensitivity to other measures).

• Use BISG and test threshold vs continuous probability specification.

• Portfolio level:

• Calculate raw differences in likelihood and level of dealer reserve.

• Recalculate differences controlling for geography, basic deal structure, and available

competitive factors.

• Dealer level:

• Establish a contract volume screen

• Calculate raw differences in likelihood and level of dealer reserve

• Identify statistically significant results

• Develop a dealership escalation plan

16


Helpful Sources:

• Census Bureau Geography level data: • Data: http://www2.census.gov/census_2010/04-Summary_File_1

• Documentation: http://www.census.gov/prod/cen2010/doc/sf1.pdf

• Census Surname List: • Data: http://www.census.gov/genealogy/www/data/2000surnames/names.zip

• Documentation: http://www.census.gov/genealogy/www/data/2000surnames/surnames.pdf

• Additional research and articles • AUTOMOTIVE FINANCE - WILL DEALERSHIP FINANCE RESERVE GO THE WAY OF MORTGAGE

YIELD SPREAD PREMIUMS?

HTTP://WWW.CRAI.COM/UPLOADEDFILES/PUBLICATIONS/AUTOMOTIVE-FINANCE-FE-

WHITEPAPER-0313.PDF.

• HOW THE CFPB’S AUTO FINANCING RULE AFFECTS CONSUMERS; AMERICAN BANKER,

http://www.americanbanker.com/bankthink/how-the-cfpbs-auto-financing-rule-affects-consumers-1058204-

1.html; APRIL 10, 2013.

• COMPLIANCE IN THE INDIRECT AUTOMOTIVE MARKET, KEY ISSUES IN FAIR LENDING ANALYSIS;

ABA BANK COMPLIANCE;

HTTP://WWW.ABA.COM/PRODUCTS/BANKCOMPLIANCE/DOCUMENTS/BANKCOMPL_2013_09_EFEA

TURE.PDF; SEPTEMBER/OCTOBER 2013.

http://www2.census.gov/census_2010/04-Summary_File_1/



http://www.census.gov/prod/cen2010/doc/sf1.pdf

http://www.census.gov/prod/cen2010/doc/sf1.pdf

http://www.census.gov/genealogy/www/data/2000surnames/names.zip

http://www.census.gov/genealogy/www/data/2000surnames/surnames.pdf

http://www.census.gov/genealogy/www/data/2000surnames/surnames.pdf

http://www.crai.com/uploadedFiles/Publications/Automotive-Finance-FE-Whitepaper-0313.pdf










http://www.americanbanker.com/bankthink/how-the-cfpbs-auto-financing-rule-affects-consumers-1058204-1.html



















http://www.aba.com/Products/bankcompliance/Documents/BankCompl_2013_09_eFeature.pdf



Arthur Baines

Vice President

Financial Economics

Charles River Associates

202-662-7838

[email protected]