College Rankings and What You Need to Know

Smith and US News Fall 2014

US News & World Report America’s Best Colleges:

History and Methodology

US News Rankings: History • Began in 1983 as a reputation survey; expanded to further data

collection • Institutions ranked based on Carnegie Classifications; Smith is ranked

among National Liberal Arts Colleges • Methodology changes since 2007

– Reduction in weight of selectivity, increase in grad rates – Addition of Pell Grants into graduation rate model – Addition of high school counselor reputation ratings – Increase in weight of graduation rate performance – Decrease in weight of reputation survey

• In recent years, multiple institutions have admitted publicly to providing false data to US News.

US News: Where Smith Ranks and Why

19 20 18 19

14

18 18 17 19 19

13 15

13 14 12 13

11

14 12

10

13

9 10 10 9 7 7

14

0

5

10

15

20

25

30

35

40

45

50

2015201420132012201120102009200820072006200520042003200220012000199919981997199619951994199319921991199019891988

Smith US News Ranking

US News: Top Liberal Arts Colleges 2015 (Published September 2014)

• Williams (1) • Amherst (2) • Swarthmore (3) • Wellesley (4) • Bowdoin (5) • Pomona (5) • Middlebury (7) • Carleton (8) • Claremont McKenna (8) • Haverford (8) • Davidson (11) • Vassar (11) • USNA “Annapolis” (13)

• Washington and Lee (14) • Colby (15) • Hamilton (15) • Harvey Mudd (15) • Wesleyan (15) • Bates (19) • Grinnell (19) • Smith (19)

Also: • Bryn Mawr (27) • Barnard (32) • Mount Holyoke (41)

4

30.0%

22.5%

20.0%

12.5%

10.0%

5.0%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

Graduation andRetention

Reputation Score Faculty Resources Selectivity FinancialResources

Alumni Giving

Weight in Ranking

US News: What Matters? • Reputation Survey

– Peer ratings and guidance counselor ratings. • Graduation and Retention

– The percentage of first-year students who graduate in 6 years or less. – The percentage of first-year students who are still enrolled the following fall.

• Financial Resources – Educational Expenditures per Student: Funds spent on research, public service, instruction, academic

support, student services, institutional support, and operations and maintenance, adjusted for the percent of students that are undergraduate.

• Faculty resources: – Faculty Salaries: Salary plus fringe benefits for all faculty, adjusted for cost of living in area. (Another

spending measure). – Faculty composition: The percentage that are full-time, with top terminal degree. – Class size: Percent of fall classes under 20 students and 50+.

• Selectivity – Admit rate, SAT scores, class rank of entering students.

• Alumnae Giving – Percent of alumnae of record who donated in any amount in a given year.

US News: What Matters?

“Spend money.

As much as possible. Mostly on faculty.”

Alumni Giving (36)

Graduation and Retention (35)

Selectivity (31)

Financial Resources (19)

Faculty Resources (29)

Reputation (10)

-25

-20

-15

-10

-5

0

5

10

15

Sub-rank Relative to Overall Rank 2015

What Goes Into the Rankings

US News Rankings and Sub-Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

Faculty Resources, 29 Selectivity, 31

Financial Resources, 19

Alumni Giving, 36

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Reputation 22.5%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

US News Reputation Survey

*Peer assessment score changed in 2011 to 100-point scale; data for 11-13 are converted to 5-point scale. HS Counselor scores have only been released since 2014.

4.3 4.3 4.3 4.2 4.2 4.4 4.2 4.3 4.2 4.2

4.4 4.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

2006 2007 2008 2009 2010 2011* 2012* 2013* 2014 2015

Peer Assessment HS Counselor Assessment

Graduation and Retention 30%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Six-Year Graduation Rates (Four-year Averages)

84% 84%

86% 86% 87%

86% 85% 85% 85% 85%

75%

80%

85%

90%

95%

100%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Note: Scale 75-100%

First-Year Retention ( Four-year Average)

91% 91% 91% 90% 90% 91% 91%

92% 92% 93%

75%

80%

85%

90%

95%

100%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Note: Scale 75-100%

Predicted vs. Actual Graduation Rate

85%

83%

81% 82%

83% 82% 82% 82%

81% 82% 84%

86% 85%

86%

88%

84% 85% 85% 85%

86%

75%

80%

85%

90%

95%

100%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Predicted Graduation Rate Actual Graduation Rate

Note: Scale 75-100%

Faculty Resources 20%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

Faculty Resources, 29

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Classes With <20 Students

68%

71% 70%

68% 69%

67% 68%

66%

68% 68%

50%

55%

60%

65%

70%

75%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Note: Scale 50-75%

Additional Faculty Resource Inputs

• Faculty salaries, including benefits (up 2.6%) • Percent of faculty with terminal degree (99%) • Classes with 50+ students (5%) • Student/faculty ratio 9:1 • Percent of faculty that are full-time (97%)

Selectivity 12%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

Faculty Resources, 29 Selectivity, 31

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Average SAT Scores

1270 1263 1250 1270 1274

1300 1306 1309 1309 1333

1000

1100

1200

1300

1400

1500

1600

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Admit Rate

57%

48%

53% 52%

48% 47% 47%

45% 42% 43%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Note: Scale 25-75%

Entering Students in Top 10%

59% 61% 61%

63% 64% 66%

60% 61%

68%

62%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Financial Resources 10%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

Faculty Resources, 29 Selectivity, 31

Financial Resources, 19

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Educational Expenditures per Student

$42,758 $42,284 $42,710 $44,132 $47,378

$50,008 $50,719 $51,422 $51,922 $55,300

$-

$10,000

$20,000

$30,000

$40,000

$50,000

$60,000

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Alumnae Giving 5%

US News Rankings

Smith Overall , 19

Peer Assessment, 10

Grad/Retention, 35

Faculty Resources, 29 Selectivity, 31

Financial Resources, 19

Alumni Giving, 36

0

10

20

30

40

50

60

70

80

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Alumnae Giving Rate (Two-Year Avg)

41%

35% 36% 38%

40% 40% 38%

36% 35% 35%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Note: Scale 0-50%

The Rankings in Context

Smith vs. Sisters 2015

Smith

Wellesley

Bryn Mawr

Mount Holyoke

Overall Rank 19 4 27 41 Overall Score 84 93 80 73 Reputation Score 4.2 4.4 4.0 4.0

Graduation and Retention Rank 35 12 50 50 Average Freshman Retention Rate 93% 96% 91% 91% Predicted Graduation Rate 82% 91% 85% 83% Actual Average Graduation Rate 86% 91% 84% 82% Over-performance/ Under-performance 4 0 -1 -1

Faculty Resources Rank 29 9 25 76 % of Classes with fewer than 20 students 68% 73% 75% 66% % of Classes with 50 or more Students 5% 0.4% 3% 4% Student/Faculty Ratio 9/1 7/1 8/1 10/1 Selectivity Rank 31 8 28 60 Standardized Test Score 1220-1450 1310-1510 1210-1470 1220-1430 First-years in Top 10% of HS Class 62% 83% 65% 57% Acceptance Rate 43% 29% 40% 47% Financial Resources Rank 19 8 19 38 Alumni Giving Rank 36 11 25 48 Average Alumni Giving Rate 35% 47% 38% 31%

Smith and the Rankings

• There is widespread frustration with US News, but it can’t be ignored.

• Top weighted inputs: – Graduation/retention rates – Reputation scores – Faculty resources

• Smith is strong in reputation and financial resources. • Smith is weaker on alumnae giving and graduation

rates.

Reality of Rankings

• Peers are tightly clustered; many multi-way ties. – In 2015, ties for 5th, 8th (3-way tie), 11th, 15th (4-way tie),

19th, 24th, and 27th (all 3-way ties). – In 2014, ties for 4th (3-way tie), 7th, 9th (3-way tie), 14th,

17th (3-way tie), 20th, 22nd, 25th (5-way tie). • Measures are highly constrained, largely by economic

factors. • Movement in rankings has been accomplished by a handful

of lower-ranked schools, but requires focused, costly efforts; top 10 is highly stable.

Efforts to Affect Rankings

• Most movement within 1-2 slots up or down is noise.

• Significant effort is required for small moves. • Movement gets harder nearer the top. • Any of these efforts entail significant financial

expenditures.

Talking about Rankings • Rankings largely measure inputs, not what happens in the

classroom or the graduates we produce. • US News strongly rewards institutional wealth and expenditures. • Small changes can have what appear to be large effects on an

individual school’s ranking. • All the top schools work on the same measures to stay on top. • Managing rankings metrics to promote improvement has been

demonstrated to be expensive and sometimes unethical (Emory, Claremont McKenna, George Washington, Bucknell, Tulane).

• Rankings don’t measure success in achieving the mission.

27%

47%

53%

56%

85%

0% 20% 40% 60% 80% 100%

Rankings in national magazines

I was offered finanical assistance

This college's graduates get good jobs

Graduates admitted to top grad/prof schools

Has a very good academic repuation

Percent of Entering Students Marking Reason for Choosing Smith as Very Important: 2013