0 u s news & world report undergraduate college rankings ruth kallio, associate director for...

11

U S News & World Report U S News & World Report Undergraduate College RankingsUndergraduate College Rankings

Ruth Kallio, Associate Director for Institutional Ruth Kallio, Associate Director for Institutional ResearchResearch

Office of Budget and PlanningOffice of Budget and Planning

November 14, 2007 November 14, 2007 8:30 – 10:30 a.m., Michigan Room8:30 – 10:30 a.m., Michigan Room

Office of Budget & Planning Office of Budget & Planning

22

USN&WRUSN&WR Undergraduate College Undergraduate College Rankings BackgroundRankings Background

What do they rank?What do they rank? Undergraduate programs overallUndergraduate programs overall

Some specialized undergraduate programsSome specialized undergraduate programs Graduate and professional programs overallGraduate and professional programs overall

Specialized graduate & professional programmatic Specialized graduate & professional programmatic areas areas

When did they begin ranking programs?When did they begin ranking programs? 1983 undergraduate1983 undergraduate 1987 graduate and professional1987 graduate and professional

33

USN&WRUSN&WR UndergraduateUndergraduate College College RankingsRankings

BackgroundBackground How many undergraduate colleges and How many undergraduate colleges and

universities are ranked?universities are ranked? Institutions are grouped according to Institutions are grouped according to

Carnegie classifications and for some Carnegie classifications and for some categories, geographic regioncategories, geographic region

262 national universities262 national universities 266 liberal arts colleges266 liberal arts colleges 574 master’s universities in 4 geographic regions574 master’s universities in 4 geographic regions 320 baccalaureate colleges in 4 geographic 320 baccalaureate colleges in 4 geographic

regionsregions 1,422 institutions in total1,422 institutions in total

44

USN&WRUSN&WR Undergraduate College Undergraduate College Rankings BackgroundRankings Background

Who is U-M ranked against?Who is U-M ranked against? U-M is in the “national universities” U-M is in the “national universities”

category which includes:category which includes: Doctoral/Research Universities -- ExtensiveDoctoral/Research Universities -- Extensive Doctoral Research Universities -- IntensiveDoctoral Research Universities -- Intensive

USN&WRUSN&WR publishes two sets of national publishes two sets of national universities rankingsuniversities rankings

All national universities (N=262)All national universities (N=262) 59 of which we commonly use as “peers”59 of which we commonly use as “peers”

All public national universities (N=163)All public national universities (N=163) 33 of which we commonly use as “peers”33 of which we commonly use as “peers”

55

USN&WRUSN&WR Undergraduate College Rankings Undergraduate College RankingsBackgroundBackground

Where does U-M rank overall?Where does U-M rank overall? Prior to 1988: 7th or 8thPrior to 1988: 7th or 8th Since 1988: consistently between 21st and Since 1988: consistently between 21st and

25th25th-- Currently: 25th (in a tie with UCLA)-- Currently: 25th (in a tie with UCLA)

Where does U-M rank among public Where does U-M rank among public universities?universities? Consistently 2nd or 3rdConsistently 2nd or 3rd Currently: 3rd (in a tie with UCLA and Currently: 3rd (in a tie with UCLA and

behind California Berkeley and Virginia)behind California Berkeley and Virginia)

66

Ten-Year History of Ten-Year History of UM-Ann Arbor RankingsUM-Ann Arbor Rankings

YearYear Overall Overall ScoreScore

RankRank

1998-991998-99 8787 25th (tied w/ UCLA)25th (tied w/ UCLA)


2000-012000-01 7878 25th (tied w/ UCLA & UNC-Chapel Hill)25th (tied w/ UCLA & UNC-Chapel Hill)

2001-022001-02 7777 25th25th

2002-032002-03 7272 25th (tied w/ UCLA & Wake Forest)25th (tied w/ UCLA & Wake Forest)

2003-042003-04 7575 25th25th

2004-052004-05 7676 22nd (tied w/ Carnegie Mellon & Univ. of 22nd (tied w/ Carnegie Mellon & Univ. of Virginia)Virginia)


2006-072006-07 7575 24th (tied w/ Univ. of Virginia)24th (tied w/ Univ. of Virginia)


77

USN&WRUSN&WR Data Collection Data Collection ProceduresProcedures

Surveys of colleges and universitiesSurveys of colleges and universities Data are supposed to conform to national Data are supposed to conform to national

standards for formatting and definitions standards for formatting and definitions developed by college guide publishers in developed by college guide publishers in conjunction with representatives from the conjunction with representatives from the higher education community (aka, higher education community (aka, “Common Data Set”)“Common Data Set”)

92.4% of the institutions surveyed in 2007 92.4% of the institutions surveyed in 2007 respondedresponded

USN&WRUSN&WR will estimatewill estimate data for any data for any institution that cannot or will not provide institution that cannot or will not provide them with some or all of the necessary data.them with some or all of the necessary data.

Survey of presidents, provosts, and deans of Survey of presidents, provosts, and deans of admissions (51% response rate in 2007)admissions (51% response rate in 2007)

88

How does How does USN&WRUSN&WR arrive at the overall arrive at the overall score and ranking for each national score and ranking for each national

university?university?

Three types of componentsThree types of components ReputationReputation Measures of Educational SuccessMeasures of Educational Success ResourcesResources

99

Breakdown of the Breakdown of the ReputationalReputational Component Component

Peer Assessment RankingPeer Assessment Ranking 25.0%25.0% 12th12th Avg. academic reputation scoreAvg. academic reputation score

(survey of presidents, provosts, deans (survey of presidents, provosts, deans of admission)of admission)

Student SelectivityStudent Selectivity 15.0%15.0% 23rd23rd Avg. SAT/ACT scores of enrolleesAvg. SAT/ACT scores of enrollees 7.5%7.5% % of enrollees in top 10% of high school class% of enrollees in top 10% of high school class 6.0%6.0%

15th15th % of applicants who are admitted% of applicants who are admitted 1.5%1.5%

50th 50th

Weight in U-MOverall Score Rank

1010

Breakdown of the Breakdown of the Educational SuccessEducational Success ComponentComponent

Graduation & RetentionGraduation & Retention 25.0%25.0% 26th26th % of entering class still enrolled one year later% of entering class still enrolled one year later 4.0%4.0% 21st21st % of entering class who graduated in 6 years % of entering class who graduated in 6 years

or lessor less 16.0%16.0% 29th29th Graduation Rate PerformanceGraduation Rate Performance

(Difference between a school’s actual and (Difference between a school’s actual and predicted 6-year graduation rate for an predicted 6-year graduation rate for an entering class)entering class) 5.0%5.0% 33rd 33rd


1111

Breakdown of the Breakdown of the ResourcesResources Component Component

Faculty ResourcesFaculty Resources 20.020.0%% 69th 69th Avg. faculty salary + benefits adjusted for Avg. faculty salary + benefits adjusted for

regional differences in cost of livingregional differences in cost of living 7.0%7.0% % of undergrad class sections fewer than % of undergrad class sections fewer than

20 students20 students 6.0%6.0% 69th 69th % of undergrad class sections 50 or more% of undergrad class sections 50 or more

students students 2.0%2.0% 103rd103rd Student/faculty ratioStudent/faculty ratio 1.0%1.0% 73rd 73rd % of faculty with highest degree in their fields% of faculty with highest degree in their fields 3.0%3.0% % of faculty who are full-time% of faculty who are full-time 1.0%1.0% 83rd 83rd

Financial resourcesFinancial resources 10.0%10.0% 29th 29th Expenditures per studentExpenditures per student

Alumni GivingAlumni Giving 5.0%5.0% 83rd 83rd % of undergraduate alumni who donated money% of undergraduate alumni who donated money

to their schoolto their school


1212

U-M’s Strengths and Weaknesses U-M’s Strengths and Weaknesses in the Rankingsin the Rankings

U-M scores well on measures of U-M scores well on measures of reputation and educational successreputation and educational success

U-M’s scale causes it to rank less U-M’s scale causes it to rank less well on per capita resource well on per capita resource measuresmeasures

1313

USN&WRUSN&WR Supplementary Supplementary InformationInformation

Lists of leading institutions based Lists of leading institutions based on other information not included on other information not included in the rankings calculationsin the rankings calculations

Quantitative measuresQuantitative measures Racial diversityRacial diversity Economic diversity (U-M ranked 6th)Economic diversity (U-M ranked 6th) International StudentsInternational Students

1414

USN&WRUSN&WR Supplementary Supplementary InformationInformation

Qualitative informationQualitative information InternshipsInternships Senior CapstoneSenior Capstone First-Year Experience*First-Year Experience* Undergraduate Research/Creative Projects*Undergraduate Research/Creative Projects* Learning Communities*Learning Communities* Study AbroadStudy Abroad Service Learning*Service Learning* Writing in the DisciplinesWriting in the Disciplines

*U-M listed as a leading institution*U-M listed as a leading institution

1515

What affects the rankings from What affects the rankings from one year to the next?one year to the next?

A change in the A change in the USN&WRUSN&WR methodologymethodology

A real change in an institution’s dataA real change in an institution’s data A real change in the data for other A real change in the data for other

institutionsinstitutions A change in how many institutions A change in how many institutions USN&WRUSN&WR chooses to publish on a chooses to publish on a given rankinggiven ranking

1616

Issues/Concerns regarding Issues/Concerns regarding USN&WRUSN&WR rankings rankings

Affect of rankings on students’ decisionsAffect of rankings on students’ decisions Subjectiveness of the reputational componentSubjectiveness of the reputational component Subjectiveness of the assigned weight for each Subjectiveness of the assigned weight for each

componentcomponent Component items may not measure what they Component items may not measure what they

are assumed toare assumed to Consistency and definition problems found in Consistency and definition problems found in

data collectiondata collection Ability of institutions to manipulate their dataAbility of institutions to manipulate their data Rankings measures are biased in favor of Rankings measures are biased in favor of

private institutionsprivate institutions

1717

Issues/Concerns regarding Issues/Concerns regarding USN&WR rankingsUSN&WR rankings

Rankings falsely imply a precision in the Rankings falsely imply a precision in the numbers and a significant difference between numbers and a significant difference between one rank and the nextone rank and the next

Rankings do not capture some of the most Rankings do not capture some of the most important components of a student’s college important components of a student’s college experience -- especially learning outcomesexperience -- especially learning outcomes

1818

Alternatives to USN&WR Alternatives to USN&WR RankingsRankings

UndergraduateUndergraduate Media sponsored rankingsMedia sponsored rankings

Washington Monthly, Business Week, Kiplinger’s, Wall Washington Monthly, Business Week, Kiplinger’s, Wall Street JournalStreet Journal

OtherOther Gourman Report, Princeton ReviewGourman Report, Princeton Review

GraduateGraduate Media sponsored rankingsMedia sponsored rankings

Business Week, Wall Street Journal, etc.Business Week, Wall Street Journal, etc. OtherOther

NRC, Gourman ReportNRC, Gourman Report For more examples of other rankings see: For more examples of other rankings see:

http://www.library.uiuc.edu/edx/rankings.htmhttp://www.library.uiuc.edu/edx/rankings.htm Many of these share the same problems and Many of these share the same problems and

weaknesses of weaknesses of USN&WRUSN&WR rankings--if not more! rankings--if not more!

1919

Movements Away From Movements Away From RankingsRankings

Current protest by liberal arts colleges Current protest by liberal arts colleges on the value of on the value of USN&WRUSN&WR rankings rankings 40 institutions have pledged not to fill out 40 institutions have pledged not to fill out

the the USN&WRUSN&WR reputational survey and not to reputational survey and not to use the rankings in promotional effortsuse the rankings in promotional efforts

The national “Accountability” The national “Accountability” movementmovement Voluntary System Accountability (VSA)Voluntary System Accountability (VSA)

NASULGC, AASCUNASULGC, AASCU Annapolis Group, NAICU, The Education Annapolis Group, NAICU, The Education

ConservancyConservancy

2020

How OBP Uses How OBP Uses USN&WRUSN&WR RankingsRankings

In general In general notnot used to define our peers used to define our peers ““Official” peers (19 institutions)Official” peers (19 institutions) Big Ten peersBig Ten peers AAU peers (61 institutions)AAU peers (61 institutions)

Publish but do not endorse the Publish but do not endorse the rankings on our website rankings on our website (http://sitemaker.umich.edu/obpinfo/home)(http://sitemaker.umich.edu/obpinfo/home)

2121

How OBP Uses How OBP Uses USN&WRUSN&WR RankingsRankings

The rankings provide a quick source of The rankings provide a quick source of selected comparative information on our selected comparative information on our “peers”“peers”

Mainly use on an ad hoc basisMainly use on an ad hoc basis To identify other peers that may not be in our To identify other peers that may not be in our

traditional peer groups (e.g., Notre Dame for traditional peer groups (e.g., Notre Dame for undergraduate tuition comparisons)undergraduate tuition comparisons)

To narrow down our official peers (e.g., To narrow down our official peers (e.g., institutions with the most top ranked institutions with the most top ranked graduate programs in engineering and the graduate programs in engineering and the sciences for the Advanced Project)sciences for the Advanced Project)

0 u s news & world report undergraduate college rankings ruth kallio, associate director for...

Documents