phd rankings presentation

8/2/2019 PhD Rankings Presentation

1/25

PhD Rankings

Kat Bechkoff

Doina Chichernea


2/25

Overview

Purpose

Lit Review

Survey Model Survey Method

Findings

Conclusion Further Research


3/25

Purpose

1. Is it important torank PhDprograms?

2. Are there

appropriate PhDrankings in place?

3. Is it important torank the college or

the department?4. What factors should

be included in aranking system?


4/25

Lit Review

Two types of rankings

Objective (Publication-Based)

The Conroy-Dusansky and Dusansky-Vernon

Scott and Mitias Rankings

Subjective (Survey-Based)

US News and World Report

National Research Council


5/25

Lit Review

Publication-Based Rankings

Dominant recent approach

Look at outputs that are directly measurable

Publications

Number of pages published

Citations

Counts might be adjusted for perceivedquality or importance or they might be

adjusted to per capita counts


6/25

Lit ReviewPublication-Based Rankings

Pros Familiar to faculty and

deans Methods in place to insure

accuracy of data.Academic journalsconsidered are fairestmeasure of qualityresearch

Cons Publication counts provide

little evidence onimportance of research

Not easily subjected tostatistical testing

Problems: Which journals are

considered important?

How does one comparea page in one journalwith a page in another?

Over what time periodare the counts to bemade?


7/25

Lit ReviewSurvey-Based Method

US News and World Report Rankings

Mail surveys to dept. heads and directors of graduatestudies in those universities which had graduated at least5 PhDs in the past five years

Recipients are asked to assess each department on aninteger scale of 1 to 5 as to: reputation in terms ofscholarship, curriculum, faculty and graduate programs

Scale: 5 = distinguished, 4 = strong, 3 = good, 2 =adequate, and 1 = marginal

Each dept. was ranked based on its average response.


8/25

Lit ReviewSurvey-Based Method

Pros

Easy side by side

comparisons

Influence on newgenerations of grad

students

Cons

Vulnerable to

measurement errors

Surveys may notproperly reflect

research output

(takes time for

perceptions to

change)


9/25

Lit ReviewPublication vs. Survey Based Ranking

Ranking systems based upon publications oralternately based on surveys of reputation willpresent consistent findings

Correlation coefficients within the groups is in excess of .9

Correlations btw publication & survey basedrankings are high in statistical sense, yet slightly

lower than within groups differing underlyinginformation Correlation coefficients within groups were typically in the .

6-.8 range

(Vernon Dusansky 1998)


10/25

Lit ReviewMBA Rankings vs Research Rankings

Refer to different constituencies: Students and business practitionersAcademics

It is critical that accepted measures ofperformance capture the goals of multipleconstituencies

The risk when they do not is perverse learning- Universities will strive to meet only the measuredperformance objectives and disregard theunmeasured ones. (Meyer & Gupta 1994)


11/25

Lit ReviewMBA Rankings vs Research Rankings

Conclusion: Schools having strong MBA programs based

on popular press rankings (US News andWorld Report; Business Week) may not beespecially strong on the research dimension

Faculty research rankings are significantly

different from academic program rankings(Trieschmann et al 2000)

Focusing only on one of these measures(popular press MBA rankings) threatens a

schools long-term viability


12/25

Are PhD

Rankings

Important?

Current

Rankings?

Factors

College vs.

Department

Student-

Related

Factors

Marketing

Factors

Program

Factors

Selectivity

Factors

Faculty-

Related

Factors

Alumni-

Related

Factors

Research

Output

Factors

Placement

Factors

Model


13/25

Survey Methods

Web-based Questionnaire

7pt Likert Scale

Very important Very unimportant

2 Distributions

PhD Coordinators (21 respondents)

U.C. CoB Faculty (26 respondents)


14/25

Findings


15/25

1. Are PhD Rankings

Important?

Overall:

Important = 72.36%

Neutral = 12.76%

Unimportant = 14.88%

Conclusion: PhD rankings ARE considered

to be important!

Strongly

Agree

Strongly

Disagree

Faculty

30.8%

Coord.

47.6%


16/25

2. Are There Appropriate

PhD Rankings in Place?

Overall:

Agree = 36.17%

Neutral = 25.53%

Disagree = 38.3%

Conclusion: There are not appropriate

PhD rankings in place.

Strongly

Agree

Strongly

Disagree

Faculty

34.6%

Coord.

47.6%


17/25

3. Ranking the College vs.

DepartmentDepartment College

Conclusion: College anddepartment rankings are equally

important.

gree 74.46% 74.46%

Neutral 10.64% 10.64%

Disagree 14.89% 14.89%

Coord.

47.6%Faculty42.3%

Strongly

Agree

Strongly

Disagree

Dept.

Coord.

33.3%

Faculty

30.8% College


18/25

4. Factors

Student-

Related

Factors

Marketing

Factors

Program

Factors

Selectivity

Factors

Faculty-

Related

Factors

Alumi-

Related

Factors

Research

Output

Factors

Placement

Factors

We asked the respondents to rate each factor in terms

of its importance, since our purpose was to determine

which factors should be included in a PhD ranking.


19/25

Factors Faculty vs.

Coordinators

Wilcoxon Signed Rank test

Results indicated a significant

difference btw the response averages of

the two groups!

Test Statisticsb

-3.569a

.000

Z

Asymp. Sig. (2-tailed)

Faculty -Coordinators

Based on negative ranks.a.

Wilcoxon Signed Ranks Testb.


20/25

Factors Faculty vs.

Coordinators

Faculty members responses were more

varied than that of the coordinators. Overall, coordinators considered the ranking

factors more important than faculty did.

Descriptive Statistics

40 2.4425 .61925 1.19 4.2940 2.6927 .84218 1.15 4.46

CoordinatorsFaculty

N Mean Std. Deviation Minimum Maximum

Why was there a difference?


21/25

Objective Factors Vs.

Subjective Factors

Student-

Related

Factors

Marketing

FactorsProgram

Factors

Selectivity

Factors

Faculty-

Related

Factors

Alumi-

Related

Factors

Research

Output

Factors

PlacementFactors

Objective

CategoriesSubjective

Categories


22/25

Objective Subjective

Mean 2.733648542 2.263502455Variance 0.625635351 0.19205813

Observations 27 13

Hypothesized Mean Difference 0

df 37

t Stat 2.413533667

P(T


23/25

Overall Factor Importance

Most Important

1. Faculty support

2. Quality of job

placement

3. # of faculty

publications

(weighted by journalimportance)

Least Important

1. Undergrad student

to faculty ratio

2. Total # pages offaculty works

published

3. Work experienceof accepted students


24/25

Conclusion

Phd Rankings are important

There are not appropriate PhD rankings in

place

College and department rankings are

equally important

Objective factors are weighed heavier interms of importance than subjective

factors.


25/25

Further Research

Perhaps a more exhaustive set of factors

could be tested

Received Suggestions: Internatl exposure,Student true superstars, hands-on research,

one-to-one mentoring, # of quality PhD

seminars, student/professor match-up.

Increase sample size

Collect data on student opinions

phd rankings presentation

Documents