University of ArkansasFaculty Senate Task Force on Grades

Preliminary Report

April 19, 2005

Mandates for the Task Force

Does grade inflation at the U of A exist and is it a significant issue of faculty concern that may require policy change?

Are grades and any perceived or real grade inflation with time, or uneven grade distribution linked to uniform faculty evaluations (Purdue System)?

If there IS a relationship between Teaching Evaluations and Grading, … is it a significant problem that should be addressed?

What are Task Force Recommendations related to grades, grade definitions, etc., … that should be a faculty matter.

U of A not Unique!

Academe Interested in Grades and Grading Practices

Duke University Harvard Dartmouth Southwest Missouri State University University of North Carolina Princeton

Assignment of Grades

Philosophical Discussion (Faculty)– Why do we assign grades?– What does a grade represent?– Types

Criterion or Norm Referenced

Pragmatic Examination (Task Force)– What grades were assigned?– Are there patterns in the assignment of these grades that

may represent a greater systemic problem?

Student Evaluation of Instruction

Philosophical Issues (Faculty)– Purpose of student evaluations?– Does the Purdue System provide the necessary information

to evaluate instruction?– Differentiating a high rating from effective instruction and

possible “inflation” of grades Pragmatic Issues (Task Force)

– Are There Patterns in Student Evaluations Associated with Overall Grades Assigned Type of Course Faculty Rank

Increasing Grades?

Since mid-60s universities have seen undergraduate GPAs steadily increase

College Remedial Courses– Annual rate approaches 60% in Arkansas– 34% of Students in Arkansas Identified as

“college ready”

Numerous theories exist and the issue is commonly referred to as “grade inflation”

Grade Inflation

Economic Description– Grades Increasing Over

Time (Dow Jones) Grade Inconsistent with

Demonstrated Achievement– Student Assigned Grade of

“A” with Limited Achievement

– Does not Accurately Reflect Performance

– Covering 9 versus 17 Chapters

0

2000

4000

6000

8000

10000

12000

14000

1985 1995 2005

DowJones

The “Gold” Standard

The goal is to create a common standard to compare students and grades assigned

Use of standardized test information– ACT– CAAP “Rising Junior” Exams– GRE

Requested Information for Study

Grades Assigned and Student Evaluations (1992-2004)– Year– College– Department– Program– Section

Example of Demographic Variables – First Year Enrolled

Status (Transfer/Freshman)– High School GPA– Degree and Year Completed

Note: 1992 – 2004 Represents the Time Frame Computerized Data Available

Federal Education Rights and Privacy ACT (FERPA)

Institutional Review Board – Provided Approval

Anonymous student and faculty identification numbers used

Additional Protection: If sample size was less than 10 observations an any analysis level, information was not reported.

Methodology

Checking the Data! Initial Calculations

– Computing student GPAs independently– Correlated 1.0 with University reported GPAs

Note: Computed with only fall and spring grades

Modifications for Analysis Purposes– Conversion of grades to whole values for specific analyses

Faculty Evaluations

– Use of means in analysis of faculty evaluations

Research Design

Exploratory Data Analytic Techniques– Tukey (1978)

Graphing Descriptive Statistics Confidence Intervals Trend Analysis

Layered Overview of Results– University

-- College -- Department

-- Program -- Course -- Section

Preliminary Results: UndergraduateU of A Grades from 1992 - 2004

Mean GPAs increased from 2.76 to 2.95 during 12 year period

2.76

2.95

College Undergraduate GPA Trends with U of A GPA Trends for 1993 - 2004

2004 GPAs

EDUC= 3.52

AFLS= 3.14

ENGR= 3.06

ARSC= 2.82

ARCH= 2.82

WCOB= 2.77

University and College GPA Trends 1992 – 2004: Rescaled

Rescaling on 0.0 to 4.0 grade scale may create a different impression

Rosenthal (1973), studies on interpreting graphs

3.52

2.77

Student Evaluations and Grades Assigned: Is there a pattern?

0

10

20

30

40

50

60

A B C D F

Percent of Responses

ExpectedGrade

0

1

2

3

4

5

A B C D F

University Core Average

ExpectedGrade

The correlation between expected grade and university core average was .29, suggesting only a moderate association

86% of students expect grade of A or B

Steady decline, then levels

Based on the six Teaching Evaluation items all instructors must ask students!

What Represents an Example of Effective versus Ineffective Grading

High or Low Grades?– Indicators of success?– Indicators of ineffective instruction?– Grade inflation?

Patterns!– Identify patterns in the data that demonstrate

cause for concern or discussion

Demonstrate a Process for Assessing the Assignment of Grades

College of Education and Health Professions– Department of Educational Leadership,

Counseling and Foundations Educational Foundations Program Graduate Degree Program

Goal: To provide a method for evaluating the assignment of grades in a program area!

Graduate GPAs at the U of A

2004 GPAs

EDUC= 3.77

ENGR= 3.70

ARSC= 3.66

AFLS= 3.65

WCOB= 3.55

Overall, graduate GPAs higher and have increased.

College of Education and Health Professions: Department Graduate GPAs

Interesting!

The GPAs by Dept. are becoming more variable

Educational Leadership, Counseling and FoundationProgram Area Graduate GPAs 1992 - 2004

Ed. Stat grades declining relative to rest of department

However, not the complete picture!

3.48

3.89

Educational Foundations: Evaluating our Grading Patterns1992 - 2004

Tenure track faculty assign grades much lower than adjuncts in Ed. Stats

Volatility due to one course

Implications of Internal Analysis

We …. We the faculty in Educational Research and Policy Studies need to do a better job of overseeing/selecting our adjuncts.

We …. We need to individualize the issue of grades, expectations, and assignment of grades.

We …. We need to examine the content level and determine if expectations are commensurate with benchmark institutions.

– Multiple Regression Example

Preliminary Conclusions

Grades are increasing!– Is this problematic?

Possible Explanations/Further Research– Higher entrance scores (ACT exams)– Transfer students– Transferring credits from other institutions (Community

Colleges, Universities, etc.)– Academic expectations– Course taking patterns by students– Plus, many others ….

Preliminary Recommendations

Stair-Step Evaluation of Grades Information Data Provided to Each College for Evaluations

– Committees to examine information at each level and report

-- College -- College Reports

-- Department -- Department

-- Program -- Program

-- Course -- Course

Evaluate -- Section -- Section

Ongoing Accountability

Next Steps

Final modifications of requested data received on March 18th .

Preliminary analyses completed, more in-depth study of issues during the next four months to understand the implications.

Comprehensive Report completed and submitted to Faculty Senate fall of 2005.