WRITING MATTERS: UTILIZING SAT

WRITING TO PREDICT STUDENT SUCCESS

College Board National Forum 2012 – Miami, Florida 8:45 – 9:45 Thursday, October 25

Session Agenda

Presenter Introductions:

Chris Lucier, Vice President for Enrollment Management, University of Vermont

Jessie Donavan, Senior Enrollment Management Analyst, University of Vermont

Pam Horne, Associate Vice Provost of Enrollment Management & Dean of Admissions, Purdue University

Brent Drake, Director of Enrollment Management Analysis and Reporting, Purdue University

University Profiles

Purdue University Analysis

University of Vermont Analysis

Considerations for Future Research

University of Vermont Profile

Founded in 1791

Traditional residential campus located in Burlington, VT

Public, high research university

Residency mix:

33% VT

67% Out of State

101 UG Programs

Fall 2012 Enrollment:

UG 10, 192

Grad 1,327

Med 446

SAT Ranges:

Critical Reading 560-670

Math 570-670

Writing 560-670

Purdue University Profile

Located in West Lafayette IN

Public, Research Very Intensive institution

30,776 undergraduates, 924 professional, 7,937 graduate students

Traditional residential campus

283 programs of study

Students are admitted directly to major (91% to degree granting major, 9% exploratory)

61% of applicants admitted

Among undergraduates – 58% Indiana, 26% U.S. non-resident, 16% international

SAT Interquartile ranges

Critical Reading 510-620

Math 550-680

Writing 510-620

Purdue University Analysis

SAT Validity @ Purdue

National Studies bi-variate correlation to first-year GPA for all

3 parts range from 0.25 to 0.4 (writing usually highest)

When corrected for attenuation range from 0.40 to 0.60

National Studies multiple correlations SAT only 0.3 to 0.5

When combined with other pre-college academic

predictors (HS GPA, AP courses, years of study) range is

0.3 to 0.7

SAT Validity @ Purdue

Bivariate and multiple correlations largely the same as

national studies

Typically Math and Writing show highest bi-variate

relationship to student success measures

Prediction of first-year GPA overall typically within 0.04 of

actual first-year GPA

Results similar with subgroups, greatest difference among

Black/African American students

Multiple Correlations At A Single Institution

Multiple correlations between measures of academic success and unit level student

entering academic profiles at one institution tend to lead to correlations of low to

moderate effect sizes (Bridgeman, McCamley-Jenkins, & Ervin, 2000)

Some reasons identified in the literature (Stumpf & Stanley, 2002)

Restriction of range inherent in any one institutions academic profile

The interdependence of academic profile variables from high school

transcripts and standardized tests

The disparate academic experience of students in one institution

Multiple Correlations At A Single Institution

Stumpf and Stanley (2002) proposed examining grouped data across multiple

institutions to correct for the attenuation of existing relationships

Stumpf and Stanley (2002) utilized group variables of 25th and 75th percentiles

of standardized test scores and the percentage of a class with a high school

G.P.A. >= 3.0 to predict six-year graduation rates.

Purdue Replication

Predictor variables were data items readily available in two national data sets

Integrated Post-Secondary Education Data System (IPEDS)

25th and 75th percentile of SAT Scores

US News and World Report National Colleges data set

Percentage of students in top ten percent of high school class

Criterion variable was six-year graduation rates pulled from IPEDS

Final data set consisted of 199 institutions

Multiple correlation for model R2 = 0.8025

Use national model at university level to predict success among

different subgroups

How well are individual colleges performing versus their

predicted performance

Utilizing the SAT to Predict “Success” at UVM

Predictive Modeling @ UVM

We have built an econometric model that predicts a student’s

“success” at UVM

Success is defined as 2nd semester GPA

Historically the model has included high school rank, a high

school quality index and the highest combined SATR+SATA

SAT Writing @ UVM

In the summer of 2011 we analyzed data from classes

entering in Fall of 2009 and Fall of 2010

We found that the SATW score had a higher correlation

with 2nd semester GPA than SATR+SATA

This was the case even in colleges such as engineering

Including both the SATW and SATR+SATA in the model

strengthened the predictive value of the model

Utilizing SAT with Student Record

While there is a strong correlation between SAT

scores and 2nd semester GPA, you cannot build a

strong predictive model with SAT score alone

SAT scores are best used in conjunction with other

factors from a student’s record such as class rank,

high school GPA and high school quality

A student’s SAT scores account for 1/3 of the

predictive value of our admissions model

Other Uses of the SAT at UVM

Honors College Selection Process

Scholarship qualification

Awarding of financial aid need

based funds

Relationship Between SAT Writing

and 2nd Semester GPA

SAT Writing Band Avg. 2nd Semester GPA

350-500 2.73

500-540 2.85

550-590 2.95

600-640 3.07

650-690 3.13

700-740 3.31

750+ 3.31

Advice For Predictive Models

If don’t have the expertise and resources within

your institution use College Board ACES

Doing so also adds to national validity studies

Critically analyze your own models annually or

at least every two years

Statistically

Against your goals

What are you trying to predict and why

Recommendations for SAT Writing

If you require writing scores, use them in admissions and scholarship

selection

If you collect writing scores, report them to stakeholders

Trustees, executive officers

Faculty

State educational offices

K-12 educators

Prospective students

Continue to educate press and others regarding 2400 scale!

Considerations for Future Research

Difference between SAT Writing and Critical Reading among

international students

Other measures of academic success beyond first-year GPA

one- and two-year retention

4- and 6-year graduation

time-to-degree

grades in subsequent writing-intensive courses

Questions and Comments?