institutional research and sas: the right information, for the right decision, at the right time dr....

Institutional Research and SAS: The Right Information, for the Right Decision, at the Right Time

Dr. Joe DeHart

Executive Director of Institutional Effectiveness and Assistant to the

President

Des Moines Area Community College (DMACC)

Institutional Research and SAS:DMACC (Pronounced Dee-Mack)

• ~18,000 fall enrollment (PT > FT)

• ~600,000 service area population

• 6 campuses (I urban, 2 suburban, 3 rural)

Institutional Research and SAS:Enrollment Management and Marketing

“Institutional research and data, both historical and real-time, have been essentialessential to DMACC’s ability to increase enrollment and retain students.

Being able to access data effectively provides provides insightinsight into our core services and their impact on students’ lives…

and allows us to focus focus enrollment management and marketing efforts in ways that add value add value to our students’ experience at DMACC.

This directly benefits the studentbenefits the student and the College. ”Rob Denson, President

Institutional Research and SAS:AGENDA

A. Evolution of Institutional Research

B. Choosing a tool to facilitate this evolution

C. SAS’s role in this evolution at DMACC

1. IR Tool (examples)

2. Business Intelligence Tool (examples)

3. Predictive Analytics (examples)

D. Conclusion/ Next Steps

E. Questions?

Institutional Research and SAS:Evolution of Institutional Research

Facilitation Model

(Pre- SAS)

Decision Maker

IT Staff

Programmer

Data

IR Office


Decision Maker

IT Staff

Savvy IR Model (SAS)

IR Office

Data


Decision Maker

Data IT Staff

IR Office

Business Intelligence Model (SAS BI)


Decision Maker IT Staff

IR Office

Emerging Model (SAS BI, Predictive Analytics)

Data

Predictive Analytics

http://images.google.com/imgres?imgurl=http://nalts.files.wordpress.com/2006/06/crystal-ball.jpg&imgrefurl=http://nalts.wordpress.com/2006/06/21/controversial-and-unfounded-predictions-of-online-video/&h=546&w=400&sz=32&hl=en&start=3&um=1&tbnid=dmVaC3QrZNE9lM:&tbnh=133&tbnw=97&prev=/images%3Fq%3Dcrystal%2Bball%26svnum%3D10%26um%3D1%26hl%3Den%26rls%3DGGLD,GGLD:2005-19,GGLD:en%26sa%3DN

Institutional Research and SAS:Why SAS to Achieve our Goals?

Explosion of BI companies and products…. Oracle Discoverer, Pentaho, Web FOCUS, Business

Objects, Crystal Reports, Cognos, MS Access, SPSS, Microsoft BI, ProClarity, SAP, MicroStrategy, etc., etc.

How did DMACC come to settle on SAS for our solution?

1. It met our predefined objectives and requirements

2. I had used the software before

Institutional Research and SAS:Desired Objectives and Requirements

OBJECTIVES:• Better, consistent data• Improved Access• Improved User

Experience• Faster• Improved Efficiency

REQUIREMENTS:• Integrate with Active Directory• Control content based on user• Automation• Web and email based• No web programming• Drill-down• Appropriate for one-person • Fits in the budget

KNOW THESE AHEAD OF TIME!

SAS allows me to start with an issue and...• Gather only the data needed from Banner • Aggregate and analyze the data using descriptive

and/or inferential statistics• Quickly produce a final report

(tables, graphs, etc) in Adobe,

Word, Excel or html format

However…

1. SAS learning curve

2. Must know your data

Institutional Research and SAS:Getting to the IR Savvy Model

A. PC based (Windows)

B. Access to Banner Oracle Database

C. Access to all ODBC data

D. Two options:

1. Base SAS (SAS code)

2. SAS Enterprise Guide (GUI)

E. USED EVERY DAY

Enrollment Management Example

Institutional Research and SAS:Getting to the IR Savvy Model

“Can you tell me how many students are enrolled for each Dean by their delivery

method (online, HS, etc)?”

• Typical

• Start to Finish

• Took 5-10 minutes to create

• Took 6 seconds to run

Institutional Research and SAS:Getting to the IR Savvy ModelEnrollment Management Example

Finished Output:



Semi-technical skills to be expected from your IR person/department

1. One sql statement gather data from three banner tables

2. One procedure to aggregate the data

3. Output: file creation, look and feel, titles, footnotes, etc.


Once the program is written…

• Reusable

• Easily modified to show data for previous years

• Captures multiple steps

• Can be “upgraded to BI”

Institutional Research and SAS:From IR Savvy to BI

BI options in use at DMACC

1. Running a “canned” report

2. Creating a report from a dataset

3. Running an individualized report

4. Predictive Analytics (infancy)


What if the user wants this report each semester?

1. Ask me each term?

2. Incorporate into BI• Put report on server• Available on demand• Not person dependent• Others might want the same info

3. Point-and-Click user interface


User experience:


Access to Data: User-defined Queries• Data sets created, maintained, updated daily

– Current term enrolled students, dropped students, all students (enrolled and dropped)

– Past census data (term and annual)– Graduate follow-up data– Recruitment data

• Single version of the truth!• User saved reports (personal or shared)• All faculty and staff (not everyone sees all fields)


Recruiter Example (Report Wizard)


Recruiter Example: Output


Individualized Reports:

Because you log in, SAS knows who you are and retrieves information only for you

Examples

• Class Lists

• Grade Distribution Comparison

• Retention Comparison

• Faculty In-service Totals

Institutional Research and SAS:BI and Predictive Analytics

Up to this point…

Current and historical data

Predictive analytics lets us use past data to make better decisions regarding what is likely to happen in the future


It is a set of rules that use different statistical methods to uncover hidden patterns in the data.

Changes how research has traditionally been done.

Book: Competing on Analytics, Harvard Business School Press


Community College use of predictive modeling?• Recruitment- recruit students most likely to be

successful• Advising- ability to identify students who need

academic advising before there is a problem• Placement- students likely to struggle can be

placed in various assistance programs• Efficiency- provide services where they are

needed most• Early warning- identify at-risk students early• Fundraising- predict past donors likely to give

again


Predictive modeling at DMACC:

What is the likelihood …

1. of a first-time student to be successful in their first term?

2. of a first-time, full-time student to graduate in 3-years?

3. of a first-time student this fall to persist to the spring term?

1. Previous Career Advantage Credits

2. Total Credits Enrolled in First Term

3. Student Type (new, return, etc)

4. Sex

5. Race/Ethnicity

6. Highest Level of Education

7. First Term Type (fall, spring, summer)

8. Student Intent (transfer, job skills, etc)

9. Days Between Registration and Start

10. Days Between Application and Accept

11. Days Between Accept and Start

12. Residency Status (on application)

13. ESL Status

14. Campus (on application)

15. Days Between Accept and Registration

16. Type of Admission (Guest, New, etc)

17. Department (SH, BI, IT, etc)

18. Degree Sought (AA, AS, etc)

19. Program

20. First Generation Status

21. Single Parent Status

22. City

23. High School

24. Amount Offered for PELL

25. Age

26. Developmental Education


Some of what we know about students before classes begin

• First-term Success: 3 yrs, 30,000 students

• Graduation: 3 cohort years, 7,000 students

• Persistence: 4 yrs (fall), 18,000 students



• Mean: 56% successful• Top 2 deciles: 80%• Bottom decile: 20%

•Mean: 23% graduate•Top decile: 60% •<= 3rd decile: ~Mean

• Mean: 62% persist

• Top 3 deciles: 80%

• Bottom 2 deciles: 30%

First-term Success Graduation- 3yrs Persistence


How to best serve 3,500 students

First-term SuccessN=3,500

Top - probably successful with minimal services

Bottom- less likely to impact their success or failure (Title 3)

Middle - Most likely to impact their success

First-term Success- Evaluation: Predicted v. Actual

Fall 2006: (In Model)

Number Percent

Failure Success Failure Success0 1 . 100.00 .0.1 109 13 89.34 10.660.2 189 66 74.12 25.880.3 296 191 60.78 39.220.4 477 400 54.39 45.610.5 446 528 45.79 54.210.6 201 353 36.28 63.720.7 77 309 19.95 80.050.8 56 367 13.24 86.760.9 18 139 11.46 88.54

Fall 2003: (NOT in Model)Number Percent

Failure Success Failure Success0 58 5 92.06 7.940.1 813 9 98.91 1.090.2 219 70 75.78 24.220.3 164 115 58.78 41.220.4 250 202 55.31 44.690.5 249 286 46.54 53.460.6 178 270 39.73 60.270.7 187 287 39.45 60.550.8 97 388 20.00 80.000.9 19 150 11.24 88.761 . 4 . 100.00



Graduation- 3yrs- Evaluation: Predicted v. Actual Fall 2001 Cohort (not in model)

Number Percent

Failure Success Failure Success0 78 20 79.59 20.410.1 540 131 80.48 19.520.2 510 125 80.31 19.690.3 242 102 70.35 29.650.4 114 62 64.77 35.230.5 36 36 50.00 50.000.6 14 17 45.16 54.840.7 6 13 31.58 68.420.8 2 7 22.22 77.780.9 1 2 33.33 66.67

Fall 2002 (Not in Model)

Number Percent

Failure Success Failure Success0 64 12 84.21 15.790.1 440 77 85.11 14.890.2 482 102 82.53 17.470.3 295 124 70.41 29.590.4 150 75 66.67 33.330.5 56 55 50.45 49.550.6 28 28 50.00 50.000.7 12 18 40.00 60.000.8 2 10 16.67 83.330.9 1 1 50.00 50.00


F2S Persistence- Evaluation: Predicted v. Actual

Fall 2007 (In Model)Number Percent

Failure Success Failure Success0 5 . 100.00 .0.1 74 22 77.08 22.920.2 166 68 70.94 29.060.3 246 135 64.57 35.430.4 250 184 57.60 42.400.5 217 260 45.49 54.510.6 205 395 34.17 65.830.7 246 633 27.99 72.010.8 208 814 20.35 79.650.9 67 408 14.11 85.891 4 14 22.22 77.78

Fall 2004 (Not in Model)Number Percent

Failure Success Failure Success0 14 2 87.50 12.500.1 138 39 77.97 22.030.2 232 77 75.08 24.920.3 270 156 63.38 36.620.4 225 196 53.44 46.560.5 201 293 40.69 59.310.6 169 373 31.18 68.820.7 209 568 26.90 73.100.8 180 669 21.20 78.800.9 37 310 10.66 89.341 . 4 . 100.00


Once the Model is Shown to be Accurate:Scoring Students

Student

First Term Success Score

Persistence to Next Term Score

Graduation Score

Joe 0.23 0.27 0.08Wayne 0.51 0.51 0.22Bruce 0.49 0.61 0.29Deb 0.93 0.87 0.55Rob 0.82 0.67 0.14


Outstanding Issues with Predictive Modeling:

1. Profiling• Diverse populations• Proactive, not reactive (no denial of services)

2. Interfacing with Enrollment Management• Recruitment• Advising/Counseling

3. Model Maintenance


Next Steps:

• Key Performance Indicators

• IR priorities or help

• Succession planning for IR

• Scaling

• Cluster Analysis

Institutional Research and SAS:

Thank you!

Questions and Comments?

Contact info:

Joe DeHart

[email protected]

(515) 964-6279

mailto:[email protected]

institutional research and sas: the right information, for the right decision, at the right time dr....

Documents

datainstitutional research

ruralinstitutional research

runinstitutional research

ir savvy modelpc

ir savvy modelcan

microsoft bi

web focus

right decision