Achieving the DreamWinter Strategies Institute – 2010

Terri Manning, Carol Rush and Lane Glenn

This workshop will:• present the common requirements of the

six regional accrediting agencies and • show how Achieving the Dream strategies

can be constructed, delivered and evaluated to provide evidence of compliance with key accreditation standards and

• show how Achieving the Dream can drive an institution’s quality improvement agenda

• examples from Achieving the Dream colleges will be presented

72% of AtD colleges are accredited by the Commission on Colleges of the Southern Association or the Higher Learning Commission or North Central

While there are some differences among the accrediting agencies, all of them follow institutional quality improvement frameworks that require colleges to:•  Determine the conditions that affect the college’s faculty, staff,

and students, and the local community through needs assessment and environmental scanning.

• Establish appropriate goals and outcomes, with an emphasis on learning outcomes for courses, programs, and support services.

• Measure progress toward attaining the goals and desired outcomes.• Analyze the results of assessments and evaluations to document

the need for improvements in programs and services.• Develop interventions or strategies to increase student, faculty,

staff, and community success, and improve programs and services.• Demonstrate how these interventions and strategies have

improved institutional quality.

NC – Higher Learning Commission has two processes: PEAQ (self-study, compliance certification) and AQIP. The college can choose but must apply to be an AQIP college. They only do one – not both.

SACS – Commission on College has two processes: the self-study, compliance certification and the QEP (quality enhancement plan). All colleges do both.

Because all six include language such as• … strengthening educational quality• …. improving institutional quality• …. ensure the quality and integrity of its academic

programs It has become critical that institutions are

able to prove and document their processes for improving institutional quality. Where is that more important than:• Student success• Student learning

If we improved the quality of student learning, what would it look like? What would we observe? What could we measure?

If we could improve the quality of a student’s experience at the college, what would need to change? What would we observe and measure?

Assessment (especially student learning outcomes, more specifically, general education learning outcomes)

Quality – demonstrating it is hard – not part of our culture to say “look how great we are”

Tying strategies to planning and budgeting

Surveyed all round 1, 2, and 3 colleges in 2008.

One-third responded (N=20) We asked them questions about how

they were using their data, strategies, etc. for accreditation, strategic planning, obtaining additional funding and for institutional effectiveness issues at their colleges.

Of the 20 who responded to our survey:• 11 SACS, 2 NC, 2 NE, 3 MS, 2 NW

14 of the 20 core team leaders were directly involved in the accreditation process at their college.

8 are going through or have gone through the accreditation process since becoming an Achieving the Dream college.

Colleges are reporting that AtD has helped them launch an IE plan at their college ….or has guided or strengthened their IE processes (18 of 20).

Currently, all six accrediting agencies mention institutional effectiveness in their criteria. North Central only once - Middle State is throughout the document.

Institutional Effectiveness Definition•an ongoing, integrated and systematic set of

institutional processes that include planning, the evaluation of programs and services, the identification and measurement of learning outcomes, the use of data and assessment results for decision-making that results in improvements in programs, service and institutional quality.

1. Higher profile for IR and use of data generally (9)

2. Policy and budget decisions (7)3. Helped define and prioritize problems in

gatekeeper and developmental courses4. Matched learning gaps to appropriate

teaching strategies5. Targeted the most appropriate teaching

strategies6. Continued, refined, strengthened or

eliminated programs and practices

Half (10) of the responding colleges have developed a new college plan since becoming involved in Achieving the Dream.

Five colleges said they used Achieving the Dream goals and measures as a part of the annual strategic plan.

AtD influenced colleges to: Provide focus for annual planning retreats

– analyze the data, consider the true state of student success at college

Places focus on reaching existing learning goals

Impetus to modify parts of the strategic plan or realign with AtD

Update college mission and goals – more student focuses, success oriented

The Language of Institutional Effectiveness (not all agencies call it institutional effectiveness)

Planning and evaluation processes that are: systematic, comprehensive, broad-based, on-going or continuous, integrated, and appropriate to the

institution, involve individuals and groups, short- and long-term, realistic analyses of internal and external opportunities and constraints, use of data necessary to support planning, to improve the quality of education, responding to future challenges and opportunities, to influence resource allocation and to improve its instructional programs, institutional services, and activities, systematic review of institutional mission, goals and outcomes.

The Emphasis on Learning Outcomes

Appears to be the hardest part of accreditation for most colleges. The assessment of learning outcomes needs to:

•be clearly stated for each educational program• values teaching and learning•be worthy of the students’ investment•result in continuing improvement in institutional quality•provide evidence of improvement based on analysis

assessment results. •be done at various levels (course, program and institutional level)

Achieving the Dream has established a set of performance measures to guide colleges in their work. These include the rates at which students: • successfully complete the courses they take, • advance from developmental to credit-bearing

courses, • enroll in and successfully complete initial

college-level, or “gatekeeper” courses, • continue enrollment from one semester to the

next,• earn degrees and/or certificates

Successful course completion

• All accrediting agencies want institutions to determine what barriers exist for their students, collect and analyze data about student learning to guide program development and service delivery, evaluate programs and services, use the evaluation results to improve programs and services, and finally, observe increases in course completion rates.

• Southern (SACS) and New England (NEASC) explicitly address course completion in their standards.

Advancement from developmental education to credit courses • All accrediting agencies are interested in institutions

developing special services, based on analysis of student outcome data, to help students attain the learning outcomes in their courses, including developmental courses. Since developmental courses prepare students to master essential basic skills required for success in college-level courses, analyzing their effectiveness is an essential part of the institutional effectiveness processes required for accreditation.

• Northwest (NWCCU) is the only regional agency that cites developmental coursework in its accreditation criteria. Western (WASC) and New England (NEASC) mention student ability levels, and programs and services designed to improve deficiencies.

Successful gatekeeper course completion

• All six agencies are very concerned about student success in general education courses, including gatekeeper courses. Their major concern is not simply with the numbers of students who complete the courses, but with the rates of success based on tracking the progression of cohorts of students over time. The latter is a key focus of Achieving the Dream.

• Accrediting agencies also want colleges to measure their students’ learning outcomes to ensure that they are mastering college-level competencies in the gatekeeper and general education courses. Much variation exists among agencies in the detail required to measure student learning outcomes. Southern and North Central give colleges the freedom to define their student learning outcomes. Middle States, New England, Northwestern and Western list specific outcomes the colleges must measure.

• SACS and NEASC criteria directly address course completion.  

Term-to-term persistence

• All six agencies want colleges to take steps to strengthen course content, classroom strategies, student orientation, and counseling, advising, and other student support services to improve learning and encourage student persistence.

• NEASC and Middle States specifically address student persistence.

Completion of degrees and certificates

• All six agencies are interested in whether students attain program outcomes, which can include completion of credentials as well as licensure, employment, and baccalaureate transfer.

• NEASC is the sole agency that explicitly mentions attainment and other measures of successful completion in its standards.

http://betablog.necc.mass.edu/neasc/ NECC Self-Study Blog

http://www.necc.mass.edu/irp/planning/neasc.php “Data First”, “S Series”, and other Forms

http://www.necc.mass.edu/achievingthedream/datareports.php

Achieving the Dream Data Reports

Member of New England Association of Schools and Colleges – Commission on Institutions of Higher Education (NEASC – CIHE)

Self-Study Site Visit in October 2010 Process Began in January 2009

AppreciativeInquiry

ProcessManageme

nt

StrengthsQuest

Increased focus on student success

“Data First” Forms

Student Success “E Series” and “S Series” Forms

Institutional Effectiveness Statements

“Data First” Forms

•Complete First

•Supporting Data for Each Standard

“S Series” Forms

•Retention and Graduation Rates•Other Measures of Student Success•Licensure Passage and Job Placement Rates

•Completion and Placement Rates for Short-Term Vocational Programs

Institutional Effectiveness

•4.51 The institution’s principal evaluation focus is the quality, integrity, and effectiveness of its academic programs. Evaluation endeavors and systematic assessment are demonstrably effective in the improvement of academic offerings and student learning.

“S Series” Forms

•Course Completion•The “Maryland Model” •Supplemental Data Reports

Fall 2007 = 69.3% Fall 2008 = 68.5% Fall 2009 = 72.5%** Highest Total Completion Rate Recorded

“S Series” Forms

•Course Completion

•The “Maryland Model” •Supplemental Data Reports

Fall 2007 – 2008 = 66% Fall 2008 – 2009 = 70% Fall 2009 – 2010 = ? 4-Year Goal = TBD

“S Series” Forms

•Course Completion•The “Maryland Model”

•Supplemental Data Reports

Data Team Reports• Effect of Age on Course Completion Rates

for Developmental and Gatekeeper Courses

• Effect of Taking Developmental Courses Immediately When Prescribed

• Effect of Gender on Course Completion Rates for Developmental and Gatekeeper Courses

• Comparison of Supplemental Instruction v. Traditional

Response to Specific Standard Criterion•4.45 The institution’s approach to

understanding student learning focuses on the course, program, and institutional level. Data and other evidence generated through this approach are considered at the appropriate level of focus, with the results being a demonstrable factor in improving the learning opportunities and results for students.

Response to Specific Standard Criterion• A variety of alternative instructional models and

interventions have been put in place for math courses in response to Achieving the Dream information regarding low completion rates for math students. Examples include supplemental instruction, a modular section format, self-paced/individualized instruction, accelerated options, and short refresher courses. An inter-departmental math retention team is working on professional development in these areas and on the continuum of skills from developmental through college-level math.

Key Performance Indicators (KPIs)•Transition and Developmental Course

Completion Rates•Gatekeeper Course Completion Rates•Retention•Degrees and Certificates Conferred•Diversity of Faculty and Staff

Public Disclosure (Standard Ten)•Program Review and Outcomes Assessment

•Data Team Reports•“Community Conversations”•Only Mass. college to share CCSSE results

A New Vision A New Focus on Student Success

A Search for Data

Only 58 % of the Applicants were given a Placement Test = ACCUPLACER

Students who placed into developmental courses were not required to take those courses – it was only recommended

The lower the ACCUPLACER score, the less likely a student was to take the course recommended

Low numbers of students passed the developmental courses

Nearly 85% of the students who took the

ACCUPLACER test, placed into at least one developmental course.

COURSE Placed Enrolled Successful RDG 050 9% 56% 62% RDG 080 42% 66% 74% ENG 030 8% 52% 60% ENG 070 25% 60% 70% ENG 161 67% 85% 66% MTH 050 61% 25% 70% MTH 052 67% 58% 53% MTH 100 20% 58% 55% MTH 157 13% 26% 75%

2192 new students enrolled, 1289 took ACCUPLACER (58.8%), 120 placed into RDG 050 (9.3%), 67 enrolled in RDG 050 (55.8%), 42 of them earned an A, B or C (63 %)

ATD Year 1 - 2006-2007 was focused on:

Design and implement an integrated developmental education program

Design and implement processes and interventions to enhance achievement of student learning outcomes

Design and implement a “transition to college” intake process with a comprehensive orientation for all first-time students

Collection of data from support strategies already in use

ATD Year 2 - 2007-2008 was focused on:

The increased use of ACCUPLACER test for applicants

The use of ACCUPLACER scores for placement in developmental courses

Identification of Gatekeeper Courses The use of Supplemental Instruction and other strategies in more courses

Pilot use of an Orientation Program for entering students

ATD Year 3 focused on: Developing strategies to implement new policies Common course syllabi, learning outcomes and

common outcomes based finals were implemented for the 4 math courses

Data analysis identified outcomes in each math course that were not being mastered by the students

Data collection and analysis were required of each ATD team, but project design did not always

have a plan for data collection and analysis 3 Learning communities were piloted

Course outcomes were being reviewed for the first time

Development of assessment tools for learning outcomes

Use of data to inform decisions Course redesign and instructional

strategies

Upgraded the Institutional Research capacity• Hired a programmer/analyst• Upgraded a programmer/analyst to full-time

Infrastructure capacity extended• LAN upgrade• Bandwidth increase• PBX upgrade• Help desk added

Software DataTel extensions and additions• Data Orchestrator – end user direct query of data• Business Objects/Crystal Reports - data dashboards• iStrategy – will freeze data at different points in

time, comparative data analysis and longitudinal studies of cohort groups

• Retention Alert – case files for at-risk students and enables strategies such as mid-term reports

• Provides the ability to collect, retrieve and manage raw data as well as display it through the portal site

Provide evidence of Mission Achievement

Data collection to provide evidence for all Middle States Standards

Continuous Quality Improvement

A new IE Plan was adopted in Fall 2009.

The data systems are in place and the support staff is working well as a team

Additional developmental course sections were scheduled (thinking ahead of the budget required)

Systematic processes of planning, budgeting, implementation, and assessment

Strategies of ATD have inspired the use of systematic processes across many other campus initiatives

These systematic processes have been integrated into the IE Plan that was submitted to Middle States

ALL incoming students have to take ACCUPLACER

ALL first-time, full-time, students who place into developmental coursework have to enroll in developmental course work

ALL students who require two or more developmental courses are required to enroll in a Personal Development Student Success Course (PDV 160) for one credit

No registration after first week

1. To increase by 2% the percentage of first-time degree-seeking students who complete ACCUPLACER testing Now = 84% (2005 = 59%)

2. To increase by 2% the percentage of first-time degree-seeking students who immediately take the developmental course work as recommended Reading = 85% (2005 = 61%) English = 83% (2005 = 56%) Math = 76% (2005 = 40%)

3. To increase retention by 2.0%Now = 3.6% increase overall over 2008-2009

4. To increase student success in the fourteen developmental and gatekeeper courses using common course outcomes and a common assessment tool Now = 0 – 20% depending on the courseDevelopmental = 4% increase on average

5. To have all 2D and 3D entering students take a PDV course (683 completed in Fall 2009)

6. To develop more cohort groups of developmental students (160 students in 8 different groups)7. Orientation program available for all entering

students8. SSSA’s = team that provides case management

advising for all first semester students

Elimination of most 1 day/week for 3 hours sections

Common Student Outcomes Common Course Syllabus Common multiple choice final exam

based on outcomes Comparison of grade on final exam and

final course grade (good correlation) Identification of poorly learned outcomes Working to develop teaching strategies to

help students to learn these outcomes

Course Enrollment SuccessFall ’05 Fall ’09 Fall ’05 Fall ’09

RDG 050 67 258 42 (62%) 184 (71%)

RDG 080 354 578 261 (74%) 443 (77%)

ENG 030 50 56 30 (60%) 38 (68%)

ENG 070 187 518 132 (68%) 341 (66%)

MTH 050 177 597 124 (64%) 383 (64%)

MTH 052 451 496 238 (53%) 299 (60%)

Nearly 200 more developmental sections were taught by adjunct faculty

Students in the eight developmental learning communities were not as successful as hoped

Case management advising system very effective for “intake” but ineffective for registration after first semester

Two new “blended” math courses developed

65 of the college’s 95 faculty involved in ATD

Faculty not involved in ATD have developed their own student success strategies

Two local high schools using ACCUPLACER tests and a summer bridge program was piloted last summer.

A Title III Grant was obtained in October 2008

New curriculum development (funding was pursued because of the ATD success)(AACC/NSF – MentorLinks grant)

A mentor program for training the adjunct faculty in better teaching methods(College of the Canyons – NSF)

Simulators are being used to help students in allied health courses to be more successful

More student clubs, activities, and sports teams

Faculty are interested in creating an honors program, a leadership program and an undergraduate research program to provide tools for the success of non-developmental students as well as for the newly successful developmental students

Ultimately, the ATD grant will lead to the development and implementation of success strategies for all students at WCCC

1. Use the Achieving the Dream improvement process as the basis for your self-study and quality improvementefforts.

2. Include on the self-study team people who are familiar with or serve on the Achieving the Dream core team, data team, or strategy implementation team.

3. Obtain broad institutional involvement in strategy development and implementation.

4. Ensure that Achieving the Dream strategies will help the college further its mission and strategic goals.

5. Establish outcomes for each Achieving the Dream strategy, assess whether the outcomes are met, and use the assessment data to document improvement

6. Don’t confine your assessment measures to grades and retention.

7. Use and document formative evaluation (indicators that occur along

the way) to inform programmatic changes.

8. Cite the evaluation reports for Achieving the Dream strategies as sources in your accreditation report.

9. Use one of the Achieving the Dream strategies for the QEP topic if it has

the potential for improving student learning in a significant way

10. Use one of the Achieving the Dream strategies as an Action Project for

AQIP.11. Use the Achieving the Dream “culture

of evidence” continuous improvement process as the basis for the college’s institutional effectiveness function.

12. Use Achieving the Dream as an “umbrella” for student success

efforts across the college.13. Integrate Achieving the Dream

into the college’s strategic planning processes.

http://www.cpcc.edu/planning Click on “studies and reports” Title: AtD Accreditation Session Contact information

•Terri Manning•terri.manning@cpcc.edu•Lane Glenn• lglenn@necc.mass.edu•Carol Rush•rushca@wccc.edu