Finally. . . Informed Decision Making
Presented by
Stephanie Houston & Laura EvansColton-Redlands-Yucaipa ROP
NACTEI Conference May 14, 2004
Introduction
• What are Regional Occupational Programs/Centers?
• Where is CRY-ROP and who does it serve?
• What threats is CRY-ROP facing and how does it respond?
• What is CRY-ROP’s vision of using data to drive decision making?
Data and Student Academic Performance
• High School Performance
–No Child Left Behind
–API Scores
–SAT 9 Scores
–HSEE passing rates
What are the implications of the NCLB legislation, API, HSEE…etc?
• Emphasis on academic standards and course content
• Emphasis on data analysis
• Expanded attention to teaching strategies that reach all students
• Potential impact on student electives
• Expanded attention to assessment strategies
• Development of intervention and remedial opportunities for under performing students and schools
Prepare. . .
• Look at the NCLB legislation, HSEE & API as opportunities to show how ROP or other career & technical classes reinforce and support student achievement.
• Anticipate change--and prepare for it!
Focus on the positive…
• 2/3rds of the Stanford 9 reading comprehension questions are textual and functional reading items which are predominant in ROP courses
• ROP or other career & technical instructors can show how they support and strengthen academic skills through occupational applications
Connect…
• ROP or other career & technical courses can assist students who do not do well in traditional courses by teaching academic skills in context
• ROP or other career & technical courses may provide alternative graduation requirements in some districts
• ROP or other career & technical should be part of the solution in improving student performance
No Child Left Behind• Accountability utilizing scientifically based
research– Use of rigorous systematic and empirical methods– Adequacy of data to justify the general conclusions– Reliance on methods that provide valid data across
multiple measurements and observations– Use of control groups– Details for replication– Acceptance by a peer-reviewed journal or approved by
a panel of independent experts
Data Collection Methods• Registration forms• Attendance/Completion forms• Certificate request forms• Perkins core indicators• Student surveys
– Exit– Satisfaction– Follow-up– Student Transcript Data Collection Form
• Shared data files/databases
Tools of the trade
• Scantron
• Customizable databases
• Crystal Reports software
• SPSS software
• Partnerships with other schools
• Research/studies
Partnerships for Advancing Student Success (PASS) P-16 Council
Vision- To create a system of education that all students deserve, that our workforce expects, and that a democracy requires
P-16 Council
• Pre-school through post-secondary education• “Systems” thinking to encompass the
interdependent and interacting components of the education system
• System-wide coherence providing a cohesive education from childhood to adulthood
• Legal Memorandums of Understanding in place to ensure appropriate use of data
P-16 Council
• Data elements are shared and analyzed to strengthen program quality and guide decision making
• CRY-ROP is represented on the San Bernardino Consortium Data Oversight Committee
• CRY-ROP is represented on the CAL-PASS Steering Committee/Users Group
UCR CAROCP Study
• Completed by the University of California, Riverside - School Improvement Research Group (SIRG)
• Multi-dimensional research project to study the impact of ROCP courses on 11th and 12th grade students
UCR CAROCP Study Goals
• Documentation of the overall effectiveness of California’s Regional Occupational Centers and Programs by – Comparing ROCP students to a
demographically comparable “control” group– Examining 7 outcome domains
UCR Study Outcome Domains
• High school GPAs
• SAT-9 test performance
• Graduation rates
• School attendance rates
• Job placement
• Job opportunities
• Admissions to post-secondary education
UCR Study Design
• An electronic database to integrate information• Three data sources
– High school transcript records
– Telephone follow-up interviews
– High school satisfaction survey
• 11th & 12th graders from – ROC/Ps
– A control group of students NOT on a UC admission track, not in Special Day Classes
Inland Desert Tech Prep Consortium
• Partnership among 43 high schools and community colleges
• Perkins funded• Data collection automated• 23,399 high school students supported in 2003• 1,171 vocational students transitioned to 3
colleges and took vocational courses in 2003
HORIZONS Study
• Commissioned by San Bernardino County Superintendent of Schools School to Career Horizons Grant
• Conducted on P-16 data files
• Limited data available, study needs to be expanded
• Generalizations only – sample too small
HORIZONS StudyAverage 2000-2002 SAT9 Score by Category
400
450
500
550
600
650
700
750
CRA
URA
UCR
A RA UA
UCR
CR
R
UCA UR
U
UC CA
C
NONE
A
Category
Scor
e
HORIZONS StudyChanges in Average SAT9 Scores - 2000-2002
Must be in Category for All 3 Years
193
32
57
-16
0016
-12
17
-120
-70
-20
30
80
Reading Math Language Science Social Studies
Subject
Ch
ang
e in
Ave
rag
e S
core
CTE
N-CTE
HORIZONS Study2000-2002 High School Student Records Matched to 1998-2002/3
Community College Records
14.2%
19.3%
9.5%8.8%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
CTE&ROP CTE Only ROP Only No CTE or ROP
HORIZONS Study2000-2002 Category Matches To College Files
20.8%
14.9%
5.9% 5.6%4.6% 4.5%
0.0%
8.6%8.7%8.9%
13.6%
18.2%
20.2%18.8%
23.6%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
UR
A
UC
RA
UR
UA
RA
UC
A
UC
R U R
UC
CR
C
Non
e A
CA
HORIZONS Study
Award Type Number of Awards Category NameAssociate of Arts (A.A.) degree 1 UCRACertificate requiring 6 to fewer than 18 units 13 UC Certificate requiring 6 to fewer than 18 units 9 UCertificate requiring 6 to fewer than 18 units 9 UCR Certificate requiring 6 to fewer than 18 units 5 UCRACertificate requiring 6 to fewer than 18 units 2 UACertificate requiring 6 to fewer than 18 units 1 CCertificate requiring 18 to fewer than 30 semester units 2 UCRCertificate requiring 18 to fewer than 30 semester units 1 UCAssociate of Science (A.S.) degree 5 UCRAAssociate of Science (A.S.) degree 1 UA
UCR CAROCP Study
• Common elements that distinguish ROC/P courses involved in the study from traditional 9-12 educational programs:– Yearly labor market analysis– Business advisory committee– Inclusion of academic standards necessary for
success in the career area taught– Student involvement in real-life practice of the
career skills
UCR CAROCP Study• 2002
– 7 ROC/Ps– 5 Career Programs: Auto, Business, Child
Development, Health & Marketing– 1545 ROC/P Students
• 2004– 22 ROC/Ps– 8 Career Programs: Auto, Business, Child
Development, Environment Tech, Health, Public Safety, Marketing & Technology
– 2282 ROC/P Students
UCR CAROCP Study Instrumentation
• Four original data collection and management instruments were developed– Student Transcript Data Collection Form– Telephone Follow-up Survey– High School Student Satisfaction Survey– Integrative relational database capable of
merging data form all sources, built in Microsoft Access
UCR CAROCP Study Major Findings
• Raised/comparable high school GPAs
• Improved class attendance
• Better wages
• More likely to be promoted on the job
• Good rates of further education
• Preference for ROC/P over high school
• Low appreciation of value of education
• Resistance to academic classes
UCR CAROCP Study
• Better Wages– 2002: Average salary of $7-$10 per hour, with
ROC/P students earning $1.44 more per hour
– 2004: Average salary of $7-$13 per hour, with ROC/P students earning $2.40 more per hour
UCR CAROCP Study• Raised high school GPAs
– 2002: ROC/P students entered training programs with academic records below Control Group. By the completion of 12th grade, ROC/P students raised their GPAs by more than three-tenths of a point – a better rate than the Control Group
– 2004: ROC/P students entered training programs with academic records below Control Group. By the completion of 12th grade, ROC/P students raised their GPAs by about three-tenths of a point – same pace as the Control Group
UCR CAROCP Study
Class Attendance
8.28
3.58
11.62
9.19
4.92
8.54
4.21
9.04
0
2
4
6
8
10
12
14
10th Grade 12th Grade 10th Grade 12th Grade
2002 2004
Day
s A
bse
nt/
Yea
r
ROCP Group
Control Group
UCR CAROCP Study
• On the Job Promotions– 2002: ROC/P students 28%, Control Group 21%– 2004: ROC/P students 25%, Control Group 23%
• Raises– 2004: ROC/P students 25%, Control Group 19%
UCR CAROCP Study
• Further Education-Comparable– 2002: ROC/P students 54% transition rate;
Control Group 53% transition rate– 2004: ROC/P students 51% transition rate;
Control Group 52% transition rate
UCR CAROCP Study
• Value Statements– ROC/P students “Very much” like career education
course work
– “Only a few” high school classes are relevant for ROC/P students in terms of getting jobs or preparing them for adult life
– ROC/P students report low levels of appreciation for Math, Science and Social Studies
– ROC/P students report matching enthusiasm for English Language Arts, Music, Art , Drama and ROC/P course work
Conclusions
• California’s ROC/Ps have a positive impact on students
• Positive impacts are evidenced in academic achievement, the labor market and continuing education
• Full implementation of an accountability system for career and technical education can document the social and economic payoff of programs nationwide
Sources & Links
CAROCP– http://www.carocp.org/
CAL-PASS – www.cal-pass.org
P-16– www.edtrust.org– www.csulb.edu/~acaproj/partnerships.html
NCLB– www.nochildleftbehind.gov
Questions?