ACT WorkKeys Training for Illinois (NCLB) Assessments:

The Chicago Model

Loretta Begley

Chicago Public Schools

Office of High School Programs

lmbegley@cps.k12.il.us

PSAE / ACT HISTORY• NCLB and AYP

• Implications for the Employment Community

College

ACT WorkKeys

Post Secondary Study

EconomicDevelopment

CareerReadiness

NCLB

ACT Exam

Workforce

How States Are Administering the ACT

  Illinois Colorado Michigan Tennessee Wyoming

Prairie State Achievement Exam

Colorado ACT

Michigan Merit Exam

Tennessee

Voucher Program

Wyoming Voucher Program (Hathaway)

2001Renewed in 2005 for 5–10 years

2001Renewed in 2005 for 5 yrs

Piloted in spring 2006Full implementation in spring 2007

1995 2006–07 school year

The ACT (English, Math, Reading, Science, Writing); WorkKeys: Applied Math and Reading; Illinois State Board of Education-developed science test

The ACT (English, Math, Reading, Science)

The ACT (English, Math, Reading, Science, Writing); WorkKeys: Applied Math and Reading; Michigan-developed math, social studies, and science tests

The ACT (English, Math, Reading, Science)

The ACT (English, Math, Reading, Science); WorkKeys: Applied Math, Reading for Information, Locating Information

Measure achievement of grade 11 students relative to the Illinois Learning Standards for reading, mathematics, and science. Participation in PSAE is a condition of receiving a regular high school diploma. Results are used to satisfy NCLB/AYP requirements.

ACT results are used as part of the state’s accountability plan.

Designed to measure student achievement relative to Michigan standards. Results used for Michigan Merit financial awards and to satisfy NCLB/AYP requirements.

Requirement for state scholarship program eligibility.

Determine eligibility for Hathaway Scholarships.

WorkKeys Recap

Illinois PSAE Exam = 20% Total• 50% Reading for Information• 50% Applied Mathematics

An ACT study found an 80% correlation between the ACT test and WorkKeys assessments

• indicating that an intervention causing an increase in WorkKeys scores would cause a subsequent increase in the ACT score

• Increasing a WorkKeys Reading score From Level 4 to Level 5 would cause:

– An increase of 4-8 composite score points due to the WorkKeys portion on the PSAE .

– An improved ACT score resulting in possible increase of 5-8 points on the PSAE.

– A total PSAE Reading score increase of 9-16 points.

Must Achieve Level 5 to have a chance of achieving AYP

Resources For WorkKeys Training

ACT Approved Products:

• Targets for Instruction

• Practice Test …One released Version

• KeyTrain and WIN software

Texts and Test-Prep Services

• McGraw-Hill

• Cambridge, Kaplan, Princeton Review

What is Key-Train?Key Train is a comprehensive training curriculum that is

designed specifically for the Work Keys assessments. Each module includes pre and post test; time-in-lessons; date stamp, and a variety of reporting features.

Modules include:

• Reading for Information (included on PSAE)

• Applied Mathematics (included on PSAE)

• Locating Information

• Writing

• Applied Technology

• Teamwork

• Listening & Observation

• Basic Skills (Level 2)

• Career Skills

Utilizing Key-Train

How much time does it take for a student to move up one level?

Estimated time of completion is 6-12 hours of work in one subject area to complete one level.

Time alone is not an indicator of success.

Teacher Identification of Learning Gaps (reports) accompanied by good instructional practices is critical.

Instructional Planning• Don’t “teach the test”; teach the skills needed for success on

the test. (Use EPAS strand analysis / WorkKeys Practice Exam / KeyTrain software to identify those skills)

• Begin at the End. What do you want the final “product” to look like. (Skill Level 5)

• Use Data Analysis resources to identify instructional strengths and weaknesses.

• Learning Gaps-include skill and level to address. (How do you determine correct skill level? How do you assure rigor? And how do you identify progress toward the next level?

• How and where will WorkKeys skills be incorporated within the curriculum?

Teachers and School Professionals

Concerns

Students

Instructional Practice• Appropriate use of Instructional Software.

• Make sure students are using calculator and formula sheet (AM).

• Make it “count” – incorporate in instructional practice.

• Monitor student usage and reports. Look for serious effort.

• Provide extended instruction and reinforcement.

• Set goals and provide incentives for successful movement to the next level.

• Elicit parental support by providing individual reports.

N-CREL Recommendatio

ns

For Technology Integration

Curriculum-Specific Applications

• If technology is to be used to produce improvements in student achievement, teachers must see a direct link between the technology and the curriculum for which they are responsible (Byrom, 1998).

• Professional development for technology use should demonstrate projects in specific curriculum areas and help teachers integrate technology into the content. In particular, professional development activities should enhance teachers' curriculum, learning, and assessment competencies and skills as well as classroom and instructional management competencies and skills.

• Specific content can help teachers analyze, synthesize, and structure ideas into projects that they can use in their classrooms (Center for Applied Special Technology, 1996).

Infuse it into the curriculum• A good professional development program is

job embedded and tied to learning goals: It provides activities in the context of practice.

• The best integration training for teachers does not simply show them how to add technology to their what they are doing. "It helps them learn how to select digital content based on the needs and learning styles of their students, and infuse it into the curriculum rather than making it an end in itself," notes Fatemi (1999).

• "Using technology effectively also requires having a wide repertoire of teaching approaches."

New Roles for Teachers• Technology encourages teachers to take on new and expanded

roles, both inside and outside of the classroom. Within the classroom, technology supports student-centered instruction.

• The teacher assumes the role of coach or facilitator while students work collaboratively (Jones, Valdez, Nowakowski, & Rasmussen, 1995; Kupperstein, Gentile, & Zwier, 1999). Outside of the classroom, technology supports teacher collaboration. Instead of working in isolation, teachers can work together on school wide programs. They can help find solutions to problems, act as peer advisors to provide information and feedback, and collect data to test hypotheses (Lieberman, 1996; Little, 1982). Their new roles may involve distance collaboration with cross-school peer groups and study groups through telecommunications (Kosakowski, 1998).

• Professional development for technology use provides opportunities for teachers to become comfortable and effective in these new roles.

Data and Analysis

• REA Data and Analysis http://research.cps.k12.il.us

Reading, Math and Science

Plan, Explore, Plan, ACT

Strand Analysis

WorkKeys Practice Exam, Item Analysis

• KeyTrain Reports Usage and level Reports by grade Export total school use in Excel PDF – View and Print Repots,

• Other Services / Strategies / Formative Assessments

Programs, Curriculum and Test Prep

• Small Schools, Career Academics, ETC, Avid, Transformation Schools, IB, Excelerator, Gear Up

• Test Prep – Facts and Myths

EPAS•Educational•Planning and•Assessment•System

EXPLORE, PLAN, and ACT assessments in Reading, Math, English, and Science Reasoning

All are published by ACT, Inc.

EPAS assessments are all linked to:

•Illinois Learning Standards

•ACT College Readiness Standards (formerly Standards for Transition)

•Pathways for Transition

Reading for Information Applied Mathematics Locating Information

(not part of PSAE)

33 questions 33 questions 38 questions

Skill level 3-4 5 6-7 3-4 5 6-7 3-4 5-6

Charac- teristics

Company policies, procedures & notices

Corporate acronyms, jargon & tech terms

Legal regs., complex & difficult concepts

Word problems & simple charts

Logic, calculations & multiple operations

Multi-step calculation, incomplete information

Workplace graphics & multiple graphics

Complex charts, maps & diagrams

ACT WorkKeys Reading Skills

• Identifying the main idea or details, both obvious and implied

• Understanding word and acronym meanings using context, ranging from explicit to implied definitions

• Applying instructions by sequencing steps, knowing when to use conditions, and cause and effect.

• Applying information to described and new situations

• Analyzing and synthesizing information to identify rationale and principles and apply them to new situations

• Generalizations and Conclusions

• Main Ideas and Authors Approach

• Meanings of Words

• Sequential, Comparative and Cause-Effect Relationships

• Supporting Details

5 Reading Strands

ACT Strand Analysis ReadingMeanings of Words Weakest Meanings of Words Weakest

StrandStrand9th Grade Explore Reading Percent Items Correct Over Time

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

80%20

03

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

(GEN) (MID) (MOW) (REL) (SUP)

Per

cent

Item

s C

orre

ct

ReadingExplore/Plan TrendsMeaning of WordsCurrent Juniors Reading EPAS Performance

from 9th Grade to 11th Grade

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th

Generalizations andConclusions

Main Ideas andAuthor's Approach

Meanings of Words Sequential,Comparative, and

Cause-EffectRelationships

Supporting Details

Per

cen

t Ite

ms

Co

rrec

t

• Basic Operations & Applications

• Graphical Representations

• Measurement

• Numbers: Concepts & Properties

• Properties of Plane Figures

• Probability, Statistics, & Data Analysis

• Expressions, Equations, & Inequalities

7 Mathematics Strands

ACT WorkKeys Math Skills • Add, subtract, multiply, and divide positive and

negative numbers

• Calculate and convert to percentages

• Convert from one unit of measurement to another

• Understand and work with fractions.

• Understand and work with ratios and proportions

• Calculate time intervals and changes as well as production rates

• Use formulas, including those involving perimeter, area, and volume

• Solve assorted word problems

9th Grade Explore Mathematics Percent Items Correct Over Time

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

80%

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

2003

2004

2005

2006

2007

(BOA) (GRE) (MEA) (NCP) (PPF) (PSD) (XEI)

Per

cen

t It

ems

Co

rrec

t

Three Areas of Weakness Stand Out

Current Juniors Mathematics EPAS Performancefrom 9th Grade to 11th Grade

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

80%

9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th 9th 10th 11th

Basic Operations andApplications

GraphicalRepresentations

Measurement Numbers - Conceptsand Properties

Properties of PlaneFigures

Probability, Statistics,and Data Analysis

Expressions,Equations, and

Inequalities

Pe

rce

nt

Ite

ms

Co

rre

ct

Mathematics Explore/Plan TrendsInconsistent Progress

• Look at the data to drive instructional decisions. • Check administrative reports to see usage

records. • Look for appropriate time on task—pretest and

lessons.• Look for MOVEMENT to the next level --increased

rigor • Start with incoming Freshman – junior year is too

late. • Consider Team teaching or instructional support

for specific content.• Identify needs for professional development.

Monitor Progress

KeyTrain Planning and Procedures

New Logons•Logons and Passwords

Students Username: ID#Students Password: ID#****New********

Note: The new School Unit ID numbers are 6 digitsTeachers Username: School ID#-00000Employee ID#Teachers Password: School ID#-Employee ID#

Lessons have been assigned for 3 topics, plus Beginning Skills.

District Default Settings

Junior’s Implementation Plan

• What does your school data show? The last week in September, your sophmore’s

and junior’s took the WorkKeys paper practice tests. Use the item analysis to identify deficiencies.

Course Analysis w/objectives report Topic and Quiz Detail report

• What teachers, classes and students will be using KeyTrain? Think about what your teachers are doing now.

• Think about how many labs you have available.

• What programs are being used at your school? (ACT Prep, Kaplan, Cambridge, Princeton Review, etc.) Can you identify deficiencies from other resources?

• How can your teachers rotate labs, to make the most of KeyTrain exposure. Incorporate KeyTrain (“WorkKeys” skills) as a component of classroom instruction. How are your teachers going to accomplish this?

• We need 6-12 hours in each area to go up one level. We need Juniors to be at Level 5, by January.

• You should be seeing movement to higher levels over time? (Course Analysis Report)

Implementation Plan

Class Planning

• Which grade level of students are going to use KeyTrain? (Freshmen, Sophomore’s, Juniors)

• Which classes will use KeyTrain? (ETC courses, Test Prep, English, Math, Science, Social Sciences). Which content area will be addressed for each

class? (English: Reading for Information / Math: Applied Math / Social Sciences or Science: Locating Information)

Have you considered innovated designs? Team teaching?

• Which classes will need to be rotated to a lab?

Lab Rotation