State Assessment Systems: Issues and Options

David T. Conley, Ph.D.

Professor

University of Oregon

Director

Center for Educational Policy Research

2

Purposes of State Assessments A state assessment can have multiple purposes

School accountability Student accountability (exit requirement) System monitoring and policy decisions Performance feedback to improve teaching College placement or admission

Most state assessments end up addressing multiple purposes

Some end up with confused purposes or no one clear purpose

3

The Power Equation of State Assessments State assessments represent an exercise by the

state of its power over local school systems State constitutions grant states control over subsidiary

governmental units, such as school districts However, long traditions of local control come into conflict

with this exercise of state authority in the educational policy arena

State assessments ultimately become a compromise between achieving state goals and incurring the political costs of exercising state power to intervene into the functioning of local schools

4

What Is Feasible? State assessments should meet the following

criteria: Be manageable by school districts Be consistent with state goals Be taken seriously by educators and students Lead educators and students toward desired

behaviors Be reasonable in cost, technical requirements and

time Meet technical adequacy standards for assessments

5

The Reliability-Validity Tradeoff Generally, assessment is a tradeoff between

reliability and validity The more standardized the assessment, the better

the reliability and the more limited the validity The more directly a reflection of actual classroom

work, the better the validity and the more challenging the reliability

Americans tend to value reliability above all else Other nations favor validity

6

Where Are We in Oregon? Oregon assessment, as originally

designed, is admirable in many respects Contained mix of methods

• Multiple choice, performance tasks, work samples Has long track record Has online component (TESA) Appears to be reasonably well accepted by

schools Meets NCLB requirements

7

Where Are We in Oregon? However, significant differences exist from

elementary to secondary levels Lower scores, less concern with the assessments Some schools take them seriously, others appear

less concerned with flat scores While scores can conceivably be used by

higher ed (via PASS), they do not appear to be used in this fashion by many students

Legislature seems to need a primer on the system each session

8

Considering Options

Note that there are tradeoffs with each option- there is no one best option

Development and implementation costs are high for any new system or substantial modification

Oregon educational system does not seem particularly ripe for innovation or significant change at the moment

9

Defend Current System This is sensible to do for many reasons

The current system aligns well (best?) with state standards

It is institutionalized and runs smoothly More educators appear to be more capable of using

test data for instructional decisions However, it does leave the state exactly where

it is currently, with an assessment that is largely in search of a purpose (beyond meeting NCLB requirements)

10

Revise Current System Augment the test to include items that enable

the test to be used for college readiness feedback and perhaps limited college placement purposes

Attach stakes to current tests so students and schools take them more seriously E.g., compensatory scale composed of GPA and test

scores, with minimum score required to avoid mandatory remediation or some other mild sanction

11

Adopt a National Test Tied to College Admission or Placement

ACT system (Explore, PLAN, ACT) or PSAT/SAT

Will be taken more seriously by students, teachers Allows national comparisons of Oregon students Is relatively efficient to administer Does not necessarily cover state standards as well

as state assessment (this needs to be documented, however)

May lead to more students considering college, based on IL and CO experiences

12

Adopt National Tests

Advanced Placement incentives or subsidy

IB subsidy Work with postsecondary institutions to

use data from these tests to recruit more students, award scholarships and credit, and place students

13

Replace or Augment State Test with End-of-Course Exams Key benchmark courses such as Algebra II,

Biology, 11th grade English Creates a better balance between reliability and

validity Development costs, implementation complexity can

be high Can align highly with state standards Can provide diagnostic feedback to students,

teachers Eventually requires specifying curriculum for tested

courses, something that will be difficult for and alien to the state

14

End-of-Course Exams Can address content knowledge and

essential skills Help limit curriculum inflation and drift Can be made a component of the course

grade Can be combined in a compensatory

fashion with other measures Some efforts are underway nationally to

develop consistent end-of-course exams

15

Develop School-based Assessment System Can be collection of evidence or

extended application-type project Data collected over time or on a one-time

basis, or both Best system for providing diagnostic

feedback to students Can be well aligned with state standards Requires local buy-in and a specific

mechanism to guarantee rater consistency across schools

16

School-based Assessments Senior year review

Student must demonstrate competency in key metacognitive skills such as logical and analytical thinking, problem solving, interpretation, teamwork

This can be done through a variety of means to accommodate college-bound and non college-bound students

No student is denied a diploma, but the information is used to judge schools and provide students with a better assessment of their current ability levels in these key capabilities for future success in college and career

17

School-based Assessments

Collection of evidence Builds upon current work sample requirement Links with PASS Can be incorporated into a senior review Can be used to provide ongoing gauge of

college readiness for students and parents Postsecondary ed can align entry-level

courses with collection performance levels

18

Overall Criteria to Consider Is system’s goal to gauge basic skills or

highest achievement? Is system to inform educators or policy

makers? Will system have any effects,

consequences, or incentives for test-takers and for educators?

Does system achieve or promote key state goals for education?

19

Thinking about a System What are the state’s high school exit standards?

What is it students should know and be able to do upon graduation?

• How close are these to college readiness standards?

What does performance look like at 8th and 10 grade for students who will be ready to meet 12th grade exit standards? How can 8th grade assessments also be “exit”

measures What assessment system generates this

information?

20

Oregon Challenges Balance of state influence-local control

What purposes should local control serve? Lack of articulation mechanisms

Joint boards notwithstanding, state has few ways in which the three sectors work in a coordinated fashion

Funding As always, do more with nothing

No 12th grade exit standards CAM had career-related standards, but no content standards

Weak connections between assessment system and classroom practices or (state) accountability system