Race to the Top Assessment Race to the Top Assessment Program: Public HearingsProgram: Public Hearings

Some Thoughts on Grades 3-8 and High Some Thoughts on Grades 3-8 and High

School AssessmentsSchool Assessments

December 1, 2009December 1, 2009Denver, ColoradoDenver, Colorado

Presenter:Presenter:Lloyd Bond, The Carnegie FoundationLloyd Bond, The Carnegie Foundation

Issues about which there is Issues about which there is consensus consensus

Intended usesIntended uses

Content, Formats, & Test AdministrationContent, Formats, & Test Administration

ValidationValidation

ReportingReporting

Consensus IssuesConsensus IssuesIntended usesIntended uses

Monitoring student progressMonitoring student progress Teacher, principal, and school accountabilityTeacher, principal, and school accountability College and work readiness (?)College and work readiness (?)

Content, Formats, & Test AdministrationContent, Formats, & Test Administration Multiple measures of student successMultiple measures of student success Paper & pencil, computer assisted, MC, short answer, extended Paper & pencil, computer assisted, MC, short answer, extended

response, and portfolioresponse, and portfolio Both mid-course and end-of-course assessmentsBoth mid-course and end-of-course assessments Multiple opportunities and multiple ways to demonstrate Multiple opportunities and multiple ways to demonstrate

masterymastery (See “My Child Doesn’t Test Well” (See “My Child Doesn’t Test Well” Carnegie Perspectives Web Site)Carnegie Perspectives Web Site)

Instructionally useful information to teachers and principalsInstructionally useful information to teachers and principals

Consensus Issues (cont’d)Consensus Issues (cont’d)

ValidationValidation Appropriate evidence claims of content fidelity and Appropriate evidence claims of content fidelity and

college/work readiness (?)college/work readiness (?)

ReportingReporting Reporting of assessment information (including Reporting of assessment information (including

assessment architecture, standards, and results) to assessment architecture, standards, and results) to the various non-technical stakeholders should be as the various non-technical stakeholders should be as straightforward as possible, understandable to literate straightforward as possible, understandable to literate adultsadults

Thoughts on Unresolved Thoughts on Unresolved and/or Contentious Issuesand/or Contentious Issues

Broad & Shallow vs. Narrow & DeepBroad & Shallow vs. Narrow & DeepConsequential Validity - “Teaching to the Consequential Validity - “Teaching to the

Test”Test”The concepts of college and work The concepts of college and work

“readiness”“readiness”Familiar vs.aspirational assessmentFamiliar vs.aspirational assessment

Broad & Shallow vs... Narrow & Broad & Shallow vs... Narrow & DeepDeep

The delicate balance between (1) learning The delicate balance between (1) learning a lot of things marginally and (2) learning a a lot of things marginally and (2) learning a modest number of important things wellmodest number of important things well

One of the distinguishing curricular One of the distinguishing curricular features of countries that consistently do features of countries that consistently do well on TIMMS and other assessments is well on TIMMS and other assessments is that they error on the side of “(relatively) that they error on the side of “(relatively) narrow and deep”narrow and deep”

Thoughts on the “what” of Thoughts on the “what” of assessmentassessment

The Grade 3-8 through High School assessments should move increasingly and continuously from:

1) assessment of declarative knowledge to assessment of procedural and conceptual knowledge

2) assessment of procedures and skills to assessment of problem solving and critical thinking

Thoughts on ValidityThoughts on ValidityGathering evidence of content & predictive Gathering evidence of content & predictive

validity should be reasonably validity should be reasonably straightforwardstraightforward

Gathering evidence regarding Gathering evidence regarding consequential validity can be problematicconsequential validity can be problematic

Consequential Validity of the Consequential Validity of the AssessmentsAssessments

Testing what is taught vs. teaching what is Testing what is taught vs. teaching what is tested tested

Using the RTT assessments for Using the RTT assessments for accountability purposes guarantees that accountability purposes guarantees that attempts to “teach to the test” will occurattempts to “teach to the test” will occur

We must therefore make the assessments We must therefore make the assessments worth teaching toworth teaching to

(See “Teaching to the Test” (See “Teaching to the Test” Carnegie Carnegie Perspectives Perspectives Web Site)Web Site)

College & Work ReadinessCollege & Work Readiness College and work “readiness” are as yet ill-College and work “readiness” are as yet ill-

defined and illusive conceptsdefined and illusive concepts ““Ready” for majoring in physical education at Ready” for majoring in physical education at

Podunk U or ready for majoring in electrical Podunk U or ready for majoring in electrical engineering at Cat Techengineering at Cat Tech

Ready for a cashier at McDonald’s or ready for Ready for a cashier at McDonald’s or ready for office manager trainee at Merrill Lynchoffice manager trainee at Merrill Lynch

It is doubtful whether a single test can serve this It is doubtful whether a single test can serve this multiple purposesmultiple purposes

Measuring important skills and Measuring important skills and abilitiesabilities

““We do not yet know how to measure all of We do not yet know how to measure all of the important things. Ergo, what we CAN the important things. Ergo, what we CAN measure becomes all important”measure becomes all important”

Aspirational AssessmentsAspirational Assessments In addition to the usual and familiar, the In addition to the usual and familiar, the

RTT assessment program should RTT assessment program should encourage and support consortia that also encourage and support consortia that also move the profession forwardmove the profession forward

Toward the Assessment of Toward the Assessment of “Understanding”“Understanding”

Students can “know” things and know how to do Students can “know” things and know how to do things without genuine understandingthings without genuine understanding

““Study to remember, and you will forget; study to Study to remember, and you will forget; study to understand, and you will remember.” understand, and you will remember.” ~Anonymous~Anonymous

Teach for regurgitation, and students will forget; Teach for regurgitation, and students will forget; teach for understanding, and they will teach for understanding, and they will remember.remember.

““The Assessment of Understanding” The Assessment of Understanding” Carnegie Carnegie Perspectives” Perspectives” Web SiteWeb Site

Final ThoughtsFinal Thoughts

Beware of the assessment tail wagging the Beware of the assessment tail wagging the content dog content dog

Don’t reinvent the wheel. Encourage consortia Don’t reinvent the wheel. Encourage consortia to take advantage of what has already been to take advantage of what has already been done and what we already know (e.g., College done and what we already know (e.g., College Board Accuplacer) Board Accuplacer)

Encourage involvement of higher education Encourage involvement of higher education faculty and industry personnel in the high school faculty and industry personnel in the high school assessmentassessment