special populations, comprehensive next generation ... · comprehensive next generation assessment...

Special Populations, Comprehensive Next Generation

Assessment Systems, and Learning Models: Transition to Assessment

for Learning

Friday, June 21, 2013: 8:00 AM-9:30 AM

Chesapeake G-H (Gaylord National Resort and Convention Center)

90 Min Session Agenda Timeframe Topic Presenters

15 minutes Introduction GSEGs and AA MAS Context WestEd GSEG Project

Dissemination Learning Models Colloquium

Susan Weigert Edynn Sato Pat Almond

15 minutes Low Performing SWD—Lessons Learned

Sue Bechard

15 minutes Learning Models/Learning Progressions

Margaret Heritage

15 minutes Smarter Balanced & Low Performing SWD

Vince Dean

30 minutes Discussion Questions Facilitation

Sandra Warren Edynn Sato

6/21/2013 NCSA 2013 Session 3447

Learning Models & Assessment of Special Populations 2

SUSAN WEIGERT, PH.D. OSEP US-DOE PROJECT OFFICER

6/21/2013 NCSA 2013 Session 3447 Learning Models

& Assessment of Special Populations 3

https://ccsso.confex.com/ccsso/2013/webprogram/Person219.html

OSEP US-DOE Projects

• General Supervision Enhancement Grants (GSEG) FY2007-2010

• Administration Change in 2008

• Advocacy of Several Disability Advocates

• Message of Inclusion

• Promise of Greater Test Access and Sensitivity



• Problem:

Promote better access to grade level instruction

for students with disabilities who are behind grade level.

How?

• One Solution:

– Formative assessments supporting Learning Analytics/Learning Progressions from the student’s access point toward the grade level standard.



Tools

• CBM– fundamental skills

Off grade level

• Traditional Formative assessment

– Normative models of difficulty

• Can we Improve formative assessment?

– Reached out to Consortia

– Sponsored a series of Colloquia

– White Papers

6/21/2013

NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations

6

LEAD PRESENTER: EDYNN SATO, PH.D.

.





Colloquium on Learning Models, Instruction, and Next Generation Assessments that Include Special

Populations

October 26, 2012

Washington Marriott at Metro Center Washington, DC



Background:

General Supervision Enhancement Grant

• WestEd, the Kansas State Department of Education, and the Louisiana Department of Education (H373X070002)

• Project Officer: Susan Weigert

• Focus of grant (general):

– Technical Assistance on Data Collection

– Priority A––Modified Academic Achievement Standards

• Dissemination

1 Learning Models Colloquium 6/21/2013 NCSA 2013 Session 3447 Learning Models


Colloquium Purpose

Learning Models: • in development in the U.S.

• proposed as the foundation for designing comprehensive next generation assessment systems—both formative and summative

Colloquium will involve discussion of: • what is known,

• what is in the works, and

• what needs to be known or investigated



Learning progressions have been proposed for use in both large-scale and classroom assessments. In both cases, they may provide more detailed information about student thinking than more traditional models of assessment. This detailed information is particularly important in the classroom, where it can be used as the first step in a formative assessment process, to impact instructional decisions and provide feedback to students, ultimately improving student learning (Alonzo & Steedle, 2008, p. 419).



PATRICIA ALMOND, PHD CATE—UNIVERSITY OF OREGON




Agenda Time Session Topic

Lear

nin

g M

od

els 8:00 a.m. 0

Form Discussion Groups

Welcome & Introductions

9:00 a.m. 1 Learning Models, Instruction and Assessment

10:00 a.m. 2 Diversity of Special Populations

10:55 a.m. 3 Technical Considerations

Learning Analytics & Data Mining

12:00 p.m. Working

Lunch

Implications for the Future

Report out

Nex

t G

ener

atio

n 1:00 p.m. 4 Learning Progression Frameworks for Use with CCSS

1:20 p.m. 5 RTTT—Special Populations in Next Generation Assessments

2:25 p.m. 6 GSEG—Special Populations in Next Generation Assessment

3:25 p.m. 7 Implications for Continued Research and Development

4:25 p.m. 8 Wrap-Up and Thank You

4:30 p.m. Adjourn

5 Learning Models Colloquium

LMC Light



Basic Agenda Format

Focused Table Discussions 15 minutes

Presentation 40 minutes

7 Learning Models Colloquium

Think Speed Dating.



Tom’s Proposition If you want one page . . .

. . . It’ll take 20 days.

If you want it today . . .

. . . It’ll take 20 pages.



Comprehensive Assessment One aspect of “comprehensive” systems was the idea that assessment would be closely linked to instruction through formative and interim assessment and would not be limited to summative, accountability assessment purposes.

Instructionally Embedded (Formative)

Interim

Summative/ Accountability



Of Particular Interest

• The learning characteristics of students eligible for Alternate Assessments based on Modified Achievement Standards

• Students whose actual achievement levels were thought to be misrepresented by ‘regular’ summative, accountability measures



Formative versus Summative?

“In CBAL they are integrated and based on a common set of models. There are strong reasons to use a common foundation for formative and summative assessment. How do you take the evidence from the formative assessment and use it in the classroom?”

“But, the formative and summative could be different. The standards are common, but the learning progression may not be the same for both assessments. I see it halfway between both of you. A summative [assessment] requires a definition of the scope and sequence.”

A common learning progression for both

formative & summative assessment.

The foundation is the CCSS—which assumed

progressions, an embedded scope & sequence.



Selected Discussant Comments

• Next generation assessments need high correspondence between interpretation of students’ progress given the model and the outcomes measurement model.

• If there are variations in student pathways, there should be correspondence in the measurement model.

• Few learning models have specifically addressed learning and progress for students with disabilities.

• Learning models need to support both instruction and assessment to inform instruction.



SUE BECHARD, PH.D. INCLUSIVE EDUCATIONAL ASSESSMENT




.

• 14 projects involving 26 states

• Lessons learned from their studies on alternate

assessments based on modified achievement

standards (AA-MAS).

• Funded by the U.S. Department of Education

in 2006-2007 under three funding sources:

• General Supervision Enhancement Grants

• Enhanced Assessment Grants

• Supplemental Funding

Lessons Learned in Federally Funded Projects That Can Improve

the Instruction and Assessment of Low Performing Students with

Disabilities (Thurlow, M., Lazarus, S., & Bechard, S. (Eds.))

Available at: www.nceo.info



Demographic Characteristics

• 4 projects looked at demographic characteristics of low

performing students. More likely to be:

– Male

– From a racial/ethnic minority

– From a low socio-economic background

– Have ELL status

• Both students with and without disabilities were low

performing.

6/21/2013 NCSA 2013 Session 3447 Learning Models & Assessment of Special Populations

22

Disability Categories

• 4 projects looked at the disabilities categories of students who might be candidates for an AA-MAS.

• All found that a majority of the students had specific learning disabilities.

• Other categories:

o Intellectual disabilities

o Speech and language impairments

o Other health impairments

o Emotional/behavioral disabilities

o Autism

o Other disability categories


23

Opportunity to Learn

4 projects looked at whether students who were candidates for

the AA-MAS had the opportunity to learn the content. All found that some students may not have had adequate opportunity to learn.


24

Learning Characteristics/Barriers

10 projects found one or more of these learning characteristics/barriers: • Difficulty interacting with print - poor reading skills • Difficulty solving problems that require multi-step solutions • Easily distracted • High vocabulary load • Lack of availability of needed accommodations • Limited meta cognition • Need clarification of instructions • Poor organizational skills • Self-monitoring skills • Slower work pace • Text structure (passage length and formatting) • Working memory capacity


25

Implications for an Inclusive

Comprehensive Assessment System

• Develop systems that support student achievement.

• Mine the data to learn more about what is happening

with low performing students.

• Develop clear participation guidelines that seamlessly

include all students in the assessment system.

• Develop assessments that incorporate the principles of

universal design in an online environment.

• Provide high-quality professional development.



Suggestions for Further Research

• Research on growth and growth expectations for this population.

• Investigations into the effects of specific accommodations, supports, and scaffolding in an online environment

• Research on professional development needed for higher education training programs.

• Investigations into the experiences of this population during testing.


27

MARGARET HERITAGE, PH.D. ASST DIR PROFESSIONAL DEVELOPMENT (CRESST)




Special Populations, Comprehensive Next Generation Assessment Systems, and Learning Models: Transition to Assessment for Learning NCSA | National Harbor, MD June 19-22, 2013

Margaret Heritage

CRESST/UCLA

Learning Models to Support Instruction and Assessment

Internal:

What is going

on inside

students’ heads

What Is A Learning Model?

External:

Goals for

student

learning



Learning Models?

• Validated learning

progressions/trajectories

• Standards progressions

• Organized curricular

goals based on internal

logic/knowledge of

discipline



Learning Progressions

Learning progressions should be based on what we know from research about student learning and need to be empirically tested

(Mosher, 2012; Pellegrino, 2011)



Equipartitioning Trajectory

• Description of mathematics and cognition

• Description of achievement level

• Description of what to expect

when children are working

toward the level

• Examples of working toward

the level

• Assessment tasks



• Can equipartition a single whole. The difference

between Level 1 and Level 2 is that here the child

is partitioning a particular shape into smaller

shares with equal areas (instead of partitioning

collection of objects). In most case for young

children, being able to create equal-sized areas

is a more complex action than dealing

collections.

• Create multiple unequal-sized parts by

“chopping” the whole into many pieces and then

allocating them

• Create the correct number of parts, but the parts

are of unequal size

• Create equal-sized parts but create too many of

them and want to “throw away” the extra parts

Children who are

working towards this

level:

Children who have

achieved this level:

(Confrey, 2010)

Equipartition Level 2: Sharing a Whole



Standards Progressions

Students develop through a series of levels

of geometric and spatial thinking. Initially,

students cannot reliably distinguish between

examples and non examples of categories of

shapes, such as triangles, rectangles, and

squares. With experience, they progress to

the next level of thinking…

(CCSS Math Writing Group, 2012)

6/21/2013


35

(Di Ranna et al., 2008)

Conceptual Flow



Put image of table from p. 69 of reflective teaching on your desk

(Di Ranna et al., 2008)

Assessment Plan



• Clear descriptions of learning pathways

• Assist understanding of how student thinking and

skills develop within domains

• Descriptions of performance: on the way to,

achieved a stage in the progression

• Clarify difficulties, common misconceptions and

challenges to learning in a domain

• Guide assessment tasks

• Provide a framework for interpretation

Utility for Instruction and Assessment



Moving Forward

• We need more research on learners’

thinking related to core ideas in

domains to develop curricula,

instruction and assessment

• We need to pay attention (more

research) on models for learners with

disabilities

• We have a lot to do!

6/21/2013


41

VINCENT J. DEAN, PH.D., MI DIRECTOR, OFFICE OF STANDARDS & QUALITY ASSESSMENTS




Implications for Special

Populations in Next Generation

Assessment

State Perspective on Smarter Balanced Assessment Consortium

Vince Dean, Ph.D.

Director, Office of Standards & Assessment Michigan Department of Education

A Balanced Assessment System

Common

Core State

Standards

specify

K-12

expectations

for college

and career

readiness

All students

leave

high school

college

and career

ready

Teachers and

schools have

information and

tools they need

to improve

teaching and

learning

Interim assessments Flexible, open, used

for actionable feedback

Summative assessments

Benchmarked to college and career

readiness

Teacher resources for formative

assessment practices

to improve instruction



Building a Logical Argument

Student Response

Evidence

Assessment Target

Claim

Career & College Readiness/ Common Core State Standards



6 Key Components of Evidence-Centered

Design

6. Develop Items or Performance Tasks

1. Define the domain Common Core Standards Math/ELA

2. Define claims to be made ELA & Math Claims

Content Specifications

3. Define assessment targets Knowledge, Skills, & Abilities

4. Define evidence required Evidence to be Elicited from Student

5. Develop Task Models Methods for Eliciting Evidence



Formative Assessment Definition for Smarter Balanced Assessment Consortium

Formative Assessment is a deliberate process used by teachers and students during instruction that provides actionable feedback that is used to adjust ongoing teaching and learning strategies to improve students’ attainment of curricular learning targets/goals.



Formative Assessment Process

Heritage, 2009 6/21/2013


49

The Formative Assessment Process Is At Every Level of the RtI Process

Intensive Evidence-based Interventions

Targeted Evidence-based Interventions

Universal Instruction

1 to 5%

5 to 10%

80 to 90%



A System of Support

*Students &

Models

Formative Practices

Learning Goals Success Criteria Learning Activity

Interim Assessments

Claims Standards Learning activities

Summative Assessments

Claims Standards

Learning Models

Learning Maps Learning Progressions



52 6/21/2013 NCSA 2013 Session 3447 Learning Models

& Assessment of Special Populations

CAT-Specific Opportunities

• Individualizing the student test event

• Increased measurement precision

• More appropriate indication of individual student growth

• Item difficulty, complexity, linguistic characteristics

• Item pool depth

• Cross-grade administration

• Administration time



CAT-Specific Challenges

• Content constraints

• Item pool size – Substantially more items are needed

– Complicated by the number of item types

• Item pool distribution shape – More rectangular than bell shaped

• Algorithm specifications – How many adaptive facets?

• Item type, difficulty, DOK, LDL, etc.

55 6/21/2013


Lessons Learned - Designing CAT for Special Populations

• Clarity of purpose is even more important when building a CAT assessment

• More comprehensive professional development is needed (e.g., item writers, test administration)

• Increase adaptive characteristics only when it adds substantial, reliable value

• Must have an item inventorying feature able to quickly report by multiple variables

• Cognitive Labs are essential



SANDRA WARREN, PH.D. ASES SCASS FACILITATOR

QUESTIONS FOR

DISCUSSION




ASES SCASS Perspective

“What is it we don’t know that is critical to the success of students with disabilities?” • Sometimes we don’t know what we don’t

know until we’ve started the journey. • Now that we are midway in this journey,

what are we learning we don’t know….and how might we move forward to support SEAs , LEAs, and schools?



ASES/NCEO NCSA Pre-Session Discussion – Action Steps/Areas

• Ensure accountability for student outcomes • Invest in strong LEA & school leadership • Use data to understand student trajectory • Ensure all students have access to AT and

professionals know how to support use • Ensure access to best practices regardless of

geographical location • Preserve integrity of graduation diploma with

multiple pathways • Clarify college vs. career readiness



Panel Discussion

To increase our understanding of: Research …. Policy …. and Practice

for special populations, comprehensive next generation assessment systems, and

learning models

“What is it we don’t know that is critical to the success of low performing students

with disabilities?”



DISCUSSION FACILITATORS: SANDRA WARREN, PH.D. & EDYNN SATO, PH.D.









special populations, comprehensive next generation ... · comprehensive next generation assessment...

Documents