Understanding Learning Models to Inform the Development of Assessment for Students in Special Populations

Edynn Sato

WestEd

Renee Cameto

SRI International

Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations

1

Cameto, R., Bechard, S., & Almond P. (Eds.). (2012).Third Invitational Research Symposium: Understanding

Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in

Special Populations. Manuscript in preparation. Menlo Park, CA, and Lawrence, KS: SRI International and

Center for Educational Testing and Evaluation (CETE).

Topic 2 Work Group • Renee Cameto

• Gary Cook

• Nancy Harayama

• Donna Lehr

• Kate Nagel

• Edynn Sato

• Alan Sheinker

Learning models reflect HOW students learn • Learning models (learning progressions and learning maps)

describe the typical pathways that students take as they develop understanding from novice to mastery of concepts and skills, based on research and conceptual analysis (Smith et al, 2006).

• Learning models visually and verbally articulate an hypothesis about how learning will typically move toward increased understanding for most students (Hess, 2009.)

• Learning models differ in scope, breadth, and grain size (Heritage, 2011)

3

Understanding Learning Progressions and Learning Maps to Inform the Development of Assessment for Students in Special Populations

Learning models reflect HOW students learn (cont.)

• Learning models propose the intermediate understandings that are “reasonably coherent networks of ideas and practices…that contribute to building a more mature understanding…the important precursor ideas may not look like the later ideas, yet crucially contribute to their construction.” (Taking Science to School, 2007)

• Learning models reflect systematic consideration of interactions among the learner, the content, and the context for learning, as well as the dynamic, cumulative outcomes of these interactions (Cameto, Bechard, & Almond, eds., 2012).

Characteristics of Learning Progressions and Learning Maps

Are organized around

“big ideas” or “unifying

ideas”

Consist of pathways

and key learning targets

Multiple pathways are

implied

Provide potential

interim learning targets

built upon earlier

learning along multiple

(and interrelated)

pathways (or strands)

Reflect the “universe” of

cognition (abilities),

concepts (knowledge), and

skills related to a domain

Consist of networks of

sequenced learning targets

Multiple pathways are

embedded

Are sometimes more

detailed than progressions

Based on cognition, concepts,

skills

Multi-dimensional

Can be sequential and/or

cyclical

Can have a range of granularity

Have components that are:

Static=based on normative data

and/or Dynamic=based on

individual learning data

Do not address rate of learning

Reflect consideration of

interactions among learner

characteristics, content, and

context

Validation is data-based and

iterative

Learning Progressions Learning Maps

Learning Models

Premises

• Without a clear definition of how students represent knowledge and develop competence, valid inferences about students’ thinking processes, including misconceptions, strengths, and abilities , and how to effectively support student learning are questionable.

• Special population students may not follow the “most common” manner in which learning is hypothesized to advance over time toward understanding.

• Upfront consideration of the interaction among learner characteristics, content, and context is critical to ensure effective instructional practices and valid interpretations of assessment outcomes—there needs to be a shift in practice: upfront consideration of our full range of learners

Some student characteristics to consider upfront:

English language learners Students with disabilities

Various levels of literacy and fluency

in native language and in English

Range of ability to communicate in

English

Learning is affected by factors that

include:

o Socio-cultural background and

experiences

o Amount and length of formal

education (in country of origin as

well as in the U.S.)

Students have multiple ways of

expressing and receiving information

Students have the challenge of learning

English while learning rigorous

academic content

Heterogeneous population of students

Variable rates of learning, and rates

may vary more across time and content

Learning is affected by factors that

include:

o Socio-cultural background and

experiences

o Familiarity with content, methods

(e.g., accommodations used), and

context

Students have multiple ways of

expressing and receiving information

Heterogeneous population of students

Recall from previous presentation:

• Learning models “represent a promising framework for developing meaningful assessments, allowing both large-scale and classroom assessments to be grounded in models of how understanding develops in a given domain” (Alonzo & Steeldle, 2008).

• Learning models provide a cohesive cognitive framework that is crucial for correspondence between the components of a comprehensive assessment system (Cameto et al, 2012).

• Learning model assessment seeks to identify where a student falls along a learning progression while also maintaining the ability to determine whether the student is proficient on a given grade level standard.

How do we understand “where students are” vis-à-vis learning targets?

Academic Target

• Common Core State Standard: Reading 7.3A: Analyze Elements

• Analyze how particular elements of a story or drama interact (e.g., how setting shapes the characters or plot)

Target construct: Ability to determine how the plot of a story or drama is advanced in response to characters’ actions

Example profile: Ethan (7th grade)

• can follow 1-2 step verbal directions with frequent verbal prompting

• uses oral communication; can sound out letter sounds and common consonant blends

• likes reading simplified fiction and non fiction books (2nd grade level)

• enjoys listening to audio books

• shows growing understanding of characters and understands common emotions in others

• can paraphrase simple paragraphs and understands cause and effect

• intellectual disabilities

• moderate visual impairment (20/150 with correction) and needs to wear eye glasses for near and distance work

• poor hand eye coordination skills and fine motor limitations

• needs extra time to read and finish assignments, and loses his place when reading

• needs an anti-glare computer screen

• has some off task behaviors and can be distracted especially by peers but is easily redirected

Ethan: Background Skill/Knowledge

Background Skill/Knowledge

The

stu

den

t h

as/h

ad

op

po

rtu

nit

ies

to

dem

on

stra

te/u

se t

his

sk

ill/k

no

wle

dge

him

/her

self

a

lot

WIT

HO

UT

an a

dap

tati

on

o

r ac

com

mo

dat

ion

The

stu

den

t h

as/h

ad

op

po

rtu

nit

ies

to

dem

on

stra

te/u

se t

his

sk

ill/k

no

wle

dge

him

/her

self

a

lot

WIT

H a

n a

dap

tati

on

or

acco

mm

od

atio

n

The

stu

den

t h

as/h

as

op

po

rtu

nit

ies

to

dem

on

stra

te/u

se t

his

sk

ill/k

no

wle

dge

him

/her

self

le

ss f

req

uen

tly

WIT

HO

UT

an

adap

tati

on

or

acco

mm

od

atio

n

The

stu

den

t h

as/h

as

op

po

rtu

nit

ies

to

dem

on

stra

te/u

se t

his

sk

ill/k

no

wle

dge

him

/her

self

le

ss f

req

uen

tly

WIT

H a

n

adap

tati

on

or

acco

mm

od

atio

n

The

stu

den

t h

as/h

as m

inim

al

op

po

rtu

nit

ies

to

dem

on

stra

te/u

se t

his

sk

ill/k

no

wle

dge

him

/her

self

(e

.g.,

hea

rd o

r re

ad a

bo

ut

it)

Ability to identify how a story’s or drama’s plot unfolds in a series of episodes.

X

Knowledge that elements of a story or drama (setting, characters, plot) interact with each other

X

Ability to determine how the plot of a story or drama is advanced in response to characters’ actions

Cognitive background

knowledge

Student Need

(Yes/No)

Variable Features

Ability to identify how a story’s or drama’s plot unfolds in a series of episodes.

Yes VF1. Remind student of definition of plot VF3. Remind student of definition of character traits

Knowledge that elements of a story or drama (setting, characters, plot) interact with each other

Yes VF4. Provide a simplified Freytag's pyramid showing the structure of dramatic piece: exposition, rising action, climax, falling action, and dénouement (or adaptation of same) – VF5. Explicitness of elements in the text

Categories of Support

Student Need

(Yes/No)

Variable Features

Sensory (VF: Perceptual Language and Symbols)

Yes P1: Delivery mechanisms by which the question is perceived. (e.g., read aloud verbatim/read aloud paraphrase, pictures, large print, printed text, symbols, rebuses, concrete objects, description of objects or images, P3: Delivery parameters for oral presentation of material. (e.g., speed of reading, volume, amount of expression used, student ability to pause, stop and/or repeat information read aloud); Digital text with or without automatic text to speech; Digital Braille with or without automatic Braille to speech; Enlarged materials/pictures; Sign language, auditory amplification, CCTV – close circuit TV, to increase size of font, vary contrast; Assistive visual devices- prescription glasses, monocular, contact lenses, dark glasses; Anti glare screen; Answer in answer booklet L5: All key information in sign language for students who utilize this method of communication

Cognitive (VF: Perceptual/Receptive: Skill & Fluency/ Expressive,)

Yes EXCERPTED C1. Depth of knowledge of the content – C2. Complexity of the content (e.g., length of scenario , number of supporting details included, richness of context) C3. Item/task format (selected response vs. constructed response, performance, etc.) C4. Adjustable levels of challenge (teacher able to adjust) Options for supporting background knowledge – C5. Pre-teach background content (pre-teach definitions of unfamiliar words or concepts unrelated to the standard; pre-teach means teaching a student for the first time the definition of a word or concept that is included in the narrative of a test item but not part of the construct being measured) C6. Provide analogies and examples

C7. Provide hyperlinks to multi-media

Options for guiding exploration and information processing: Use consistent signals/cues. (Signals/cues may include designations in

assessments such as line numbers in passages, symbols for directions [e.g., stop signs to stop, arrows to continue], or behavioral gestures indicating where a student should mark a response) Options for supporting critical features, big ideas, and relations: Provide familiar organizational tools [e.g., tables] and familiar concrete objects and/or using familiar organizational processes [e.g., how highlighting is used]) C13. Provide graphic organizers

C14. Outline information

C15. Highlight information

Physical (VF: Skill & Fluency)

Yes P2: Supports for the use of equipment required for the task. (e.g.,communication board, CD player; reprogram communication board to include punctuation, capitalization, etc.) S1:Response mode options. (e.g., pointing, speech and verbalization, writing, signing, switch or other assistive device/augmentative communication device, eye gaze; for lowest functioning students: predictable behavioral response, tolerate assistance such as hand over hand) S2: Supports for composing a response in text. (e.g., speech to text, written by teacher, keyboarding, word prediction software) S3: Supports for manipulating physical materials. (e.g., use of velcro, non slip mat, enlarged materials, reading line guide teacher manipulation of materials; In writing, student can manipulate cards with punctuation symbols on them and velcro on back to apply correct punctuation to a sentence) S4: Supports for manipulating digital/electronic equipment. (e.g., pointers, teacher manipulation of equipment, spoken commands, stylus for input, larger keyboard/buttons, adaptive mouse) S5: Practice tutorials with unfamiliar physical materials or digital/electronic equipment. (Practice tutorials can be used to introduce students to new item formats or modeled examples using materials that are not construct relevant or new tools to support test taking) S6: Practice with familiar equipment

Linguistic Yes L1: Level of abstraction required of student. (e.g., concrete objects, images, text) L4: All key information in the dominant language (e.g.,English) is also available in prevalent first languages (e.g.,Spanish) for second language learners; Use of translation software, bilingual dictionary

L6: Use of multiple representations. (e.g., physical models, demonstrations, acting out scenarios) L7: Alternate syntactic levels (simplified text). L11: Read language and symbols aloud.

Socio-Linguistic

Yes L2: New vs. pre-taught vocabulary and symbols

L3: Embedded support for vocabulary and symbols. (e.g., technical and non-technical glossary, hyperlinks/footnotes to definitions, illustrations, background knowledge) L8: Highlight essential elements, words, or phrases.

Executive

functioning/

Executive

processes

(VF: Executive)

Yes E1: Prompts and scaffolds to estimate effort, resources, and difficulty E2: Prompts, scaffolds, and questions to monitor progress, to "stop and think", and for categorizing and systematizing. E3:Representations of progress. (e.g., before and after photos, graphs and charts) E4: Guides, checklists, graphic organizers, and/or templates for goal setting, prioritizing, breaking long-term objectives into reachable short-term goals, self-reflection, and self-assessment. E5: Adjust levels of challenge and support. (e.g., adjustable leveling and embedded support, alternative levels of difficulty, alternative points of entry)

Emotional (VF: Affective

Yes EXCERPTED Teacher options for providing supports for attention and

engagement: A1. Cover up part of text so student isn’t overwhelmed

A2. Prompt student to engage/re-engage

A3. Provide verbal/gestural prompts

A4. Provide feedback to support engagement A5. Provide supports to reduce student frustration (e.g., noise reduction, extended test taking time, contingencies, number of items administered at one time) Task options for engagement (task refers to the

assessment items, scenario, and materials): A9. Provide students with choices for personal control of age-appropriate content when construct is not impacted (e.g., choice of topic or theme) MAY NOT BE APPLICABLE FOR STATEWIDE ASSESSMENTS

A10. Provide students with choices for personal control of task context when construct is not impacted NOT MAY NOT BE APPLICABLE FOR STATEWIDE ASSESSMENTS

A11. Enhance relevance, value, and authenticity of tasks

A12. Heighten salience

A13. Variety of stimuli A14. Vary amount of context supporting tasks (e.g., discrete tasks vs. scenarios)

UDL CATEGORIES

CATEGORY OF

SUPPORT

Receptive

(representation)

Expressive

(expression)

Engagement

Sensory Delivery mechanisms by which the question is perceived read aloud verbatim, large print, Delivery parameters for oral presentation of material

Access needs and strategies to address needs

Cognitive (knowing) Remind student of definition of plot Remind student of definition of character traits Provide a simplified Freytag's pyramid showing the structure of dramatic piece: exposition, rising action, climax, falling action, and dénouement Explicitness of elements in the text Remind student of prior experiences with the target construct Remind student of materials or activities used to teach foundational reading/English language arts skills. Provide alternate forms of key concepts

Alternate syntactic levels (simplified text)

Item/task format (selected response vs. constructed response, performance, etc.) Indicate where a student should mark a response

Locate items near relevant text

Use consistent signals/cues.

Physical Supports for manipulating physical materials. Supports for manipulating digital/electronic equipment Practice tutorials with unfamiliar physical materials or digital/electronic equipment. Practice with familiar equipment

Supports for composing a response in text Response mode options

Linguistic Level of abstraction required of student. (e.g., concrete objects, images, text) Use of multiple representations Read language and symbols aloud.

.

Socio-linguistic Embedded support for vocabulary and symbols

Highlight essential elements, words, or phrases

Embedded support for vocabulary and symbols. Highlight essential elements, words, or phrases

Emotional Cover up part of the text so the student isn’t overwhelmed

Prompts, scaffolds, and questions to monitor progress, to "stop and think", Provide verbal/gestural prompts

Provide supports to reduce student frustration. Provide feedback to support engagement, Prompt student to engage/re-engage, , Use student’s typical reward system. Provide optimal student positioning Administer assessment at optimal time of day for student engagement

Executive

functioning/

Executive processes

Remind student of the function of tools/features designed to aide comprehension and processing of information. (e.g., highlighting, graphic organizers, captions, and headings) Reread question/stimulus. Highlight information

Adjust levels of challenge and support. (e.g., adjustable leveling and embedded support, alternative levels of difficulty, alternative points of entry)

Use consistent signals/cues.

Note: Supports provided to address student needs should not alter the targeted construct.

Summary

• Academic target/construct

• Student characteristics

– Student academic needs (content implications)

• Cognitive background knowledge

• Categories of support

– Sensory

– Cognitive

– Physical

– Linguistic

– Socio-linguistic

– Executive functioning

– Emotional

• UDL categories

– Receptive (representation)

– Expressive (expression)

– Engagement