assessment considerations casp convention 2015 rodrigo enciso, m.a., lep san diego unified school...

54
Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo Enciso

Upload: darlene-golden

Post on 17-Jan-2016

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

1

Assessment Considerations

CASP Convention 2015

Rodrigo Enciso, M.A., LEPSan Diego Unified School District

Page 2: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

2

Problems in SLD Identification

Have you ever ran into:• A struggling student with an FSIQ in the mid-80’s, caused by a

Working Memory Index score in the high 60’s, is not made eligible for Special Education (Sp Ed) services since a severe discrepancy is not met.

• Sp Ed administrator suggest qualifying a student as SLD by calculating a significant discrepancy between 81 IQ score and 103 written expression score.

• Psychologist administers 4 or 5 different complete test batteries before a processing deficit emerges.

• Psychologist has to administer the C-TONI because “it tends to give you a higher IQ score than comprehensive batteries” so that you can get a 21 point discrepancy.

Page 3: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

3

Case Sample: what do our test measure?

• Student initially evaluated in 2009 within a month of beginning 7th • African American descent, and according to California law, IQ instruments were not utilized. • Several memory ability measures were used and all fell within the Average to High Average ranges. • Visual processing scores also fell within the Average range. • Results on tests of executive functioning (D-KEFS) indicated two out of six subtest with below average

scores. The examiner’s conclusion was a sensory-motor skills deficit (also known as visual-motor integration).

• Above Average/Superior reading comprehension and word reading. • Low Average range in math operations (SS=89) and Below Average range in math reasoning (SS=81) (math

reasoning requires solving mathematical word problems; none to very little writing demands)• It was indicated that student met the criteria for special education services as a student with a Specific

Learning Disability (SLD) in math reasoning due to a severe discrepancy and a processing deficit in sensory-motor skills.

• Subsequently, the student was again administered an academic achievement test in 8th grade. This time she scored within the Average range in reading and math, and within the Above Average/Superior in written expression. Scores for basic math was 99 and for math reasoning was 92; math fluency was 82.

• Triennial assessment 2012, 10th grade: Average visual-motor integration (VMI-V); Average math scores. Math classroom measures supportive of grade level skills. RSP teacher recommends exiting.

Page 4: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

Need for Change

• We know more now than we did even 20 years ago.• Current research provides more accurate

information about what our tests and other assessment tools measure.

• We can’t just keep doing the same things just because that’s how they’ve always been done.

• Expert consensus is that the Ability-Achievement Discrepancy Model is flawed and problematic (LDA 2010).

4

Page 5: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

5

Could we just squash the Ability-Achievement Discrepancy Model?

Page 6: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

6

Trends in Assessment

From Dawn Flanagan’s presentation at the CASP 2012 Meeting of the Minds convention

Page 7: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

Theory and Research of Contemporary Cognitive Abilities

7

Page 8: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

8

CHC theory of cognitive abilities (Cattell-Horn-Carroll)

• Knowledge of what our tests measure• Represents, in order of contribution, the most

recent integration of the original and widely influential works of: Raymond Cattell – “C” John Horn – “H” John Carroll – “C”

• CHC theory is the most comprehensive and empirically supported psychometric theory of the structure of cognitive and academic abilities (McGrew, 2005).

Page 9: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

9

Contemporary CHC Theory:(5th Wave)

• Includes 16 broad cognitive abilities – Approximately 9 broad abilities measured by current batteries

• Includes 80 narrow cognitive abilities – Approximately 38 narrow abilities measured by current batteries

Page 10: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

10

Broad Cognitive Abilities Based on CHC Theory Measured by Mayor Test Batteries

Crystallized Intelligence (Gc): The depth and breadth of knowledge and skills that are valued by one’s culture.Fluid Reasoning/Intelligence (Gf): The deliberate but flexible control of attention to solve novel “on the spot” problems that cannot be performed by relying exclusively on previously learned habits, schemas, and scripts. Auditory Processing (Ga): The ability to detect and process meaningful nonverbal information in sound.Long-Term Storage and Retrieval (Glr): The ability to store, consolidate, and retrieve information over periods of time measured in minutes, hours, days, and years.Short-Term Memory (Gsm): The ability to encode, maintain, and manipulate information in one’s immediate awareness. Processing Speed (Gs): The ability to perform simple repetitive cognitive tasks quickly and fluently.Visual Processing (Gv): The ability to make use of simulated mental imagery (often in conjunction with currently perceived images) to solve problems.

Page 11: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

11

Getting Away from Discrepancy: Exclusion of general ability or “g”

• Does not mean that the integrated CHC model does not subscribe to a separate general human ability or that g does not exist.

• Rather, g was omitted by McGrew (1997) and Flanagan et al., (2000) because it was judged to have little practical relevance to the selection, organization, and interpretation of tests around referral concerns for Learning Disability evaluations.

• In addition, current literature is demonstrating the inadequacy of g in predicting specific areas of achievement and the fact that certain broad cognitive abilities predict significant variance in academic skill acquisition above and beyond that of g. (Flanagan et al., 2013)

• Knowledge of specific broad and narrow cognitive abilities also provides richer information regarding intervention than g. (Flanagan et al., 2013)

Page 12: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

12

Three Methods in IDEA for SLD Identification

1. 56337(a) A severe discrepancy exists between the intellectual ability and achievement…

2. 300.309(a)(2)(i) The child does not make sufficient progress… using a process based on the child's response to scientific, research-based intervention.

3. 300.309(a)(2)(ii) The child exhibits a pattern of strengths and weaknesses in performance, achievement, or both… that is determined by the group to be relevant to the identification of a specific learning disability, using appropriate assessments

Page 13: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

13

“Alternate” Method of SLD Identification

• Section (b)(10)(C)(2) – expands the methods of identification to include references to (i)Response to Intervention (RtI), and (ii)Pattern of Strengths and Weaknesses (PSW).

• (ii) The pupil exhibits a pattern of strengths and weaknesses in performance, achievement, or both, relative to age, State-approved grade-level standards, or intellectual development, that is determined by the group to be relevant to the identification of a specific learning disability, using appropriate assessments, consistent with 34 C.F.R. sections 300.304 and 300.305; and ……. exclusionary criteria are ruled out (extrinsic factors)

Page 14: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

14

2014 CA Regulation Updates

• Substitution of “… and has a severe discrepancy between intellectual ability and achievement” with “the public agency may consider whether a pupil has a severe discrepancy between intellectual ability and achievement” (b)(10)(B) (emphasis added).

• Was California guilty of Illegal practice for the last 10 years?

Page 15: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

15

We can always trust in the law, right?We can trust in the legislators and politicians to get it right, right?The Ed Code would never steer us wrong ……or would it?

Page 16: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

16

Impracticality of the Law:Guidance with complexity

The procedure for calculating a severe discrepancy using standardized test scores was almost eliminated from the recent California Reg. updates in 3030(b)(10). • Not found in Federal regulation• Some people complained of a lack of guidance, and so it was left in!

It reads:“In determining the existence of a severe discrepancy, the individualized education program team shall use the following procedures: 1. When standardized tests are considered to be valid for a specific pupil, a severe discrepancy is demonstrated by: first, converting into common standard scores, using a mean of 100 and standard deviation of 15, the achievement test score and the ability test score to be compared; second, computing the difference between these common standard scores; and third, comparing this computed difference to the standard criterion which is the product of 1.5 multiplied by the standard deviation of the distribution of computed differences of students taking these achievement and ability tests. A computed difference which equals or exceeds this standard criterion, adjusted by one standard error of measurement, the adjustment not to exceed 4 common standard score points, indicates a severe discrepancy when such discrepancy is corroborated by other assessment data which may include other tests, scales, instruments, observations and work samples, as appropriate.”

Page 17: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

17

Contradiction in Law??Interesting Observations

Despite retaining the procedure (calculation) for “determining the existence of a severe discrepancy”, the same SLD section undermines such calculation:

• 3030(b)(10)(B) The decision as to whether or not a severe discrepancy exists shall take into account all relevant material which is available on the pupil. No single score or product of scores, test or procedure shall be used as the sole criterion for the decisions of the IEP team as to the pupil's eligibility for special education.

• 3030(b)(10)(B)(3) If the standardized tests do not reveal a severe discrepancy as defined in subdivisions 1. or 2. above, the IEP team may find that a severe discrepancy does exist….

• 3030(b)(10)(C) Whether or not a pupil exhibits a severe discrepancy as described in subdivision (b)(10)(B) above, a pupil may be determined to have a specific learning disability if…

Page 18: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

18

What about those Pesky “Basic psychological processing disorder” checkboxes?

• Inconsistency between Ed Code & contemporary research on cognitive abilities (i.e., CHC theory)

• 3030 (b)(10) Specific learning disability means a disorder in one or more of the basic psychological processes…. include attention, visual processing, auditory processing, sensory-motor skills, cognitive abilities including association, conceptualization and expression.

• What is meant by cognitive abilities?

Page 19: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

19

What are Possible Definitions forAssociation, Conceptualization and Expression?

(From a team at Grossmont High School District)

• Association: long term memory deficits; poor cause/effect sense; trouble with part/ whole relationships.

– Long-term memory– Short-term memory

• Conceptualization: trouble using information in complex manner; transfer and generalization of learned materials is difficult; poor abstract thinking. Problems with understanding and reasoning, generalizing, and problem-solving.

– (Fluid Reasoning)– (Executive Processing?)

• Expression: poor organization and expression of thoughts orally, gesturally, or in writing; poor vocabulary; may be able to understand language at age level.

– linguistic and academic knowledge (Crystalized Intelligence) – Oral language– (Executive processing?)

What about Processing Speed?

Page 20: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

20

Now that we have talked about cognitive abilities, let’s talk about academic achievement!

• Must assess in areas of suspected disabilities: – what is the reason for referral? – what are the areas of concern?

• In SLD, 8 academic achievement areas are recognized by IDEA/CCR for identification of under achievement.

Page 21: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

PSW Basics - CASP 2014 Rodrigo Enciso 21

Achievement Part of SLD in Ed Code

(b)(10)(C) 1. The pupil does not achieve adequately for the pupil's age or to meet State-approved grade-level standards in one or more of the following areas, when provided with learning experiences and instruction appropriate for the pupil's age or State-approved grade-level standards: (i) Oral expression. (ii) Listening comprehension. (iii) Written expression. (iv) Basic reading skill. (v) Reading fluency skills. (vi) Reading comprehension. (vii) Mathematics calculation. (viii) Mathematics problem solving

Page 22: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

22

Ask CASP

• How much testing is enough testing?• Is the WJ-III enough?• Are measures for broad reading, broad math, and

broad written language enough?• Are the following areas needed to be assessed? If

so, what measures can be used?– Oral Language– Math Problem Solving– Written Expression– Reading Comprehension

Page 23: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

23

Best Practice

• Use multiple sources of academic information as part of a comprehensive assessment of learning disabilities, to include standardized assessments, curriculum based measures, progress monitoring data, and classroom performance.

• Diagnostic information for SLD identification = the merger between cognitive ability and academic achievement information

Page 24: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

24

Underutilized Sources of Information: Standardized achievement batteries

Riverside Publishers website about the WJ-III NU:• “Normative Update Tests of Achievement match all eight areas of IDEA

legislation”• “Expanded Broad Achievement clusters that contain 3 tests measuring

basic skills, fluency, and application”. – For example, Broad Reading Cluster contains subtests related to decoding

(Letter-Word Identification), reading speed (Reading Fluency), and comprehension (Passage Comprehension).

– In the WJ-III, math reasoning is measured by the Applied Problems subtest (administered if obtaining a Broad Math score), and can be supplemented by administering the second math reasoning subtest called Quantitative Reasoning (administering both subtests composes a Math Reasoning Cluster).

– Quality of written expression is measured by the Writing Samples and Writing Fluency subtests.

Page 25: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

25

Underutilized Sources of Information: continued

• Most major standardized academic achievement batteries include several subtests in order to tailor your assessment to the reason for referral/areas of concern.

• Administration of the entire battery is not needed for the vast majority of the cases.

• The WJ-III tests of achievement has about 18 subtests to choose from, but a lot of the useful features from this and other achievement batteries are not utilized or underutilized. – For example, the WJ-III achievement battery has 3 subtests that assess

phonological awareness (one of the strongest predictors of basic reading skill acquisition); Which education specialist administers all 3 of these subtests when assessing a student for a referred basic reading skills concern?

Page 26: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

26

A New Model Of SLD Identification: Patter of Strengths and Weaknesses

The use of the PSW model:• Promotes the use of good evaluation practices (e.g., use of the 4

pillars of assessment-RIOT; Sattler, 2008)– Seeks to explain a student’s observable academic deficiencies (the

consideration of ecological validity) – Highlights the importance of ruling out exclusionary factors– Deemphasizes decision making based on “single score or product of

scores” • Is grounded in research based theoretical models (e.g., CHC

theory)– Focuses assessment of cognitive abilities that have been found,

through current research, to correlate with the specific academic areas in question (Flanagan et al., 2013).

Page 27: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

27

PSW: general principlesAlthough diverse models of PSW exist, they follow these general principles:

1. The Full Scale IQ is irrelevant except for Intellectual Disability(ID)diagnoses. 2. Children classified as SLD have a pattern in which most academic skills and

cognitive abilities are within the average range. However, they have isolated weaknesses in academic and cognitive functioning. This conforms to Sally Shaywitz’ (2003) declaration that dyslexia is “an isolated weakness in a sea of strengths.”

3. Each model demands that we “match” deficits in specific cognitive processes to the specific area of academic concern without testing children with numerous measures in an attempt to find a deficit.

4. Most cognitive abilities that do not relate to the area of academic concern are average or above.

(OSPA, 2008)

Page 28: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

28

PSW: research-based models

• Dual Discrepancy/Consistency (DD/C) model proposed by Flanagan, Ortiz, and Alfonzo

• Consistency-Discrepancy model proposed by Naglieri

• Concordance-Discordance model proposed by Hale & Fiorello

Page 29: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

29

PSW: operational definition• Having an operational definition of SLD is important to implementation

consistency • The operational definition in the Dual Discrepancy/Consistency (DD/C)

model of SLD includes the following essential components:Evidence of -

Cognitive weakness/deficit (< 85)Academic weakness/deficit (< 85)Cognitive strengths (> 90; important for differential diagnosis) Consistency between cognitive & academic deficits (must be related)Weaknesses are manifested in environment (ecological validity)

Flanagan et al. (2013)

Page 30: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

30

PSW: cognitive strengths within average range

An important premise within the operational definition of SLD is that the domain-specific cognitive weakness and academic weakness are both found within “an otherwise typical” pattern of functioning (SS 90 or above). This is particularly important for differential diagnosis (identifying true SLD cases).

Example of problematic outcomes:I was tasked with re-evaluating a middle school student who had continued to display academic and behavioral challenges and atypicalities since kindergarten. This student’s educational disability appeared to change at every evaluation depending on the combination of scores obtained at that particular time, initially identified as having a language impairment, then as having ID, then as having SLD, and then back to ID. I was being asked to look for ED; while at the end, the student was none of those things but rather demonstrated a more complex disability.

Autism Spectrum Disorder (ASD).

Page 31: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

31

apparently reading about the causes of cancer can give you cancer

Page 32: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

32

PSW: consistency between cognitive & academic deficits

Another crucial principle is that the identified cognitive and academic weaknesses must be theoretically and empirically related. That is, there must be research-based evidence that the academic skill in question is found to be impacted by the cognitive deficit identified; such is the knowledge gained from CHC theory.

Example of problematic outcomes:A student being identified as SLD based on –

a visual processing deficit (cognitive weakness) and an oral language academic deficit (academic weakness).

Practices that violate this principle can be regarded as “fishing” for a deficit, rather than being focused on the cognitive abilities most related to the reason for referral.

Page 33: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

33

If the problem can’t be seen, heard, or felt, is it really a problem?Hillary Clinton wishes 'Benghazi' non-issue would go away already

Page 34: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

34

PSW: weaknesses are manifested in environment

Without an academic weakness being manifested and observed within a natural context (e.g., the classroom), any findings of that academic weakness through standardized tests would be irrelevant. In other words, if the problem does not present itself in the real world… then the problem does not exist.

Example of problematic outcomes:I was asked to consult in a case where an initial psycho-educational evaluation had been conducted. The student had been referred for reading and writing difficulties by the classroom teacher; math had been reported as a strength. However, based on the standardized test scores obtained, the student was reported to have a learning disability in math and found to have average skills in all other areas.

What’s a possible hypothesis?

Page 35: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

PSW Basics - CASP 2014 Rodrigo Enciso 35

What is CASP’s position on the PSW model?

Refer to handout-CASP Position Paper: Specific Learning

Disabilities and Patterns of Strengths and Weaknesses

Page 36: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

36

Research Based Practice: Closing the gap

The work of Flanagan, Ortiz, and Alfonso (2013) facilitates the practical implementation of the research on contemporary assessment through their Cross Battery Assessment approach.

• Brought all batteries and interpreted them within CHC theory– Applying CHC theory and research of learning disabilities into test selection

• From test kit focus to theory focus– Combines subtests from different intelligence batteries according to reason for

referral• Psychometric and theoretically defensible• An extension of XBA, the Culture-Language Interpretive Matrix (C-LIM),

assists in determining the validity of obtained test results with ELLs that serves as an additional method for interpreting test results in the identification of SLD in ELL populations.

Page 37: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

37

Research Based Practice: Closing the gap

• “Cross Battery Assessment is intended to enhance measurement by providing a theoretical framework within which any and all tests can be understood in terms of their measured construct and that it provides rigorous psychometric rules so that the composites that are formed can be defended as being both reliable and valid.”

–Dr. Samuel Ortiz

Page 38: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

38

What Does Your Test Measure?

Page 39: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

39

Relationship Between Cognitive Abilities and Academic Achievement

Refer to “Relations Between CHC Abilities and Specific Areas of Academic Achievement” from the Essentials of Cross Battery Assessment books.

Page 40: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

40

Another view at current research of cognitive abilities and academic achievement relationships.

Page 41: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

41

PSW: clinical judgment is important

Regardless of approach or model used, it cannot be stressed enough that clinical judgment is important. There needs to be an emphasis in the convergence and analysis of all information. • The California education code section 3030(j)(4) reads:

“No single score or product of scores, test, or procedure shall be used as the sole criterion for the decisions of the individualized education program team as to the pupil's eligibility for special education.”

Similarly, we must always keep in mind that tests do not evaluate, clinicians do. In my opinion, the PSW method does just this, moves the focus away from IQ scores and discrepancy formulas and places it on the whole child.

Page 42: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

42

Case Study: Records & Interviews

Lots-O is an 8 year-old student repeating the 2th grade. Concerns: basic reading skills, spelling, and speed of written work completion.

Records Review • phonemic awareness has been a constant challenge for Lots-O throughout the years. • reading and writing performance at the below basic level; • he had made progress on sight words, but extremely low reading fluency and little to no progress • he displayed some difficulty with spelling patterns, • struggled to finish written work, was slow to copy from the board and seemed to copy letter by letter. • his math performance was slightly better at the below basic to basic level. • EL student – English Language Development is early advanced/advanced (CELDT scores). Interviews- current teacher• displays strength in math-problem solving and basic addition facts. • can read simple and familiar text. • Weaknesses: not being able to decode unfamiliar texts and knowing only minimal sight words. • his writing often does not make sense and makes it hard to read. • has a great attitude, is willing to try everything, does not get easily discouraged and asks for help.

Page 43: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

43

Case Study: Observation

Observation• Was observed struggling to read words in a small group.• Low compared to his peers (3 correct vs. 10-15 correct). • A peer was heard saying to Lots-O “come on, that’s a kindergarten word.” • Able to follow directions like the rest of the class although he was sometimes slow

(e.g., last to follow directions on a few occasions)• During testing he demonstrated self-monitoring skills and prompt completion of

tasks that were easier for him. At times he required extra teaching of some tasks for him to understand what he is asked to do.

Page 44: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

44

Case Study: Cognitive TestingKaufman Assessment Battery for Children, 2nd edition (KABC-II)

Woodcock Johnson Tests of Cognitive Abilities, 3rd edition (WJ-III) – selected subtestsComprehensive Test of Phonological Processing (CTOPP-2)

CompositeSubtests

Standard Score

Percentile rank DescriptiveCategory

Long-Term Retrieval KABC-II 92 30th Average

Atlantis 7 16th Low Average

Rebus 10 50th Average

Delayed Recall KABC-II 83 13th Below

Atlantis Delayed 6 9th Below

Rebus Delayed 8 25th Average

Fluid Reasoning NI NI NI

Story Completion KABC-II 12 75th Average

Pattern Reasoning KABC-II 6 9th Below

Concept Formation WJ-III 79 8th Below

Short-Term Memory (Working) WJ-III 108 70th Average

Numbers Reversed 101 52nd Average

Auditory Working Memory 113 81st High Average

Short-Term Memory (Memory Span)

Memory for Words WJ-III 83 13th Below

Auditory Processing WJ-III

Auditory Attention 88 21st Low Average

Processing Speed WJ-III 65 1st Extremely Low

Visual Matching 63 1st Extremely Low

Decision Speed 70 2nd Below

Crystallized Intel./Knowledge KABC-II 100 50th Average

Verbal Knowledge 9 37th Average

Riddles 11 63rd Average

  Cluster/Factor

Scores

Standard Battery

Scores

Extended Battery

Scores

SS %ile SS %ile SS %ile

 

Cognitive Fluency

65 1st

      Retrieval Fluency 85 16th

      Decision Speed 70 2nd

     Rapid Picture Naming

73 3rd

Executive Processes

NI NIConcept Formation 79 8th Planning 101 53rd

      Pair Cancellation 94 34th

Mather, N., & Jaffe, L. (2002). Woodcock-Johnson III: Reports, Recommendations, and Strategies. New York: John Wiley & Sons

Composites/Subtests Standard Scores Percentile Rank Descriptive Phonological Awareness 69 2nd Below Elision 5 5th Below Blending Words 4 2nd Below Phoneme Isolation 6 9th Below Phonological Memory 76 5th Below Memory for Digits 6 9th Below Nonword Repetition 6 9th Below Rapid Naming 73 3rd Below Rapid Digit Naming 5 5th Below Rapid Letter Naming 6 9th Below

Page 45: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

45

Case Study: Academic TestingWoodcock Johnson Tests of Achievement, 3rd edition (WJ-III Ach.)

Lots-O’s spelling errors appear to be phonemic errors. In writing, he demonstrated poor organization, frequent erasing, and immature handwriting, although some meaningful sentences were noted. After a review of his responses on math calculations , an analysis of his errors suggests that Lots-O may inadvertently use the wrong operation (e.g., given two addition and two subtraction problems he may correctly perform addition operation for all four problems). Lots-O may be lacking the ability to switch strategies to attack this type of task as well as having difficulty following the process (keeping track of steps along the way, remembering steps or what the symbols mean). Additionally, his fluency with math facts is weak, only completing 13 simple calculations in three minutes.

Academic area Standard Score Broad Reading 61 (including basic reading skills) Broad written language 68 (including spelling) Academic Fluency 60 math reasoning 90 picture vocabulary 97 oral comprehension 89 math calculations 64 math fluency 72

Page 46: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

46

Case Study: Additional considerations

Lots-O’s pattern of performance and observations suggest weaknesses in some executive functioning skills. Although the WJ-III Executive Processes Cluster indicates some Average range performances, Lots-O demonstrated signs of difficulty with shifting attention to different aspects of a task, planning and organizing information presented, and selecting appropriate strategies. Lots-O may have a hard time trying a different strategy when a current one is no longer working, sifting through non-important information, etc. He also appears to rely heavily on context clues in order to be successful. This pattern was seen on the individual subtests of Long-Term Retrieval from the KABC-II and Fluid Reasoning of both the KABC-II and WJ-III. His processing speed ability and fluency in retrieval may also be further affected by executive functioning weaknesses. Executive functioning skills should be monitored and of consideration in future assessments.

Page 47: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

47

Sample PSW Eligibility Statement

The criterion for Specific Learning Disability state that a child must have a disorder in a psychological process that manifests itself in the imperfect ability to listen, think, speak, read, write, spell or perform math. The student then exhibits a pattern of strengths and weaknesses in performance, achievement, or both, relative to age, State-approved grade-level standards, or intellectual development, indicating the presence of a learning disability. There is an identified pattern of processing deficits in ______________’s _____________ (and weaknesses in ____________), which are associated with deficits in the academic achievement areas of ____________________________. Therefore, __________ meets the criteria for the Specific Learning Disability (SLD) handicapping condition as defined by the educational code.

Page 48: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

48

Sample PSW Eligibility Statement

There is an identified pattern of processing deficits in Lots-O’s phonological processing (auditory processing) which is associated with deficits in the academic achievement areas of basic reading skills and spelling, as well as processing deficits in processing speed/fluency in long-term memory retrieval which are associated with various aspects of academics, particularly with fluency and academic processes requiring automaticity (e.g., reading fluency, math facts, writing process). Therefore, Lots-O meets the criteria for the Specific Learning Disability (SLD) handicapping condition as defined by the educational code.

Page 49: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

49

Ask CASPQUESTION

A student referred for difficulty with reading. Got these standard scores using the KABC-II:Sequential-115Simultaneous-78Learning-73Planning-121Knowledge-87FCI-91

1. What does this mean and where do you go from here?

RESPONSELooks like your student may have a deficit in long-term memory retrieval (Learning), which may be reflected in her acquired knowledge (Knowledge) performance. This condition would impact reading, particularly at younger grades. I would also follow up with retrieval fluency type measures like Rapid Naming (e.g., CTOPP-2; which also has a measure of phonological awareness, another very important process for reading acquisition). I wouldn't worry about the Simultaneous score, there may be several reasons why it is low and it is not believed to be as important. Don’t forget to consider all your other information from your assessment. You should consider analyzing your info from a PSW model perspective.

Assessment Discussions- CASP 2015 Rodrigo Enciso

Page 50: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

50

Ask CASP (cont.)QUESTION (cont.)

A student referred for difficulty with reading. Got these standard scores using the KABC-II:Sequential-115Simultaneous-78Learning-73Planning-121Knowledge-87FCI-91

2. Due to the huge differences between the scores, what would you recommend to use as a valid measure of intelligence (IQ) to indicate at least average intellectual ability and compare to cognitive and academic weaknesses for SLD identification?

RESPONSEI would recommend getting results of each of the 7 broad abilities (CHC) and then using the Cross Battery Assessment software for PSW which helps you generate an Intellectual Ability Estimate (IA-e). Just looking at the limited scores obtained so far, the student likely has an overall average intellectual ability (somewhere between 90 and 110), and I would guess probably at least SS 100 using this procedure.There are other ways of doing it, although I am not a big fan of those other ways, but an alternative is to use the Planning (Fluid Reasoning) score as a cognitive measure since that is one of the two best indicators of intelligence (the other being Cristallized Intelligence/Knowledge). If the student demonstrates a below average range academic weakness (<85), you probably won't have a problem finding a significant discrepancy.

Assessment Discussions- CASP 2015 Rodrigo Enciso

Page 51: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

51

Some more case studies for review

• SAMPLE CROSS BATTERY ASSESSMENT RESULTS - Buzz

• Shady-Eighties (XBA)• Discrepancy in Private School (WJ-III)

Page 52: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

52

ConclusionThe PSW model offers a better way of understanding students’ strengths and weaknesses, and a more targeted way of assessing for SLD. PSW model is:

• approved to be used per education regulations • expert preferred • theoretically and research based • resolves issues and problematic outcomes of the traditional Discrepancy model

The “Discrepancy Model,” by its impracticality and inconsistent application, appears to have the unintended consequence of drawing the focus away from other very important aspects of learning and cognitive functioning. Therefore, it is essential to include in our assessment practices the knowledge gained from research on contemporary cognitive assessment. It will be wise to remember that test scores are only as good as the interpretation that we can generate from them. Thus, the PSW model lends its self to a more meaningful, practical, theory focused, as well as whole-child focused, assessment of cognitive abilities.

Page 53: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

53

References and RecommendationsFlanagan, D. P., McGrew, K. S. & Ortiz, S. O. (2000). The Wechsler intelligence scales and Gf-Gc theory: A

contemporary approach to interpretation. Needham Heights, MA: Allyn & Bacon.Flanagan, D. P., Ortiz, S. O., & Alfonso, V. C. (2013). Essentials of cross battery assessment (3rd ed.). Hoboken, NJ:

Wiley. Learning Disability Association of America (2010). White paper on evaluation, identification, and eligibility criteria

for students with specific learning disabilitiesMcGrew, K. S. (2005). The Cattell-Horn-Carroll theory of cognitive abilities: Past, present and future. In D. P.

Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues. New York, NY: Guilford Press.

Oregon School Psychologists Association (2008). SLD Pattern of Strengths and Weaknesses Committee: Technical assistance paper to the Oregon Department of Education.

Sattler, J.M. (2008). Assessment of Children: Cognitive Foundations (5th Ed.) San Diego, CA: Jerome M. Sattler, Publisher, Inc.

www.cross battery.comwww.iqscorner.com/search/label/CHC%20definitions?m=0http://assessingpsyche.wordpress.com/

Also, for great (and free) Steven Feifer webinars visit: Pearsonclinical.com -> Prek-16 education and special needs -> Training -> Webinar -> Cause and Effect http://www.pearsonclinical.com/education/webinars/topicallisting/cause-and-effect.html

Page 54: Assessment Considerations CASP Convention 2015 Rodrigo Enciso, M.A., LEP San Diego Unified School District 1 Assessment Discussions- CASP 2015 Rodrigo

Assessment Discussions- CASP 2015 Rodrigo Enciso

54

Contact Info

Rodrigo [email protected]