analyzing intervention efficacy: revealing the nature of special education kenneth a. kavale...
TRANSCRIPT
ANALYZING INTERVENTION EFFICACY: REVEALING THE
NATURE OF SPECIAL EDUCATION
Kenneth A. Kavale
Distinguished Professor
Regent University
SPECIAL EDUCATION Is special education special?
Perceptions
Optimism
Pessimism
SPECIAL EDUCATIONSpecial Education
Meanings
Special – teaching students with special needs
Special – using special instruction
Special Education
Variable Outcomes
EFFICACY OF SPECIAL EDUCATIONMethods – Do they work?
Scientific Method
Empirical Evidence
Usable Knowledge
Research Synthesis
Methods
Narrative
Box-Score
Meta-Analysis
META-ANALYSISQuantitative Research Synthesis
RigorousSystematicMethods
Problem FormulationSampling
Study Classification (Coding)Data AnalysisInterpretation
META-ANALYSISEffect Size Statistic
ES = ME – MC
SDC
META-ANALYSISInterpretation
Z-score
-Percentile Equivalent-
Binomial Effect Size Display
-Practical Significance-
META-ANALYSISCommon Language Effect Size
-Research Significance-
Power Analysis
-Small (.20), Medium (.50), Large (.80>)-
NATURE OF SPECIAL EDUCATION
Definition
“Specially designed instruction…to meet the unique needs of a child with a disability”
Features
Unique
Exclusive
GOAL OF SPECIAL EDUCATION
Medical Model
Goal: Correcting or reversing altered cognitive processes
Itard
Process Training
GOAL OF SPECIAL EDUCATION“Process training is, in fact, one of the oldest forms of
education and, despite periodic discontinuities in its practice, it has continued unabated into our own day”
L. MannOn the trail of process
SupportHistoricalClinical
Philosophical
PSYCHOLINGUISTIC TRAINING
Samuel A. Kirk
Illinois Test of Psycholinguistic Abilities (ITPA)Research Evaluations
Hammill and Larsen (1974)
“the idea that psycholinguistic constructs, as measured by the ITPA, can, in fact, be trained by existing techniques
remains nonvalidated”
PSYCHOLINGUISTIC TRAININGMinskoff (1975)
Skepticism about psycholinguistic “can be dangerous if it leads to the abolition of training methods that may be beneficial”
Newcomer, Larsen, and Hammill (1975)
“literature raises doubts regarding the efficacy of presently available Kirk-Osgood psycholinguistic training programs”
PSYCHOLINGUISTIC TRAININGLund, Foster, and McCall-Perez (1978)
Mixed findings making it “not logical to conclude either that all studies in psycholinguistic training are effective or that all
studies in psycholinguistic training are not effective”
Hammill and Larsen (1978)“the cumulative results…failed to demonstrate that
psycholinguistic training has values”
What is really known about the efficacy of psycholinguistic training?
META-ANALYSIS AND THE EFFICACY OF SPECIAL EDUCATION
Psycholinguistic Training
ES = .39
Psycholinguistic Training by ITPA
ITPA Subtest Mean Effect Size
Percentile Equivalent
Power Rating BESD
From (%) To (%)
Auditory Reception .21 58 Small 45 55
Visual Reception .21 58 Small 45 55
Auditory Association .44 67 Small-Medium 39 61
Visual Association .39 65 Small-Medium 40 60
Verbal Expression .63 74 Medium 35 65
Manual Expression .54 71 Medium 37 63
Grammatic Closure .30 62 Small 42 58
Visual Closure .48 68 Medium 38 62
Auditory Memory .32 63 Small 42 58
Visual Memory .27 61 Small 43 57
META-ANALYSIS AND THE EFFICACY OF SPECIAL EDUCATION
Verbal Expression = better than 6 months of general education instruction (ES = .50)
Are there more efficacious methods of teaching language?
EFFICACY OF PROCESS TRAININGEffectiveness of Process Training
Method Mean Effect Size
Percentile Equivalent
Power Rating
Irlen lenses - .02 49 Negative
Perceptual-motor .08 53 Negligible
Diet (Feingold) .12 55 Small
Modality Instruction .14 56 Small
Social Skills .23 64 Small
Psycholinguistic .39 65 Small-Medium
EFFICACY OF PROCESS TRAININGPerceptual-Motor Training Programs
Program Mean Effect Size Percentile Equivalent
BESD
From (%) To (%)
Kephart (1960) Perceptual-motor
.06 52 48 52
Frostig (1964)
Visual-perceptual
.10 54 47 53
Cratty (1969)
Motor learning
.11 54 47 53
Getman (1965) Visuomotor
.12 55 47 53
Barsch (1967) Movigenics
.16 56 46 54
Delacato (1959) Neurological patterning
.16 56 46 54
EFFICACY OF PROCESS TRAINING PROGRAMS
Special Education Hall of Fame
ES = .12
EFFICACY OF PROCESS TRAININGModality-Matched Instruction
Modality Mean Effect Size
Percentile Equivalent
BESD
From (%) To (%)
Auditory .18 57 45 55
Visual .09 54 47 53
Kinesthetic .18 57 45 55
ES = .14
Learning = Substance over Style
EFFICACY OF PROCESS TRAINING
Process Training
ES = .16
No empirical support
EFFICACY OF PROCESS TRAININGWhy is process training ineffective?
ProblemHypothetical Constructs
Product vs. Process
PositiveClinical
NegativeEmpirical
EFFICACY OF PROCESS TRAININGBelief
“You cannot kill it. It simply bides its time in exile after being dislodged by one of history’s periodic attacks upon it and then returns, wearing disguises or carrying new noms de
plume…but consisting of the same old ideas, doing business much in the same old way.”
L. MannOn the trail of process
Brain Gym
CREATING EFFECTIVE SPECIAL EDUCATION
Process TrainingPathology Model
“Curing”
Instructional Imbalance ModelMismatch
Effective SchoolsClearly defined curriculum
Clear instructional objectivesFocused classroom instruction
Monitor student progressStrong instructional leadership
CREATING EFFECTIVE SPECIAL EDUCATION
Learning Process Model
Process-Product Paradigm
Teacher Behavior – Student Outcomes
Best Practice
All students can achieve
Active engagement
Organized classrooms
Emphasis on basic skill instruction
Meaningful and interesting learning
EFFECTIVE SPECIAL EDUCATIONAL PRACTICE
Modifying the Delivery of Instruction
Effective Instructional Practices
Practice Mean Effect Size
Common Language Effect Size
BESD
From (%) To (%)
Mnemonic Instruction
1.62 .87 18 82
Self-Monitoring 1.36 .83 22 78
Reinforcement 1.17 .80 25 75
Self-Questioning 1.16 .79 25 75
Drill & Practice .99 .76 28 72
Strategy Instruction .98 .75 28 72
Feedback .97 .75 28 72
Direct Instruction .93 .75 29 71
Visual Displays .90 .74 29 71
Computer-Assisted Instruction
.87 .73 30 70
EFFECTIVE SPECIAL EDUCATIONAL PRACTICE
Effective Instructional Practices (Table con’t)
Practice Mean Effect Size
Common Language Effect Size
BESD
From (%) To (%)
Repeated Reading .76 .71 32 68
Error Correction .72 .70 33 67
Early Intervention .71 .70 33 60
Formative Evaluation .70 .69 33 67
Peer Mediation .64 .67 35 65
Diagnostic-Prescriptive Teaching
.64 .67 35 65
Peer Tutoring .62 .67 35 65
Positive Class Morale
.60 .66 36 64
Grouping .43 .62 40 60
Cooperative Learning .40 .61 40 60
Increased Time .38 .61 41 59
EFFECTIVE SPECIAL EDUCATIONAL PRACTICE
Special Education
General Education Teaching – Learning Model
Adapted and Modified for the Purposes of Special Education
EFFECTIVE SPECIAL EDUCATION RELATED SERVICES
Effective Special Education Related Services and Activities
Average ES = .65
Subject Area Mean Effect Size
Common Language Effect Size
BESD
From (%) To (%)
Memory Training 1.12 .79 25 75
Prereferral 1.10 .78 26 74
Cognitive Behavior Modification
.74 .70 32 68
Stimulant Medication .62 .67 35 65
Counseling .60 .66 35 65
Consultation .55 .65 36 64
Rational-Emotive Therapy .50 .64 38 62
Attribution Training .43 .62 39 61
Placement .12 .53 47 53
EFFECTIVE RELATED SERVICESPlacementES = .12
Where vs. What
PrereferralES = 1.10
RTIEffective as Intervention
EFFECTIVE SPECIAL EDUCATION INSTRUCTIONDelivery of Instruction Modified
Effective Special Education Instruction
ENHANCED ACADEMIC PERFORMANCE
Subject Area Mean Effect Size
Percentile Equivalent
Common Language Effect Size
BESD
From (%) To (%)
Handwriting 1.32 91 .82 22 78
Oral Reading 1.31 90 .82 22 78
Language 1.27 90 .82 23 77
Reading Comprehension
1.04 85 .77 27 73
Word Recognition .98 84 .75 28 72
Narrative Writing .97 83 .75 28 72
Math .96 83 .75 28 72
Spelling .87 81 .73 30 70
Vocabulary .85 80 .73 30 70
EFFECTIVE SPECIAL EDUCATION INSTRUCTION
ES = 1.06
One Year of General Education Instruction
ES = 1.00
EFFECTIVE SPECIAL EDUCATION INSTRUCTION
Reading Comprehension and “Real” Effects
Meta-Analysis X
ES = 1.13
Meta-Analysis Y
ES = .98
Real Effect = 1.05
EFFECTIVE SPECIAL EDUCATION INSTRUCTION
Techniques
Metacognition (e.g., self-questioning, self-monitoring)ES = 1.63 and 1.33
Text enhancement (e.g., advanced organizers, mnemonics)ES = 1.09 and .92
Skill training (e.g., vocabulary, repeated reading)ES = .79 and .69
COMPARING SPECIAL EDUCATIONDIRECT INSTRUCTION
ES = .93
Modality-matched instruction
ES = .14
DI is 6.5 times more effective
DI
Over 11 months credit in achievement
EVALUATING SPECIAL EDUCATION
Special Education
Increasing Efficacy
Why?
SPECIAL educationMega ES = .15
special EDUCATIONMega ES = .89
EVALUATING SPECIAL EDUCATION
special EDUCATION 6 times more effective than SPECIAL education
SPECIAL education = 6 percent advantage/56th %-ilespecial EDUCATION = 31 percent advantage/81st %-ile
ProblemSPECIAL education
25% negative ESspecial EDUCATION
0% negative ES
SPECIAL education not necessary
EFFECTIVE SPECIAL EDUCATIONSpecial Education
Variable and Unpredictable
Psycholinguistic Training
Theoretical Expectation
(ES ± SD)
(.39 ± .54)
Range = -.15 to .93 (negative to large)
Where will ES fall?
EFFECTIVE SPECIAL EDUCATION
SPECIAL education
ES = .15
Theoretical Expectation
(.15 ± .48)
More variable than effective
(-.33 to .63)
Medium ES possible but also increased risk
EFFECTIVE SPECIAL EDUCATION
special EDUCATION
ES = .89
Theoretical Expression
(.89 ± .87)
More effective than variable
(.02 to 1.76)
May not work but possibility twice as effective
EFFECTIVE SPECIAL EDUCATIONspecial EDUCATION
Reduces risk (no negative ES)but is
CAPRICIOUS(variable, unpredictable, indeterminate)
Special Education should not be
Prescriptive(Do A when X or Y)
OPTIONS(Try A when X or Y
orTry B when X or Y)
EFFECTIVE SPECIAL EDUCATION
SPECIAL education when transformed into
special EDUCATION
now includes
Instructional Validity
EVIDENCED-BASED PRACTICE
EFFECTIVE SPECIAL EDUCATIONWhy is evidenced-based practice not used?
Tradition“We have always used it”
History“It has worked before”
BandwagonRhetoric, no evidence
BeliefStrong conviction about the truth
EFFECTIVE SPECIAL EDUCATIONWhen evidenced-based practice is not used
RESEARCH-TO-PRACTICE GAPResearch findings “are embraced by some, ignored
by others, and modified to suit the routines and preferences of still others”
Gersten, Vaughn, Deshler, & Schiller
1997
EFFECTIVE SPECIAL EDUCATIONResearch-to-Practice Gap
SUSTAINABILITYFailure to use instructional practices supported by evidence
INDIVIDUALIZED INSTRUCTIONBarriers
UNCERTAINTYProgram may not work
…andRISK
Program may produce negative outcomes
EFFECTIVE SPECIAL EDUCATIONWho chooses options?
TEACHERS
Teachers
Dogmatic beliefs
replaced by
Rational choices (“what works”)
EFFECTIVE SPECIAL EDUCATION
Special Education
Science
(theoretical and empirical knowledge)
+
Art
(interpretation for initiating action)
EFFECTIVE SPECIAL EDUCATION
Teacher’s Goal
State of the Art
(what has been shown to be possible)
evolves into
State of Practice
(current ways of providing instruction)
TEACHER ATTRIBUTES(Evidenced Based)
The special education practitioner must possess the:
Energy of a hurricane
Efficiency of a computer
Adaptability of a chameleon
Courage of Hercules
Patience of Job
Wisdom of Solomon
Persistence of the Devil
EFFECTIVE SPECIAL EDUCATIONConclusion
“Special education practitioners will need to go beyond the scientific basis of their work…and must be mediated through the teacher’s own creative rendering of best practice…because quality
education for special education students will always be based on the artful application of science”
Kavale & Forness
1999