Laboratory Measures of ADHD

Adam B. Lewin

November 19, 2003

What are Laboratory Measures?

Laboratory Measures

• Techniques where behavior is observed under standardized conditions– Usually involving stimuli designed to evoke

the specific behavior of interest

• Excluded: – Naturalistic observations in unstandardized

conditions– Techniques designed primarily to elicit & observe

physiological responses.

(Frick, 2000 – J Clin Child Psychology)

Laboratory Measures

• Direct, systematic behavioral observations conducted in a clinic or research setting where efforts have been made to approximate more naturalistic situations (e.g., school, home, etc.).

(Barkley, 1991 – J Ab C Psychology)

• Often limited ecological validity

Laboratory & Performance-Based Measures

• A Broader Perspective:– Techniques suitable for use in

research settings

– Often limited implementation clinically• Time consuming• Non-diagnostic

– Despite clinical utility

Ecological Validity

• The extent to which predictions based on a laboratory measure can be extended to naturalistic setting

– e.g., Does the result of a laboratory task reflect actual attentional problems?

Ecological Validity: Barkley’s Criteria

• 1. Does the laboratory measure show differences between ADHD & control groups?

• 2. Correlations with other laboratory measures with well-established ecological validity

• 3. Sensitivity to experimental manipulations know to affect the criterion

• 4. Correlations with ecological criteria

Laboratory Measures

• 142 studies comparing ADHD & normal controls– 439 comparisons– Variety of tasks & neurocognitive tests

• Plethora of instruments employed– CPT, WISC-R, WCST, MMFT, Stroop

» (Rapport et al., 2000)

Laboratory Measures

• Reliable vs. unreliable instruments for detecting group differences– Reliable: CPT, WISC-R Coding & Arithmetic, Visual

recall tasks– Unreliable: Tapping, pegboard, Trails A, WISC-R Mazes

• Distinguishing characteristics:– Involve recall and/or recognition – Require use of the phonological loop– Pacing – experimenter controlled – Response stimulus not continuously displayed

» (Rapport et al., 2000)

MEASURES OF ATTENTION

Continuous Performance Tasks

• Originally designed to detect lapses in attention during seizures.

» (Rosvold et al., 1956)

• Usually computerized assessments of sustained attention

• Can be visual, auditory, numerals, characters, shapes

• Subject must respond to target embedded in a series of distracter stimuli

Continuous Performance Tasks • Three General Models:

» (Rapport, 1993)

• X-Version – respond to a target stimulus– Reverse X-Version: inhibit response to a

stimulus• AX-Version – respond to a target stimulus

only when it is preceded by a different target• Double Letter Version – respond only to an

immediately repeated stimulus» (Friedman et al., 1978)

X

Continuous Performance Tasks • Assessment of sustained attention:

– Number of Correct Responses– Errors of Omission (EO)

• Number of Target Stimuli Missed

• Believed to assess sustained attention & impulse control:– Errors of Commission (CE)

• Responding after a non-target stimulus» (Sostek, Buchsbaum, & Rapoport, 1980)

• Weakly correlated with error scores from the MFFT

• Both CEs and OEs significantly correlated with CPRS & CTRS Hyp & Inattn scales

» (Barkley, 1991)

Conners’ CPT

• Computer-assisted assessment of attention

• 14 minutes

• X & Reverse X Versions; AX Version

• 6 Trial Blocks; 3 Sub-blocks per trial• 20 Trials Each• Interstimulus interval varies from 1, 2 or 4 seconds

Conners’ CPT

• Numerous output data:– Correct responses, OEs, CEs, reaction times,

Index score– CPT-II: Clinical Confidence Interval

• Norms for general population and children 4-18 diagnosed with ADHD

• Low false positives and negatives (<10-15%)• Practice effects are minimal• Sensitive to pharmacological treatment

changes

Conners’ CPT• Some Advantages/Disadvantages

– Avoids false negatives by frequent target presentation– Chance of impulsive errors maximized due to the continuous

level of responding– Questionable ecological validity

• Correlates with analogue measures of attention (.25-.35)

• Relates to parent & teacher ratings of inattention & hyperactivity– CPRS/CTRS & CBCL

» (Barkely 1991)

Conners’ CPT II• Normative sample of 2,686 clinical and nonclinical

subjects. • T-Scores & the following classifications:

– markedly, moderately or mildly atypical, within the average range, and good or very good performance.

• New confidence index that is the percentage out of 100 clients that would be correctly classified based on a profile.

• CPT-II provides an overall index, for research comparisons with the CPT IAges 6 years and older– (Kiddie CPT for ages 4-5).

TOVA

• Test of Variables of Attention (TOVA)» (Greenberg & Waldman, 1993)

• Two 11-minute computerized tasks (one for children under age 5)

• Easily discriminated visual stimuli– Square with a small square adjacent either to the

top (target) or bottom

• Attempts to eliminate confounds due to learning difficulties

TOVA

• First task presents the target infrequently (1:3 ½ )– Designed to elicit boredom & thus measure

sustained attention

• Second task presents the target frequently (3 ½ :1)– Designed to measure impulsivity

• Normative data in 2-year intervals for children 4-19 (10 year intervals for adults)

• Diagnostic utility not well documented

Auditory CPT

• Tape of a 96-word list, of 20 different monosyllabic words, read 6 times

• Target word is “dog”

• 20 times per 96-word trial

• Respond by giving “thumbs up”

• Test-retest only .67-.84

• Scoring is difficult

Gordon Diagnostic System• Gordon Diagnostic System (GDS)

» (Gordon, 1979)

• Portable machine• Visual

– Vigilance (numerical AX task)– Distractibility (Random numbers flash in proximity to the

target)– Delay (points awarded for delaying response at least 6

seconds)• Measure of response inhibition• Correlates with hyperactivity ratings by parent & teachers• Not proven sensitive to medication effects

GDS

• Auditory– Vigilance Task: Subject responds to

numbers that are heard instead of seen– Interference Task: Random number through

the headphones. The subject performs on the standard Vigilance (or Distractibility) tasks while having to contend with the confusing auditory input.

IVA

• Intermediate Visual & Auditory CPT (IVA)

• Half the targets are visual (the characters are a "1" and a "2“) and half are presented audibly through the computer's speaker

• 13 minutes

Other CPT Tasks

• Children’s Checking Task (CCT)» (Margolis, 1972)

– Paper & Pencil– “Cancellation Task”– Mark numbers listed in rows on a page as they are

read on a recording – Circle discrepancies between the list & recording

• 30 minutes; Scores include OE & CE• Strong correlations with other measures of

attention• Better ecological validity than other CPTs

Other CPT Tasks• Matching Familiar Figures Test (MFFT)

» (Kagan, 1966)

• Measure of attention and impulse control– Match-to-sample task– Identify the identical matching target picture from an array of six

similar stimuli– 12 or 20 trial versions– Measures of response latency and errors

• Fails to differentiate ADHD from controls & medication effects

• Adolescent norms unavailable• Not recommended for clinical use

TEA-Ch

• Test of Everyday Attention for Children (TEA-Ch; Manly et al., 1999)

• Norms for ages 6-16• 9 Game-like subtests• 3 Domains

– Selective Attention (2 subtests)– Sustained Attention (5 subtests)– Attentional control/shift (2 subtests)

• Approximately 2 hours to complete; subtests can be administered individually

TEA-Ch

• Children with ADHD show significant impairment on sustained attention & attentional control tasks (compared to clinical controls)

• Differences on selective attention tasks not significant.

» Heaton et al., 2002

Neuropsychological Tests

Neuropsychological Measures

• Stroop Word-Color– Timed test measuring the ability to inhibit or suppress

automatic responses– High % of false negatives (53%)

» (Barkely & Grodzinsky, 1994)

• Trail Making Test– Trails B – Attentional Shift– Very high false negatives (80-82%)– Overall classification <54% correct.

» (Barkely et al., 1992; 1994)

• Mixed results • Not consistent in identifying group differences

Neuropsychological Measures

• Wisconsin Card Sorting Task (WCST)– Computerized/manual administration– Participant must correctly sort a series of colored

geometric shapes according to an set of rules unknown to the subject.

– After each “sort,” the only feedback is correct/incorrect– Rules must be deduced from this feedback– Rules change on each successive trial– Requires an ability to shift attention

» (Mirsky et al., 1991; Heaton et al., 1993)

– Not recommended for diagnostic use. • False negative 61-89%;

MEASURES OF ACTIVITYMEASURES OF ACTIVITY

Measuring Activity Level

• Two primary classifications:1. Binary Devices – Respond in an “all or

nothing” manner when movement exceeds a threshold value

2. Proportional Devices – Measure motor activity in direct proportion to the magnitude of movement.

(Tyron, 1984)

Measuring Activity Level

Binary Devices• Mercury Switches

– Position change sensors– The “wiggle chair”

– Not consistent in identifying group differences

– Not related to parent hyperactivity ratings

• Pedometers– Activated by the impact of the foot & ground

• Photoelectric cells

Actometers Proportional Devices• Actometers – ankle or wrist

– Modified self-winding watches– Movements of the limb corresponds to

movement of the watch’s hands– Sensitive to stimulant drug effects– Laboratory actometer ratings not significantly

related to parent ratings of hyperactivity at home

– Ankle actometers relate to CPT CEs.(.37)» (Barkely et al., 1975; Ullman et al., 1978)

Parent Rating Scales

Rating Scales• Should address aspects of the following:

– core features of ADHD– symptom severity and development– level of impairment– comorbid conditions

• Advantages:– Standardized– Decreases subjectivity– Cost-effective method for multiple informants– Can be completed prior to evaluation– Access to infrequently displayed behaviors that may be missed in

observation periods» (Anastopoulos 2001)

Rating Scales

• Potential limitations:– Assume informant is familiar enough with the

subject to accurately complete the measure– The informant must be able to understand the

questions– Adult psychopathology may distort parent

perceptions of the child– Parental or teacher tolerance of behavior may

influence ratings.» (Anastopoulos 2001; Sattler 2002)

Rating Scales• Conners’ Rating Scales

– Parent & Teacher revised versions• Children 3-17• Parent 80 items (27 on the Short-form)• Teacher 59 items (28 on the Short form)• Short form has limited scales – focus on ADHD/ODD

symptoms– Self-report for adolescents 12-17

• Conners-Wells Adolescent Self Rating Scale• 87-item (27 on the short form)

– Male & female norms in 3 year intervals– Rating on a 4-point scale

Rating Scales• Conners’ Rating Scales• Ratings based on the previous month• Excellent psychometric properties• Simple comparisons between teacher & parent

versions• ADHD Index (parent form): 12 items• Hyperactivity Scale moderately related to total

hyperactivity score during analogue observation» Barkely 1991

Rating Scales

• Behavioral Assessment Scale for Children (BASC)

» (Reynolds & Kamphaus, 1992)

• Parent, teacher, and self ratings scales; Student observation system

• Ratings over the previous 3 months• Preschool, child and adolescent versions• 130 items on a 4 point scale (parent & teacher)• 170 True/False items for the self-report form

Rating Scales

• BASC

• Scales include: adaptability, aggression, anxiety, attention problems, atypicality, conduct problems, depression, hyperactivity, leadership, learning problems, social skills, somatization, study skills, & withdrawal

• Provides index of Adaptive Skills

Rating Scales

• BASC• Excellent psychometrics; correlates highly

with the CBCL & Conners’; moderately with the PIC.

• More predictive of ADHD status than the CBCL

• 88% of the sample correctly identified as ADHD (using the Attention subscale)

» (Ostrander et al., 1998)

Rating Scales• Achenbach Child Behavior Checklist (CBCL)• Recently revised with new normative sample• Parent, teacher, & self-rating form of behaviors

over the past 6 months• Externalizing, Internalizing, and Total Problem

Scale– Attention & hyperactivity profiles

• Perhaps the most frequently used broad-banded measure in research

» Anastopoulos & Shelton, 2001)

Rating Scales

• DSM-IV SNAP-IV ADHD Checklist» (Swanson, 1992)

• For parents, teachers, caregivers• DSM-IV symptoms on a 4 point rating scale• Does not include:

– rating on impairment in function– information of symptoms across setting – symptom chronicity ratings – symptom onset data

Rating Scales

• SNAP-IV• Sample items:

– Does not seem to listen when spoken to– Often “on the go” or acts as if “driven by a

motor”– Often has difficulty waiting for a turn

• Research screener for ADHD• Adult version is available

Rating Scales

• Personality Inventory for Children– 280/420 item parent rating form (True/False)– Lacks an inattentive subscale

• Devereux Scales of Mental Disorders– Caregiver rating form– 5 point scale based on the previous month– Only 4 inattention items, 3 impulsivity items & 3

hyperactivity items– No hyperactivity or impulsivity subscale– Modest psychometrics

Rating Scales

• Behavior Rating Inventory for Executive Function (BRIEF; Gioia et al., 1996)

• Designed to assess several aspects of executive functioning– Inhibition, Shift, Emotional Control, Working

Memory, Planning/Organization, Organization of materials, Monitoring

• For children ages 5-18

• 86 items; Parent & Teacher versions

Rating Scales

• BRIEF• May help differentiate ADHD subtypes

» (Barkely, 1997)

• May be useful in identifying difficulites associated with ADHD (e.g. poor behavioral initiation; planning, organization)

• Working Memory & Inhibit Scales moderately predictive of ADHD diagnosis (Predominately Inattentive or Combined Type)

Structured Interviews

Structured Interviews• Diagnostic Interview Schedule for Children IV (DISC-IV)

» NIMH 1997

• Developed for use in epidemiological studies of childhood behavioral disorders

• 6 major sections; 24 diagnostic modules– 30 DSM-IV diagnoses may be generated

• Graded question format– Questions on symptom onset, severity, duration– Stem questions asked of everyone– Contingent questions– Primarily responded to by yes or no

Structured Interviews

• DISC-IV• Administered to parents of children 6-17

– Youth version for ages 9-17

• 45-90 minutes administration time• Clinician administered, computer entered• Good reliability & validity• Gathers information on ADHD symptoms &

comorbid disorders• May result in an overestimation of psychiatric

symptoms

Structured Interviews• Diagnostic Interview for Children & Adolescents-

IV (DICA-IV)» (Reich et al., 1996)

• Screening measure for psychiatric disorders• Revised to address major child and adolescent DSM-

IV diagnoses• Clinician administered, computer entered• Parents of children age 6-17 years• 28 Diagnostic Categories; 5-20 minutes each• 6 “High Risk” areas, e.g., CD, alcohol & street drug

use, MDD, suicide, PTSD

Structured Interviews

• DICA-IV• Two child forms available• Allows for probing beyond simple yes/no format• Requires more clinical skill & expertise than the

DISC to administer• Clinicians have the option to not administer

certain categories• Critical item screen

Structured Interviews

• Schedule for Affective Disorders and Schizophrenia (K-SADS)

» (Puig-Antich & Chambers, 1978)

• Semi-structured interview for parents and children• Children ages 6-17• Typically takes 30-90 minutes• K-SADS-PL – version most useful for ADHD

assessment

» (Kaufman et al., 1997)

Structured Interviews

• K-SADS• Unstructured introductory interview

– Demographics, present complaints, development, prior psychological treatment, family/peer relationships, academic functioning & hobbies.

• Screening Interview (82 symptoms divided into 20 diagnostic areas)– Rate symptoms with regard to current & most severe

past occurrence– 27-28 questions on ADHD section

• Diagnoses based on both parent & child report

Structured Interviews

• K-SADS– Reliability good for ADHD (.63 for present

diagnosis; .55 for lifetime diagnosis) but slightly less than structured interviews.

• Assesses age of onset, impairment, cross-situation criteria, academic achievement.

• No absolute guidelines for resolving informant discrepancies

• Requires substantial training relative the DISC & DICA

Behavioral Observation

Systems

Behavioral Observation

• Why observe?– Parent evaluations may be biased due to

distress or other psychological factors– Intense behaviors may be perceived as

more frequent– Increased objectivity

• Ecological validity vs. experimental control

Behavioral Observation• Direct observation of a child’s behavior

– Analogue/laboratory settings– Naturalistic observation

• “Checklist” vs. Anecdotal Observation– ABC data is useful in understanding overall behavioral patterns– More intensive for the observer

• Data collection– Interval – Frequency– Duration– Latency

•

Behavioral Observation

• Requires clear operational definitions of target behaviors– Objective, observable characteristics– Replicable (clear; unambiguous)– Discriminating information; eliminate judgment

» (Baer, Wolf & Risley, 1968)

• Observer training and agreement

Behavioral Observation

• BASC Student Observation System• Coding of classroom behaviors• 15 minute observation period

– Checklist of 65 adaptive and maladaptive behaviors

– Occurrence of behaviors during 30 3-second intervals

– Additional observations (e.g .teacher reaction)

Behavioral Observation

• BASC Student Observation System– Good for initial assessment– No Normative Data– Brief sampling window– Difficult to examine behavioral contingencies

Behavioral Observation

• ADHD Behavioral Coding System» (Barkely; 1990)

• Laboratory-based• Child is observed completing an academic

task– Instructed to work on the assignment– Stay seated– Do not touch toys & stimuli in the room

Behavioral Observation

• ADHD Behavioral Coding System– 15 minute observation– Off-task behavior, fidgeting, leaving the seat,

vocalizing, or playing with the toys – 30-second interval recording– Can be used to code behavior during the CPT

Behavioral Observation

• ADHD Behavioral Coding System– Good inter-observer agreement (.77-.85)– Children with ADHD tend to score higher– Sensitive to medication effects– Lacks normative data– Questionable ecological validity– Interval/frequency recording

General Conclusions

- When incorporating laboratory measures into clinical assessments, keep in mind ecological validity.- In case of disagreement, consider the most

ecologically valid source of information

- Diagnosis and treatment planning should not be based on a single piece of information.

END