Journal of Communication Disorders 51 (2014) 51–63

Contents lists available at ScienceDirect

Journal of Communication Disorders

Construct-related validity of the TOCS measures: Comparison

of intelligibility and speaking rate scores in children with andwithout speech disorders

Megan M. Hodge *, Carrie L. Gotzke

Department of Communication Sciences and Disorders, University of Alberta, Edmonton, AB, Canada T6G2G4

A R T I C L E I N F O

Article history:

Received 14 April 2014

Received in revised form 8 June 2014

Accepted 30 June 2014

Available online 12 July 2014

Keywords:

Intelligibility

Children

Cerebral palsy

Dysarthria

Speech sound disorders

Speaking rate

A B S T R A C T

This study evaluated construct-related validity of the Test of Children’s Speech (TOCS).

Intelligibility scores obtained using open-set word identification tasks (orthographic

transcription) for the TOCS word and sentence tests and rate scores for the TOCS sentence

test (words per minute or WPM and intelligible words per minute or IWPM) were

compared for a group of 15 adults (18–30 years of age) with normal speech production and

three groups of children: 48 3–6 year-olds with typical speech development and

neurological histories (TDS), 48 3–6 year-olds with a speech sound disorder of unknown

origin and no identified neurological impairment (SSD-UNK), and 22 3–10 year-olds with

dysarthria and cerebral palsy (DYS). As expected, mean intelligibility scores and rates

increased with age in the TDS group. However, word test intelligibility, WPM and IWPM

scores for the 6 year-olds in the TDS group were significantly lower than those for the adults.

The DYS group had significantly lower word and sentence test intelligibility and WPM and

IWPM scores than the TDS and SSD-UNK groups. Compared to the TDS group, the SSD-UNK

group also had significantly lower intelligibility scores for the word and sentence tests, and

significantly lower IWPM, but not WPM scores on the sentence test. The results support the

construct-related validity of TOCS as a tool for obtaining intelligibility and rate scores that are

sensitive to group differences in 3–6 year-old children, with and without speech sound

disorders, and to 3+ year-old children with speech disorders, with and without dysarthria.

Learning outcomes: Readers will describe the word and sentence intelligibility and

speaking rate performance of children with typically developing speech at age levels of 3,

4, 5 and 6 years, as measured by the Test of Children’s Speech, and how these compare with

adult speakers and two groups of children with speech disorders. They will also recognize

what measures on this test differentiate children with speech sound disorders of unknown

origin from children with cerebral palsy and dysarthria.

� 2014 Elsevier Inc. All rights reserved.

1. Introduction

The ability to produce spoken language that is understandable to others is typically achieved by age four years; by this agemost children can use speech successfully in their daily communication interactions. For example, Coplan and Gleason (1988)

* Corresponding author at: Department of Communication Sciences and Disorders, University of Alberta, Edmonton, AB, Canada T6G2G4.

Tel.: +1 780 492 0833.

E-mail addresses: mhodge@ualberta.ca, megan.hodge@ualberta.ca (M.M. Hodge), cgotzke@ualberta.ca (C.L. Gotzke).

http://dx.doi.org/10.1016/j.jcomdis.2014.06.007

0021-9924/� 2014 Elsevier Inc. All rights reserved.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6352

reported the following ages at which 90% of parents assigned a given level of intelligibility to their child when asked‘‘How much of your child’s speech can a stranger understand?’’ 50% at 22 months, 75% at 37 months, and 100% at 48 months.These findings are consistent with the guideline provided by Flipsen (2006), who suggested the following formula: age(years)/4 � 100 = % of conversational speech that is intelligible to unfamiliar adults (i.e., age 1 year = 25%, 2 years = 50%intelligible, 3 years = 75% and 4 years = 100%). Weiss (1982) reported the following developmental milestones for speechintelligibility, based on the mean of word identification scores obtained from a 200-word orthographic transcription of aspontaneous speech sample and from a set of 25 single words, when audio recordings of the speech samples and wordswere judged by an unfamiliar adult: 25% by 18 months, 50% by 2 years, 64% by 2.5 years, 80% by 3 years, 92% by 3.5 years and100% by 4 years. More recently, Rice et al. (2010) reported the percentage of intelligible utterances calculated from spontaneousspeech samples using the SALT software (Miller & Chapman, 1991) for 176 children in their control group (unaffected byspecific language impairment) at 6-month age intervals from 2.5 to 9.0 years. As expected, mean scores increased with age;the lowest score (74%) was reported for the 2.6–2.11 age interval and the highest score (94%) was reported for the 8.6–8.11 ageinterval. Children aged 3.6 to 4.11 had scores between 84% and 87% and children 5.0 and older had scores greater than 90%.

Children with moderate or more severe developmental speech delays and/or disorders often have reduced intelligibilitythat may persist beyond the preschool years. Gordon-Brannan and Hodson (2000) reported intelligibility scores for imitatedsingle words on the Children’s Speech Intelligibility Measure (CSIM) (Wilcox & Morris, 1999), a sentence repetition task, and a100-word spontaneous speech sample for children between 4.0 and 5.6 years with varying levels of phonological proficiency(adult-like to severe involvement). No child in their study had known neuromotor involvement that might affect speechproduction and all children had age-appropriate receptive language. Children in the ‘‘adult-like’’ subgroup obtained highmean scores in the three conditions (84% on the CSIM, 96% on the imitated sentences, 95% on the spontaneous sample). Meanscores of the children with the most severe phonological impairment were 35% on the CSIM, 44% on the imitated sentences,and 48% on the spontaneous sample. Flipsen (2006) compared several different versions of the Intelligibility Index (Shriberg,Austin, Lewis, McSweeny, & Wilson, 1997), based on spontaneous speech samples, for children between 3.0 and 8.11 withnormal (or normalized speech) and with speech delay. All age means in the group with normal speech were above 95.5% andincreased to a maximum of 99.5%, as age increased. Age means in the group with speech delay ranged from 80.8% for the 3year-olds to 95.3% for the 8 year-olds.

Children with cerebral palsy (CP) and developmental dysarthria have a chronic neuromuscular impairment underlyingtheir speech disorder that often delays early speech learning and reduces speech intelligibility. CP is defined as a group ofdisorders of movement and posture that limit activity and are attributed to non-progressive disturbances in the fetal orinfant brain (Bax, Goldstein, Rosenbaum, Leviton, & Paneth, 2005). The impact of CP on a child’s motor control for speechproduction may impair the strength, speed, accuracy, coordination and endurance of the muscle groups used to speak(Workinger, 2005). These impairments constrain development of differentiated, precise, dynamic actions of the oralarticulators and their coordination with the respiratory–phonatory system to produce clear, efficient speech patterns, whichcan limit the success of spoken interactions (Hodge, 2010; Pennington, 2008). Pennington, Roelant, Thompson, Robson,Steen, and Miller (2013) reported intelligibility scores for the CSIM and word identification of connected speech samples(combination of imitated and spontaneous utterances) for 15 five to eleven year-old children with CP who underwent anexperimental speech therapy procedure. Mean intelligibility scores obtained six weeks prior to treatment, based on threeunfamiliar listeners, were 46.8% (SD 18.7) for the CSIM and 43.7% (SD 24.7) for the connected speech sample. These resultsindicate significant intelligibility deficits for the group as a whole, compared to what is expected for similar-aged peerswithout speech disorders, and are most similar to the mean scores of the children with the most severe phonologicalimpairment (without dysarthria) reported by Gordon-Brannan and Hodson (2000).

Speech intelligibility measures are desirable to estimate the severity of a child’s speech disorder, which is used todetermine the need for and nature of intervention and to potentially discriminate between children with and without speechdisability (Hodson, 2004). Increasing intelligibility is often a primary therapeutic goal for young children with speechdisorders, regardless of the underlying reason for the intelligibility deficit; intelligibility has been identified an importantoutcome measure to evaluate change in a child’s ability to make his or her speech understandable to others, (e.g., McLeod,Harrison, & McCormack, 2012; Pennington, Miller, Robson, & Steen, 2010; Pennington et al., 2013). However, assessment ofintelligibility remains a challenge because of differences in how narrowly or broadly it is defined and the multiple factorsthat can affect its measurement (Flipsen, 2006; Hodge & Whitehill, 2010; Kent, Miolo, & Bloedel, 1994; Miller, 2013). Theseinclude variables related to the talker, the speaking task and environment, phonetic and linguistic characteristics of thespoken material, listener characteristics, judging task and listening environment. This is particularly the case for pre-literatechildren, children at younger ages developmentally, and/or children with central nervous system impairment, whereadditional factors related to language competence and ability to attend to, and perform the speaking task, may furthercomplicate intelligibility assessment.

2. Brief summary of the Test of Children’s Speech (TOCS)

TOCS (Hodge, Daniels, & Gotzke, 2009) was developed to provide a standard protocol for obtaining intelligibility andspeaking rate measures from children with developmental language ages as young as three years. The impetus fordeveloping TOCS was to provide a means to obtain reliable and valid intelligibility measures from English-speaking childrenwith dysarthria. TOCS measures are based on a narrow definition of intelligibility, that is, the extent to which listeners who

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 53

are not familiar with the child’s speech can identify the words in audio recordings of utterances produced by the child. On acontinuum of signal-dependent and signal-independent measures of intelligibility, (Miller, 2013), TOCS was designed to becloser to the signal-dependent end because the listener’s task is to understand the child’s spoken words based on the soundsignal without broader contextual cues beyond those contained in the utterance. TOCS word test items were selected usingthe phonetic contrast approach described by Kent, Weismer, Kent, and Rosenbeck (1989). Forty consonant, vowel andsyllable shape contrasts are sampled in minimal pair contexts in 78 items. Three parallel forms of the word test have beendeveloped. The sentence test is similar to the sentence format of the Assessment of Intelligibility of Dysarthric Speech createdfor adults (Yorkston & Beukelman, 1981), but the vocabulary and morphosyntax of the utterances were developed to beappropriate for children with developmental ages between 3 and 7 years. Items are selected randomly from pools of 2 to 7word utterances to make a unique 80-word form for each administration. The user selects the longest utterance length toappear in a form (3, 4, 5, 6 or 7 words), based on the child’s mean length of utterance, to ensure that the language level of theitems is within the child’s ability. For example, for a child with an estimated MLU between 3 and 4 words, nine 2-word items(18 words), 10 3-word items (30 words), and 8 four-word items (32 words) would be selected randomly from the respectivemaster sentence length pools to create a ‘‘unique’’ 80 word sentence test. This serves to reduce a listener’s ability to predictthe words in an item that the child will say. During item selection, the software ensures that content words (nouns, verbs,adjectives and adverbs) do not repeat within a test so that the items that are selected for a given test will contain uniquecontent words. Most of the items in the 2-word pool and many of the items in the 3-word pool are noun and verb phrases, asopposed to true sentences. Complex utterances only appear in the 6- and 7-word utterance pools. The software controls thenumber and type of complex utterances that are selected for each test that has 6- and 7-word utterances so that these are thesame across all tests.

For both TOCS test formats, the child imitates a model of each item, presented with a representative photograph to givesemantic context. The word(s) in the item are shown in print below the picture. At a later time, audio recordings of the child’sproductions of the items are presented to listeners who transcribe orthographically the words perceived from the recordings.These transcriptions are compared to the stimulus items spoken by the child to determine the number of words that areidentified correctly by the listener. In each test format, the percentage of words identified correctly out of the total wordsspoken provides the intelligibility score.

Like the sentence format of the Assessment of Intelligibility of Dysarthric Speech, the duration of each of the child’s recordedutterances in the TOCS sentence test can be measured and summed to determine total speaking time in minutes. This valuecan be used to calculate two speaking rate measures: words per minute (WPM) and intelligible words per minute (IWPM).Persons with CP have been reported to have slower speaking rates than expected for their age (DuHadway & Hustad, 2013;Nip, 2013). Yorkston and Beukelman (1981) reported that IWPM (a measure that combines intelligibility and rate) provides amore sensitive measure of disorder severity than intelligibility scores alone for adults with acquired dysarthria. Hodge andGotzke (2014) reported that when both rate and intelligibility scores are taken into consideration, the TOCS imitatedsentence sample provides a more discriminating measure of severity of speech disorder for children with CP and speechimpairment than intelligibility scores alone. However it is not known how IWPM scores compare among children withtypically developing speech, speech sound disorder of unknown origin and children with CP and dysarthria.

Software (TOCS+) has been developed to standardize and expedite (1) creating and presenting the items for eachadministration, (2) digitally recording young children’s utterances directly to the computer hard drive as .wav files, (3)playing these to listeners for word identification using orthographic transcription, and (4) capturing and initial scoring of thelistener’s transcriptions (see www.TOCS.plus.ualberta.ca).

2.1. Purpose and research questions

In keeping with the original purpose for developing the TOCS, initial evaluation of its reliability (delayed alternate forms,inter-rater) (Hodge & Gotzke, 2010) and criterion-related validity (Hodge & Gotzke, 2014) has focused only on children withdevelopmental dysarthria and CP. CP is the is the most common neurodisability in children (Msall & Park, 2009) anddevelopmental dysarthria has been identified as the most frequently occurring motor speech disorder in these children(Webb & Adler, 2008). The purpose of the current study was to evaluate construct-related validity of the TOCS intelligibilityand rate measures by comparing groups known to differ on the construct (Crocker & Algina, 1986): young children withtypically developing speech who differ in age (3, 4, 5 and 6 years of age), and children with and without speech disorders.Two groups of children with speech disorders were included (children with speech sound disorders of unknown origin andchildren with CP and dysarthria). All but one of the children with CP and dysarthria were also participants in previous studiesthat examined the reliability and criterion-related validity of TOCS. In addition, TOCS measures were obtained from a group of15 women with typical speech production who served as an adult group for comparison. The specific research questionsaddressed were:

(1) D

o TOCS scores show the expected increase with increasing age in children with typically developing speech andlanguage (TDS) between 3 and 6 years, and are the older children’s scores comparable to the scores obtained for adults?

(2) D

o a group of children between 3 and 6 years with speech sound disorders of unknown origin (SSD-UNK) and age-appropriate receptive language and a group of children with CP and dysarthria (DYS) have significantly lower TOCS

intelligibility scores than children of the same age with typically developing speech?

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6354

(3) D

o children with CP and dysarthria, who are able to perform the speaking tasks required for the TOCS measures, haveWPM scores that differ from the two groups of children without CP and dysarthria?

(4) D

o IWPM scores, which combine intelligibility and rate, differ significantly among the three groups, being highest for theTDS and lowest for the DYS group (where both reduced intelligibility and slower rates are expected)?

Affirmative findings for these questions would be interpreted as support for construct-related validity of the TOCS.

3. Method

3.1. Ethical considerations

The procedures followed in the study were in accordance with and approved by the Health Research Ethics Board at theUniversity of Alberta and the research ethics boards at the community health and hospital sites where children wererecruited. Informed consent was obtained from the parents of the children and from the listeners. The children also assentedto participate. Parents were provided with payment to cover their parking expenses for attending the data collectionsessions. Each listener who judged a child’s recordings received an honorarium of 10 dollars for participating. The 45listeners who judged adults’ TOCS recordings were enrolled in an undergraduate psychology program and received classcredit for participating.

3.2. Participants

All participants lived in western Canada and had English as their first language. All talker participants had normal hearingbased on an audiological assessment conducted within one year of the date of recording or a standard hearing screening atthe time of recording (Alberta College of Speech-Language Pathologists and Audiologists, 2008). All listener participants forthe children’s recordings passed a standard hearing screening at the time of their data collection session. Listeners whojudged the adults’ recordings reported no history of or current hearing loss.

3.2.1. Talkers

Children were recruited via convenience sampling from local community programs and two rehabilitation hospitalsin western Canada. All children in the TDS and SSD-UNK groups were judged to have oral-pharyngeal structures withinnormal limits based on a screening (Dworkin & Culatta, 1996) conducted by a speech-language pathologist. There were48 children in the TDS group, with 12 at each age level: 3, 4, 5 and 6 years. Parents reported no concerns about theirchild’s speech and language development and each child scored at or above the 16th percentile on the receptive,expressive and articulation subtests of the Fluharty Preschool Speech and Language Screening Test (Fluharty-2; Fluharty,2001). There were also 48 children in the SSD-UNK group. This included 12 3 year-olds, 12 4 year-olds, 13 5 year-olds and11 6 year-olds. The investigators were unsuccessful in recruiting a 12th 6 year-old without a concomitant receptivelanguage delay but were able to recruit an additional 5 year-old with SSD-UNK and age-appropriate receptive language.Children were recruited via referral from community speech-language pathologists (were receiving or on waiting lists toreceive intervention for SSD) or via parents who responded to recruitment posters because they were concerned abouttheir child’s speech. The articulation screening subtest of the Fluharty-2 was also administered to the SSD group. Forty-sixof the children scored at or below the 9th percentile and two scored at the 16th percentile. One of these two children wasreceiving speech therapy focusing on word intelligibility beyond the single word level at the time of recruitment, whilethe other had notable vowel errors, which were not captured by the Fluharty-2 articulation screen. Forty-seven of thechildren in the SSD group had age-appropriate receptive language (�16th percentile) based on the Fluharty-2 or anotherstandardized test of receptive language administered by the referring speech-language pathologist. One child’s receptivelanguage was not assessed using a standardized test but was judged to be age-appropriate by the referring speech-language pathologist. Forty-three children in the SSD group scored at or above the 16th percentile on standardized testsof expressive language.

Twenty-two children with a diagnosis of CP participated. They ranged in age from 3 to 10 years (M 6.1 years; SD 2.3). Themedical diagnosis of CP was made by experienced pediatric physiatrists who provided clinical services to children with CP atthese hospitals. The majority of children had spastic cerebral palsy with varying degrees of involvement. Type of cerebralpalsy and distribution of motor impairment of the 22 children are described in Table 1. The children (12 girls; 10 boys)represented the full range of severity (levels I to V) on the Gross Motor Function Classification System Expanded and Revised(Palisano, Rosenbaum, Bartlett, & Livingston, 2007). All children had developmental receptive and expressive language agesof at least 3 years as reported by their clinical speech-language pathologists. All children were able to perform the tasksrequired. All of the children were judged to be dysarthric as determined independently by two experienced speech-languagepathologists. They ranged in severity of speech impairment from mild to profound and all used speech as their main means ofcommunication for social interaction.

Fifteen women between the ages of 18 and 30 years were recruited locally for the adult group. Inclusion criteria includedEnglish as their first language, no history of speech therapy, no history of smoking in the past 5 years and a normal neurologicalhistory. They were also required to be healthy at time of recording (e.g., free from colds that would affect their voice).

Table 1

Characteristics of the child participants with cerebral palsy and dysarthria.

Child CAa

(years)

CPb type Sex GMFCSg

level

Word test

intelligibility

M (%)

Sentence test

intelligibility

M (%)

WPM

M

IWPM

M

Sentence test

longest item

(words)

1 3 Spastic quadriplegia Fe IV 16.7 37.9 59.9 22.7 4

2 3 Worster-Droughtc Mf I 56.5 62.7 80.5 50.5 4

3 4 Ataxic M II 22.2 21.9 60.8 13.3 4

4 4 Spastic hemiplegia (L)d M II 22.5 32.9 115.1 37.9 4

5 4 Spastic quadriplegia F IV 29.5 35.1 73.2 25.5 5

6 4 Spastic quadriplegia F IV 55.1 59.1 112.8 66.4 5

7 4 Worster-Drought F I 19.9 29.9 78.5 24.8 6

8 4 Spastic quadriplegia M IV 53.5 69.6 81.5 56.7 6

9 5 Spastic quadriplegia F IV 53.4 80.6 96.6 77.3 6

10 5 Worster-Drought M II 49.0 66.0 96.0 63.0 6

11 5 Spastic diplegia M III 60.7 61.7 109.1 67.3 6

12 6 Spastic diplegia F III 30.3 45.0 116.4 52.4 7

13 6 Spastic quadriplegia F IV 17.5 13.7 65.1 11.0 4

14 6 Worster-Drought M I 27.3 41.2 120.0 49.6 5

15 7 Spastic diplegia F III 62.0 85.2 126.9 108.9 7

16 8 Spastic hemiplegia (L) F II 32.9 17.7 95.4 16.9 6

17 8 Spastic quadriplegia F IV 40.1 28.6 83.4 38.9 7

18 8 Mixed spastic-dyskinetic M V 10.1 10.3 79.3 3.9 6

19 9 Mixed spastic-dyskinetic M III 21.4 25.9 51.7 13.4 7

20 10 Spastic quadriplegia F IV 12.8 25.2 99.7 25.1 7

21 10 Spastic quadriplegia F IV 47.0 76.9 90.9 72.5 7

22 10 Spastic quadriplegia M V 7.7 13.0 63.9 8.3 3

a CA is the chronological age.b CP is cerebral palsy.c Worster-Drought is the Worster-Drought Syndrome, (i.e., congenital suprabulbar paresis.d L is left.e F is female.f M is male.g GMFCS is the Gross Motor Function Classification System Level (Palisano et al., 2007), which uses a 5-level system to classify a child’s abilities and

limitations in gross motor function, with level I indicating the highest level of abilities and fewest limitations.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 55

They were judged to have articulation, voice and resonance characteristics within normal limits based on ratings of theiraudio recordings of a standard reading passage by two registered speech-language pathologists.

3.2.2. Listeners

Adults who spoke Canadian English as a first language, passed a standard hearing screening, and were enrolled in orgraduate of a university degree program served as listeners. A total of 399 listeners (age range 18 to 38 years) participated.

3.3. Recordings

All recording sessions took place in an acoustically treated sound booth. Recordings were obtained using a head-mountedmicrophone (Shure WH20) connected to an Audio Buddy preamplifier. The output cable from the preamplifier was routedthrough the wall of the sound booth and then to the microphone input on the computer’s inboard audio card. TOCS+ softwarewas used to present the word and sentence tests. A demonstration of the software can be found at www.TOCS.plu-s.ualberta.ca. One of the three formats of the TOCS word test was assigned randomly to each talker within a group. The sameproportion of talkers in each group completed each of the three word test forms. TOCS+ software randomized thepresentation order of the 78 stimulus items, presented each item by playing a verbal model and showing a picture cue andword, and recorded the child’s productions of the stimulus items directly to computer as digital audio (.wav) files (SR 48 kHz,QS 16 bit). Children were instructed to listen for the word, remember it, and then say it when cued by the software. Fourpractice items were presented before the test items. Three breaks (15 s screen animations) were provided to sustain thechildren’s attention. The adult subjects read the word test stimuli from the TOCS+ presentation screens. On average, it tookabout 12–15 min to record a word test. In general, administration to younger children and those with CP took longer than forolder children in the TDS and SSD-UNK groups.

Each talker participant was also administered a unique form of the TOCS sentence test. The longest stimulus item was7 words for the adults. For the children in the TDS group, the longest item was 4 words for the 3 year-olds, 5 words for the4 year-olds, 6 words for the 5 year-olds and 7 words for the 6 year-olds. For the children in the SSD-UNK and DYS groups,parents were asked to estimate the average number of words the child used in an utterance when speaking at home. Thechild was also engaged in an introductory conversation at the beginning of the session to obtain a short spoken languagesample to augment the parent’s estimate of MLU in making a decision about the longest item length for the child’s TOCS

sentence items. As with the word test, children were instructed to listen to the words that go with the picture, remember

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6356

them, and then say them when cued by the software. Two practice items were presented before the test items. Breaks in theform of 15-second screen animations were provided after every 10 items. The adult subjects read the sentence test itemsfrom the TOCS+ presentation screens. On average, it took about 10 min to record a sentence test.

3.4. Preparation of the recordings for analysis

TOCS+ software automatically creates a transcript (key) of the stimulus items in a given test when it generates a word orsentence test. A research assistant checked each target stimulus item in the word and sentence test transcripts against whatthe child actually said in the test recordings. This was verified by playing and observing the audio waveform of the child’srecording using Adobe Audition (1992) audio editing software. Changes were made as needed to match the transcript to thechild’s utterances. The final key was the corrected TOCS transcript. At the same time that the transcripts were checked, anyextraneous comments by the child and examiner in the recordings were removed that occurred before or after the child’sproduction of the test item.

3.5. Intelligibility measures

Three independent listeners transcribed orthographically the items in each set of recordings for each child and test format(word and sentence). Of the three listeners who judged a child’s test, two were enrolled in a speech-language pathologytraining program and one was enrolled in or a graduate of a different program of study. Each listener transcribed only twotests—a word test for one talker and a sentence test for a different talker. Listeners heard each word test item once and eachsentence item no more than twice. They were instructed to type in the words that they heard the talker say and encouragedto guess if they could not understand a word or words. Prior to judging the word test, listeners were shown a screen with alist of 250 words and instructed to read silently through the list of words. They were told that they might hear some of thewords in the list during the judging task. The list contained all the words in the three forms of the TOCS word test and thepractice items. This procedure was used to give listeners the same prior knowledge of potential words before judging a wordtest. Before the test items were presented for judging, the practice items recorded during administration of the word andsentence tests were played to listeners to practice the judging task and to familiarize them with the talker’s voice, for eachtest format, respectively.

The TOCS+ software has an algorithm for determining the number of word matches for an item in the test transcript withthe words that a listener types in and creates a ‘draft’ score based on the number of matches. After completion of a listeningsession, a research assistant corrected the draft scores after comparing the corrected TOCS transcript with the listener’sresponses for the sentence test format and checked both test formats for homonyms and misspellings in the listener’sresponses. Then, the percentage of words in each listener’s responses that matched the target words spoken was determinedfor each talker participant’s tests. The mean of the three listeners’ results served as the intelligibility score for each testformat for a talker.

3.6. Rate measures

The duration from onset to offset of each child’s production of each stimulus item on the TOCS sentence test was measuredusing Adobe Audition (1992) to obtain WPM scores. WPM was calculated by tallying the number of words spoken in all thetest items, based on the verified keys, and dividing the total number of words by the sum in minutes of the durations of all thetest items. To provide an estimate of measurement error, four tests were selected randomly from each group (adults, TDS,SSD-UNK and DYS) to have the utterances measured a second time by a different researcher, who was blind to the originalmeasures. The mean absolute difference was 2.9 WPM. IWPM was calculated by dividing the mean number of wordsidentified correctly by the three listeners by the sum in minutes of the durations of all the test items.

4. Results

4.1. Age comparisons for children in the TDS group and adults

Means and standard deviations for the TOCS word and sentence test intelligibility and rate scores (WPM and IWPM) foreach age level of the children in the TDS group and the adults are shown in Table 2. For each score for the children with TDS,group means increased monotonically with age and there is a general trend for standard deviation to decrease with age.Results of a 1-way MANOVA revealed significant age effects for word intelligibility scores (F(4, 59) = 18.18, p < 0.001),sentence intelligibility scores (F(4, 59) = 11.06, p < 0.001), WPM (F(4, 59) = 55.89, p < 0.001), and IWPM (F(4, 59) = 76.45,p < 0.001). Post-hoc testing was conducted using Dunnett’s C test (does not assume equal variances) using an alpha level of0.01. On the word test, the 3 year-olds had significantly lower intelligibility scores than the 6 year-olds and all of thechildren’s age groups had significantly lower scores than the adults. On the sentence test, the 3 year-olds had significantlylower intelligibility scores than the 6 year-olds and the adults but the scores for the 4, 5 and 6 year-old groups did not differsignificantly from the adults. For WPM scores, the 3 year-olds and 4 year-olds differed significantly from the 6 year-olds andall of the child age groups differed significantly from the adults. For the IWPM scores, all children’s age groups had

Table 2

Group means and standard deviations for the TOCS word and sentence test measures.

Group Age (yr) TOCS measures

Word

intelligibility (%)

Sentence

intelligibility (%)

WPM IWPM

M SD M SD M SD M SD

TDSa (n = 48) 3 70.5 12.7 86.1 8.6 112.9 12.9 96.9 12.5

4 79.9 8.9 89.9 9.1 120.7 9.4 108.5 12.5

5 82.8 6.8 92.3 4.4 130.2 6.7 120.7 8.3

6 87.6 4.2 97.4 1.0 135.6 9.9 132.2 9.4

All ages 80.2 10.5 91.5 7.4 124.8 13.1 114.5 17.0

Adults (n = 15) 18–30 94.9 1.0 98.6 0.9 174.2 16.4 172.7 16.3

SSD-UNKb (n = 48) 3 38.8 17.2 47.4 23.9 112.3 14.2 52.8 25.9

4 51.6 16.0 63.4 21.3 110.1 10.7 70.7 25.7

5 62.0 18.7 69.4 21.8 123.4 10.8 84.3 24.9

6 73.5 17.1 82.6 16.2 126.8 18.6 106.1 28.4

All ages 56.3 21.0 65.4 23.8 118.1 15.1 78.0 31.8

DYSc (n = 22) 3–10 36.3 17.8 46.2 24.2 90.6 22.3 45.5 28.8

a TDS is typically developing speech.b SSD-UNK is speech sound disorder of unknown origin.c DYS is dysarthria and cerebral palsy.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 57

significantly lower scores that the adults. In addition, the 3 year-olds had significantly lower IWPM scores than the 5 and 6year-olds and the 4 year-olds had significantly lower IWPM scores than the 6 year-olds.

Fig. 1 displays a scatter plot of the 48 children’s word and sentence intelligibility scores with age group indicated. Overlapof the age groups is evident but a trend for increasing scores and a smaller distribution of scores with age is apparent. As canbe seen in Fig. 1, the sentence intelligibility scores were higher than the word intelligibility scores for every child. Fig. 2displays the children’s WPM and IWPM scores. Again, overlap of the age groups is evident, but the general trend is for WPMand IWPM to increase with age.

4.2. Comparisons for the TDS, SSD-UNK and DYS groups

Means and standard deviations for the TDS, SSD-UNK and DYS groups on the four TOCS scores are shown in Table 2. Agelevel means and standard deviations are also reported in Table 2 for the SSD-UNK group. For all four scores, the TDS group

Fig. 1. Scatter plot of the TOCS word and sentence intelligibility scores for the children with typically developing speech, with age level in years indicated.

Fig. 2. Scatter plot of the TOCS sentence words per minute (WPM) and intelligible words per minute (IWPM) rate scores for the children with typically

developing speech, with age level in years indicated.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6358

had the highest mean and the DYS group had the lowest mean. Results of a 1-way MANOVA revealed significant group effectsfor word intelligibility scores (F(3, 115) = 61.08, p < 0.001), sentence intelligibility scores (F(3, 115) = 55.0, p < 0.001), WPM (F(3,

115) = 40.2, p < 0.001) and IWPM (F(3, 115) = 64.5, p < 0.001). Post-hoc testing was conducted with Dunnett’s C test using analpha level of 0.01. All groups differed significantly from one another on word and sentence test intelligibility scores. The DYSgroup mean WPM score was significantly lower than the mean WPM scores for both the TDS and the SSD-UNK groups but asignificant difference was not found between the means of the SSD and TDS groups. The three groups differed significantlyfrom each other on IWPM scores.

Fig. 3 displays a scatter plot of all of the children’s word and sentence intelligibility scores with group membershipindicated. The scores of the children in the SSD-UNK group have the greatest distribution and overlap the scores of thechildren with dysarthria and all age levels of the children in the TDS group. Two 3 year-olds and one 4 year-old in the TDS

Fig. 3. Scatter plot of the TOCS word and sentence intelligibility scores for the children with typically developing speech (TDS), children with speech sound

disorder of unknown origin (SSD-UNK) and children with cerebral palsy and dysarthria (DYS), with age level in years indicated for the children in the TDS

and SSD-UNK groups.

Fig. 4. Scatter plot of the TOCS sentence words per minute (WPM) and intelligible words per minute (IWPM) rate scores for the children with typically

developing speech (TDS), children with speech sound disorder of unknown origin (SSD-UNK) and children with cerebral palsy and dysarthria (DYS), with

age level in years indicated for the children in the TDS and SSD-UNK groups.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 59

group have word intelligibility scores that fall within the distribution for the DYS group (maximum score of 62.0%). Four 3year-olds and three 4 year-olds in the TDS group have sentence intelligibility scores that fall within the distribution for theDYS group (maximum score of 85.2%). While no child in the TDS group had a lower intelligibility score on the sentence testcompared to the word test, eight children in the SSD-UNK group and four children in the DYS group had the same score onboth tests or lower scores on the sentence test compared to the word test. As with the TDS group, the trend for increasingscores and decreasing distribution of scores with age is evident in the SSD-UNK group.

Fig. 4 is a scatter plot of all children’s WPM and IWPM scores. Only one 4 year-old in the SSD-UNK group has a WPM score(89.2) that falls below the lower range of the TDS group (90.6) and 11 (half) of the children in the DYS group have WPM scoresthat fall below the range of all children in the TDS and SSD-UNK groups. The child in the TDS group with the lowest IWPMscore (77.6) was a 3 year-old. Half (24) of the children in the SSD-UNK group and 21 of 22 children in the DYS group hadIWPM scores below 77.6.

Fig. 5 shows the children’s word intelligibility scores plotted against IWPM scores, the two scores that showed thegreatest differences among the three groups of children. Two 3 year-olds and one 4 year-old in the TDS group and 28 childrenrepresenting all age levels in the SSD-UNK group had word intelligibility and IWPM scores that fell below the child in the DYSgroup with the highest word intelligibility and IWPM scores (word intelligibility 62.0%; IWPM 108.9). Figs. 3–5 show positive

Fig. 5. Scatter plot of the TOCS word test intelligibility scores and the sentence intelligible words per minute (IWPM) rate scores for the children with

typically developing speech (TDS), children with speech sound disorder of unknown origin (SSD-UNK) and children with cerebral palsy and dysarthria

(DYS), with age level in years indicated for the children in the TDS and SSD-UNK groups.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6360

relationships between the variables plotted on their X and Y axes. Although Fig. 5 displays the two TOCS scores that are themost ‘‘independent’’ of each other (word intelligibility and IWPM), it has the least overlapping of coordinates and the mostlinear function, compared with Figs. 3 and 4, particularly for the children with higher scores. A strong positive correlationwas found between these two variables (r 0.93, p < 0.001).

An exploratory post-hoc multinomial logistic regression analysis was conducted to determine how well word testintelligibility and IWPM scores classified the children in the TDS, SSD-UNK and DYS groups. Using group as the dependentvariable, TDS as the reference group, and word intelligibility score as the predictor variable, a significant result was obtained(p < 0.001). The overall correct classification rate was 66.9%; TDS correct classification rate was 81.3%, SSD-UNK correctclassification rate was 58.3% and DYS correct classification rate was 54.5%. No TDS child was misclassified in the DYS groupand vice versa. Using this same procedure with IWPM scores as the predictor variable, a significant result was also obtained(p < 0.001). The overall correct classification rate was also 66.9%; TDS correct classification rate was 83.3%, SSD-UNK correctclassification rate was 56.3% and DYS correct classification rate remained at 54.5%. No TDS child was misclassified as DYS but1 child in the DYS group was misclassified as TDS. Using both word test intelligibility and IWPM scores as covariates did notimprove classification rate (64.4%), which was not unexpected, given their strong correlation. More comprehensive analysesare underway to examine the use of TOCS scores to predict membership in these three groups of children.

5. Discussion

This study examined the construct-related validity of the TOCS word and sentence tests by comparing children expectedto differ on the intelligibility and rate measures that can be obtained from these tests, based on previously publishedresearch. Four research questions were addressed, motivated by the rationale that affirmative responses to these questionswould be interpreted as support for the construct-related validity of the TOCS for discriminating severity of speechinvolvement, as indexed by intelligibility and rate measures, for young children with speech disorders, including childrenwith CP and developmental dysarthria. The results of this study affirmed support for the construct-related validity of theTOCS, and are discussed by each question, respectively.

5.1. Do TOCS scores show the expected increase with increasing age in children with typically developing speech and language

between 3 and 6 years, and are the older children’s scores comparable to the scores obtained for adults with typical speech

production?

The age group means for the children’s word and sentence intelligibility scores increased with each age level from 3 to 6years but only the 3 year-olds differed significantly from any of the other child age levels (6 year-olds). This might beexpected given that the literature reviewed in the Introduction suggested that children younger than 4 years are stillincreasing their speech intelligibility. The finding that the adult group had significantly higher word test intelligibility scoresthan all age levels in the children suggests that even by age 6 years, children do not have adult-like production of theconsonant and vowel contrasts of English that are sampled in the minimal pair items in the TOCS word test. A future paperwill describe the errors of the children and adult talkers as perceived by listeners in the word test. The finding that only the 3year-olds differed from the adults on the sentence test intelligibility scores is in agreement with the literature reviewed inthe Introduction that by age four years, children’s connected speech is expected to be as intelligible as adults’ to unfamiliarlisteners. A finding that was common for all children in the TDS group and all the adult talkers was that the intelligibilityscores for the word test were lower than for the sentence test. This might be expected given the additional context providedby the sentences. The age group means for WPM and IWPM for the children with TDS increased with increasing age with boththe 3 and 4 year-olds having significantly lower scores than the 6 year-olds. Walker and Archibald (2006) reportedarticulation rates in syllables per second for 16 children reported to have typically-developing speech and who were studiedlongitudinally between 3 and 5 years in several contexts including spontaneous speech and imitated sentences. The authorsdid not find significant age differences in speaking rate and, like the children in the TDS group in the current study, childrenhad considerable individual differences in rate. The finding that the adults had significantly higher WPM rates than all ofthe child age groups supports the generally accepted view that speech motor skills do not reach adult-like proficiency byage 6 years.

5.2. Do a group of children between 3 and 6 years with SSD-UNK and age-appropriate receptive language and a group of

children with CP and dysarthria have significantly lower TOCS intelligibility scores than children of the same age with

typically developing speech?

For TOCS word test and sentence test intelligibility and IWPM scores, the SSD-UNK group had significantly lower scoresthan the TDS group and the DYS group had significantly lower scores than both the TDS and the SSD-UNK groups. Unlike thefinding that intelligibility scores for the word test were lower than for the sentence test for all the adults and all children inthe TDS group, eight children in the SSD-UNK group and 6 children in the DYS group had essentially the same or higher wordintelligibility than sentence intelligibility scores. These were children with relatively low scores on both the word andsentence tests. For these children, it appears that the potential facilitating effect of context was offset by other factorsthat decreased their intelligibility in connected speech. The only TOCS measure that did not differ significantly between the

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 61

SSD-UNK and TDS groups was WPM. This might be expected, given that children in both groups were reported to havenormal neurological histories and the children in both groups had age-appropriate receptive language abilities and, with theexception of five children in the SSD-UNK group, age appropriate expressive language abilities. As can be seen in Table 2, allage levels in the SSD-UNK group and the DYS group had higher standard deviations on the TOCS intelligibility measures thanthe TDS group, even compared to the 3 year-olds. As can be seen in Figs. 3–5, the children in the SSD-UNK group had thewidest distribution of scores on all of the measures, overlapping the scores of the children in the TDS and DYS groups.

5.3. Do children with CP and dysarthria, who are able to perform the speaking tasks required for the TOCS measures,

have WPM scores that differ from a group of 3 to 6 year-old children with SSD-UNK?

The mean WPM score for the DYS group was significantly lower (difference of 27.5 WPM) than for the SSD-UNK group. Ascan be seen in Table 1 and Fig. 4, 16 of the 22 children in the DYS group had WPM scores of less than 100, while only fivechildren is the SSD-UNK group (three 3 year-olds, one 4 year-old and one 6 year-old) had scores below 100. On a group basis,the TOCS appears to differentiate these two groups of children with speech disorders on the basis of speaking rate, whichsupports previous reports in the literature that persons with CP and dysarthria have slower speaking rates than peerswithout neurological impairments. Of note is that 11 of the children in the DYS group had WPM scores that fell within thedistribution of the TDS group’s WPM scores.

5.4. Do IWPM scores, which combine intelligibility and rate, differ significantly among the three groups, being highest for

the TDS and lowest for the DYS group (where both reduced intelligibility and slower rates are expected)?

As expected, mean IWPM scores for the three groups differed significantly from one another, being highest for children inthe TDS group (114.5) and lowest for children in the DYS group (41.2). This was also the TOCS measure that had the greatestnumber of significant age differences in the TDS group. Unexpectedly, the word test intelligibility measure also differedsignificantly among the three groups of children. It is hypothesized that the reduced articulatory precision associated withdysarthria resulted in a greater number of errors for the children with CP, compared to the SSD-UNK group, when required tomake the phonetic contrasts distinctive for the minimal pair items on the word test. This hypothesis will be explored in afuture paper that examines error types on the word test.

6. Limitations

Recruitment of children with CP and dysarthria was more challenging than recruitment of children in the other groupsstudied. There are fewer children in the CP group (n = 22) compared with each of the TDS and SSD-UNK groups (n = 48), thenumber of children with CP at each age varied and the ages of the children in the CP group extended up to 10 years comparedto 6 years for the children in the TDS and SSD-UNK groups. In addition, results of standardized language assessments are notreported for the children with CP as current information was not available for the majority of children. Inclusion of a group ofchildren with CP who were not diagnosed with a motor speech disorder would have contributed additional importantinformation about the intelligibility of children with CP.

7. Conclusions

The increase in intelligibility scores on the TOCS word and sentence tests between ages 3 and 6 years for children withtypically developing speech and the significantly lower intelligibility scores obtained by the two groups of children withspeech disorders on the TOCS word and sentence tests correspond to what is expected, based on the literature summarized inthe Introduction. The finding that mean speaking rate in WPM was significantly lower for children with dysarthria,compared to the children with typically developing speech and with speech sound disorders of unknown origin washypothesized, based on previously published descriptions of these children’s speech, but this study provides newevidence to support this hypothesis. The strong positive correlation between the TOCS word intelligibility score and theTOCS sentence IWPM score suggests that while each of these is measuring a different aspect of speech (single word versussentence production), each measure provides a rank ordering of scores that provides a finer discrimination amongchildren, and between children and adults, than the TOCS sentence intelligibility score alone. Further investigation of thepredictive power of these scores to distinguish among children with TDS, SSD-UNK and DYS is underway. The results ofthis study support the construct-related validity of TOCS as a tool for obtaining intelligibility and rate scores thatare sensitive to group differences between 3 and 6 year-old children, with and without speech sound disorders, and to3+ year-old children with speech disorders, with and without dysarthria, and to individual differences in children withinthese three groups.

Declaration of interest

This work was supported by funding from the Canadian Language and Literacy Research Network (Project ReferenceNumber 27103000) (www.CLLRNet.ca) and the University of Alberta. TEC Edmonton, the technology transfer office at the

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–6362

University of Alberta, holds the copyright for and licenses the Test of Children’s Speech software (TOCS+). Revenue generatedfrom the license royalties are paid to the University of Alberta. The authors are solely responsible for the content and writingof the paper.

Acknowledgements

The authors especially want to thank the children, their families, and the adult talkers and listeners who participated inthe project; Leslie Wellman and Kim Cote Reschny for their assistance with data collection and management; Dr. ConnieVarnhagen for recruitment of listeners to judge the adult talkers’ recordings; and Dr. Terrance Nearey for contributing hisexpertise in speech perception and statistical analysis.

Financial and Non-financial Disclosures

The authors have no financial or nonfinancial relationships to disclose.

Appendix A. Continuing education questions

(1) The age at which conversational speech is expected to be understood by unfamiliar listeners 100% of the time, based on

the guidelines described in the articles by Coplan and Gleason (1988), Flipsen (2006) and Weiss (1982) is:(a) 3 years(b) 4 years(c) 5 years(d) years(e) None of the above

(2) T

he group of children with TDS described in the article did not differ significantly from the group of children withSSD-UNK on which TOCS measure?(a) word intelligibility scores(b) sentence intelligibility scores(c) WPM scores(d) IWPM scores(e) None of the above

(3) T

he group of children with SSD-UNK differed significantly from the group of children with CP and dysarthria on whichTOCS measure(s)?(a) word intelligibility scores(b) sentence intelligibility scores(c) WPM scores(d) IWPM scores(e) All of the above

(4) W

hich two TOCS measures, when plotted together, provided the least overlap in the children’s scores across the threegroups?(a) word and sentence intelligibility scores(b) WPM and IWPM scores(c) word intelligibility and WPM scores(d) word intelligibility and IWPM scores(e) sentence intelligibility and IWPM scores

(5) W

hich of the following statements is supported by the results of this study?(a) All age levels of children in the TDS group had significantly lower mean scores on the word intelligibility test than the

adult group.(b) WPM scores decreased as aged increased for the children in the TDS group.(c) Adults and the 6 year-old children with TDS differed significantly on the TOCS sentence test measures.(d) All children in the study had word intelligibility scores that were lower than their sentence intelligibility scores.(e) The children with CP and dysarthria had a wider distribution of word and sentence intelligibility scores than the

children with SSD-UNK.

References

Alberta College of Speech-Language Pathologists and Audiologists (2008). Hearing screening guidelines. Retrieved from hhttp://www.acslpa.ab.ca/public/data/documents/ACFC3D5.pdfi.

Adobe Systems (1992). Adobe audition (Version 1.5) [software]. San Jose, CA: Adobe Systems.

M.M. Hodge, C.L. Gotzke / Journal of Communication Disorders 51 (2014) 51–63 63

Bax, M., Goldstein, M., Rosenbaum, P., Leviton, A., & Paneth, N. (2005). Proposed definition and classification of cerebral palsy. Developmental Medicine & ChildNeurology, 47, 571–576.

Coplan, J., & Gleason, J. (1988). Unclear speech: Recognition and significance of unintelligible speech in preschool children. Pediatrics, 82, 447–452.Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Orlando, FL: Harcourt Brace Jovanovich.DuHadway, C., & Hustad, K. (2013). Contributors to intelligibility in preschool-aged children with cerebral palsy. Journal of Medical Speech–Language Pathology,

20(4), 59–64.Dworkin, J., & Culatta, R. (1996). Dworkin–Culatta oral mechanism exam and treatment system (D-COME-T). Farmington Hills, MI: Edgewood Press.Flipsen, P. (2006). Measuring the intelligibility of conversational speech in children. Clinical Linguistics and Phonetics, 20, 303–312.Fluharty, N. B. (2001). Fluharty preschool speech and language screening test (Fluharty-2). Austin, TX: Pro-Ed Inc.Gordon-Brannan, M., & Hodson, B. (2000). Intelligibility/severity measurements of prekindergarten children’s speech. American Journal of Speech–Language

Pathology, 9, 141–150.Hodge, M. (2010). Intervention for developmental dysarthria. In R. McCauley & S. McLeod (Eds.), Treatment of speech sound disorders in children (pp. 557–578).

Baltimore, MD: Brookes Publishing.Hodge, M., Daniels, J., & Gotzke, C. L. (2009). TOCS+ intelligibility measures (Version 5.3) [computer software]. Edmonton, AB: University of Alberta.Hodge, M. M., & Gotzke, C. L. (2010). Stability of intelligibility measures for children with dysarthria. Journal of Medical Speech–Language Pathology, 18(4), 61–65.Hodge, M. M., & Gotzke, C. L. (2014). Criterion-related validity of the Test of Children’s Speech sentence intelligibility measures for children with cerebral palsy and

dysarthria. International Journal of Speech-Language Pathology, 16(4), 417–426.Hodge, M., & Whitehill, T. (2010). Intelligibility impairments. In J. S. Damico, M. J. Ball, & N. Muller (Eds.), Handbook of language and speech disorders (pp. 99–114).

Oxford: Wiley-Blackwell.Hodson, B. W. (2004). Hodson assessment of phonological patterns (3rd ed.). Austin, TX: Pro-Ed.Kent, R. D., Weismer, G., Kent, J. F., & Rosenbek, J. C. (1989). Toward phonetic intelligibility testing in dysarthria. Journal of Speech and Hearing Disorders, 54, 482–

499.Kent, R. D., Miolo, G., & Bloedel, S. (1994). The intelligibility of children’s speech: A review of evaluation procedures. American Journal of Speech Language Pathology,

3(2), 81–95.McLeod, S., Harrison, L., & McCormack, J. (2012). The intelligibility in context scale: Reliability and validity of a subjective rating measure. Journal of Speech,

Language, and Hearing Research, 55, 648–656.Miller, N. (2013). Measuring up to speech intelligibility. International Journal of Language and Communication Disorders, 48(6), 601–612.Miller, J., & Chapman, R. (1991). Systematic analysis of language transcripts (SALT). Madison, WI: University of Wisconsin-Madison.Msall, M., & Park, J. (2009). The changing panorama of preschool disability: Biomedical and social risks. In M. Shevell (Ed.), Neurodevelopmental disabilities: Clinical

and scientific foundations (pp. 27–42). London: Mac Keith Press.Nip, I. (2013). Kinematic characteristics of speaking rate in individuals with cerebral palsy: A preliminary study. Journal of Medical Speech–Language Pathology,

20(4), 88–94.Palisano, R., Rosenbaum, P., Bartlett, D., & Livingston, M. (2007). Gross motor function classification system expanded and revised. Hamilton, ON: CanChild Centre for

Childhood Disability Research, McMaster University.Pennington, L. (2008). Cerebral palsy and communication. Paediatrics and Child Health, 18(9), 405–409.Pennington, L., Miller, N., Robson, S., & Steen, N. (2010). Intensive speech and language therapy for older children with cerebral palsy: A systems approach.

Developmental Medicine & Child Neurology, 52, 337–344.Pennington, L., Roelant, E., Thompson, V., Robson, S., Steen, N., & Miller, S. (2013). Intensive dysarthria therapy for younger children with cerebral palsy.

Developmental Medicine and Child Neurology, 55, 464–471.Rice, M., Smolik, F., Perpich, D., Thompson, T., Rytting, N., & Blossom, M. (2010). Mean length of utterance levels in 6-month intervals for children 3 to 9 years with

and without language impairments. Journal of Speech, Language and Hearing Research, 53, 333–349.Shriberg, L. D., Austin, D., Lewis, B. A., McSweeny, J. L., & Wilson, D. L. (1997). The percentage of consonants correct (PCC) metric: Extensions and reliability data.

Journal of Speech, Language and Hearing Research, 40, 708–722.Walker, J., & Archibald, L. (2006). Articulation rate in preschool children: A 3-year study. International Journal of Language and Communication Disorders, 41(5),

541–565.Webb, W., & Adler, R. (2008). Neurology for the speech–language pathologist (5th ed.). St. Louis, MO Mosby, Inc..Weiss, C. (1982). Weiss intelligibility test. Tigard, OR: CC Publications.Wilcox, K., & Morris, S. (1999). Children’s speech intelligibility measure (CSIM). San Antonio, TX: Psychological Corp.Workinger, M. S. (2005). Cerebral palsy: Resource guide for speech–language pathologists. Clifton Park, NJ: Thomson-Delmar Learning.Yorkston, K. M., & Beukelman, D. R. (1981). Assessment of intelligibility of dysarthric speech. Austin, TX: Pro-Ed.