The Social Communication Questionnaire: Modifications and improved diagnostic validity

Research Grant Proposal 27

Running head: MODIFICATIONS OF THE SCQ

Research Grant Proposal: Modifications and Improved Psychometrics of the Social Communication QuestionnaireRebecca McNally

Alliant International UniversityAbstractBlock format, 120 words or less

Research Grant Proposal:Modifications and Improved Psychometrics of the Social Communication Questionnaire

The autism spectrum comprises a variety of disorders characterized by marked deficits in communication and social interactions, as well as restricted interests and repetitive behaviors (APA, 2000). Impairments in communication include a delay in, or lack of development of spoken language. In individuals with adequate speech, there may be a deficit in initiating or sustaining conversation with others, in addition to stereotyped and repetitive language. There is also a lack of varied, spontaneous imaginative or imitative play. Social impairments include poor modulation of eye contact, facial affect, and gesture in social interactions. Many individuals with an autism spectrum disorder (ASD) lack social and emotional reciprocity and fail to develop peer relationships appropriate to developmental level. Ritualistic and compulsive behavior that is abnormal in focus and intensity, stereotyped movements, and preoccupations with parts of objects are also hallmarks of ASD. In order for an individual to receive a diagnosis of ASD, delays or abnormalities in these three core domains must be evident prior to three years of age and must not be better accounted for by another medical or genetic disorder. In many cases, individuals with ASD have an associated diagnosis of mental retardation (APA, 2000).

The prevalence of ASD is likely on the rise due to advances in research related to the broader autism phenotype, more accurate clinical recognition (Filipek et al., 1999), and suspected unidentified environmental factors (Rutter, 2005). The Center for Disease Controls Autism and Developmental Disabilities Monitoring (ADDM) Network (2007) reported that 1 in 150 8 year-old children in the United States have an autism spectrum disorder. Prior to the development of the ADDM report, estimates from the past decade ranged from 30-60 cases per 10,000 (Rutter, 2005). These figures make a clear argument for the importance of accurate diagnostic procedures and effective interventions.

The gold standard measures for diagnosing autism include the Autism Diagnostic Observation Scale (ADOS: Lord et al., 1999), the Autism Diagnostic Interview-Revised (ADI-R: Rutter, LeCouteur & Lord, 2003), and expert clinical judgment based on DSM-IV criteria. The ADOS is a semi-structured assessment of social interaction, communication, play and imaginative use of materials while the ADI-R is a standardized, semi-structured, investigator-based interview for caregivers of individuals suspected of having an ASD. The ADOS and ADI-R both result in a diagnostic classification that adheres to DSM-IV (APA, 1994) criteria for ASD. Because these measures require specific, intense training and because administration takes a significant amount of time (ADOS: 30 minutes-1 hour, ADI-R: 1.5-2 hours), their use in primary care or education and community based intervention programs may be limited (Filipek et al., 2005). Therefore, there is great need for a reliable and valid screening instrument based on current diagnostic criteria for ASD that is useful in individuals with a range of developmental abilities (Berument et al., 1999). The Social Communication Questionnaire (SCQ; Rutter, Bailey & Lord, 2003) is one screening measure for ASD that has gained increased support for its use in both research and clinical settings. The SCQ, in its current state, has demonstrated fair psychometric characteristics, however further development would provide researchers and clinicians with greater utility.The purpose of this research grant is to review the current reliability and validity of the SCQ and to propose a research plan to improve the measure. The SCQ, based on the Autism Screening Questionnaire (Berument et al., 1999), is a parent report questionnaire used to screen for symptoms associated with ASD (Rutter, Bailey, & Lord, 2003). The specific aims of this research proposal are to a) improve the content validity of the SCQ, and b) assess the psychometric properties of the new version of the SCQ.Design and Psychometrics of the Current SCQ

Consisting of diagnostic algorithm items from the ADI-R, the SCQ was developed to efficiently and accurately select individuals for whom a more thorough diagnostic and psychological evaluation should be completed (Rutter, Bailey, & Lord, 2003). The SCQ was designed to take less than 10 minutes for caregivers to complete and 5 minutes for clinicians, physicians, or educators to score, allowing for utility across a variety of settings.

Two forms of the SCQ are available depending on the desired use (Rutter, Bailey, & Lord, 2003). Both are intended for children with a developmental level of at least two years and a chronological age of at least four years. The Lifetime Form provides a total score that is interpreted with reference to validated cut-off scores. The cut-off score indicates that an individual may suffer from an ASD and prompts further evaluation. As with any screening measure, it is possible that false negatives will occur and caution should be taken when interpreting scores. The Current Form offers scores related to symptomatology in the past three months and may be useful in tracking symptom changes and treatment outcomes (Rutter, Bailey & Lord, 2003).

The selection of items on the SCQ was based upon algorithm items of the ADI-R, which provides an operational diagnosis based on functioning in the areas of reciprocal social interaction, language and communication, and repetitive behaviors and restricted interests (Berument, et al., 1999). SCQ items were chosen to focus on behaviors that caregivers are likely to have had the opportunity to observe and that involve easily understood concepts. Items were designed to be clear, unambiguous in wording, and to require little to no inference to answer. All questions concern lifetime manifestations of behaviors, except in instances where qualitative deviance rather than delay or impairment was likely to influence coding. These questions instead focus on the 4 to 5 year old period of the individuals development. The SCQ consists of 40 items in question (yes/no) format; a score of 1 is given for the presence of a particular abnormal behavior and a 0 for its absence. The total score ranges from 0-34 or 0-39 (one question on current language functioning is not included in the total score), depending upon the overall language abilities of the individual in question.

The primary standardization data for the SCQ was obtained from a sample of 200 individuals recruited from previous studies of ASD (Berument, et al., 1999). The sample (see Table 1) was comprised of 160 individuals with ASD (autism, Aspergers syndrome, Fragile X syndrome, Rett syndrome) and 40 individuals with non-ASD diagnoses (conduct disorder, developmental language disorder, mental retardation, and other psychiatric disorders). All diagnoses were previously confirmed by diagnostic classification based on the ADI-R as well as expert clinical judgment. To date, there have been few reported reliability studies done on the SCQ. This is the case perhaps because of the extensive research on the reliability of the ADI-R. Because SCQ items are based directly on ADI-R algorithm items, ADI-R test-retest reliability data may be applicable to the SCQ. Clearly, there is a need to do reliability studies based solely on the SCQ. Test-retest reliability data on the SCQ would be especially necessary when using the SCQ as a measure of treatment and intervention outcome and as a measure of symptom severity change over time.

Internal consistency reliability of the SCQ was explored using Chronbachs alpha (Berument et al., 1999). The reliability coefficient for the total scale was .90 suggesting excellent internal consistency. All item to total score correlations were in the range of .26-.73 with 23 of 39 over .50 (Berument et al., 1999).

Content-related validity of the SCQ was also examined through ADI-R algorithm item development. Data from the ADI and ADI-R was used in this process. First, a wide variety of items concerning individual behaviors associated with ASD were assessed to determine which items had good interrater reliability and discrimination among diagnostic groups (Rutter, LeCouteur & Lord, 2003). Next, a pool of algorithm items was generated. Items were selected that most closely adhered to specific abnormalities described in clinical descriptions and observation of individuals affected by an ASD. A small number of items that were found to have high intercorrelations with other items were removed from this pool. The final step was to group these items based on how they mapped onto the three behavioral domains of the DSM-IV and ICD-10. Only one measure of construct validity was explored. Factorial validity data suggest that a four-factor model of the 39 items is the most meaningful and accounted for 42.4% of the total variation in the data (Lord, Rutter & LeCouteur, 1994). The social interaction and repetitive behavior factors mapped onto the related ADI-R domains. The two remaining factors reflected a split into a communication deficit factor and abnormal language factor. The results of this factorial validity investigation suggest that the items of the SCQ map onto the three core deficit domains in individuals with ASD (i.e., language and communication, social interaction, stereotyped/ repetitive behaviors).Several types of criterion-related validity investigations have been conducted on the SCQ. First, concurrent validity of the SCQ was assessed by correlating scores from the SCQ and total and domain scores of the ADI-R (Berument et al., 1999). Correlation coefficients ranged from .31 to .71, both within and across domains for all comparisons. These data demonstrate mixed evidence for the concurrent validity of the SCQ when the ADI-R is used as the gold-standard measure of comparison. A higher correlation would be warranted if the SCQ can be considered a replacement screener for the ADI-R.

Using the SCQ total score, sensitivity and specificity data was analyzed (Berument et al., 1999). An examination of the receiver operating curves for the SCQ total score yielded an optimal cut-off score of 15 for predicting a diagnosis of ASD. Using this cut-off score, sensitivity was .85 and specificity was .75. With mental retardation excluded, the 15 or more cut-off generated a sensitivity of .96 and specificity of .80 for autism verses other diagnoses. A higher score of 22 or more was required differentiate between autism and other spectrum disorders. In this case, the sensitivity was .75 and the specificity was .60. Berument and colleagues (1999) report that other cutting scores may be preferable, depending on the goal of screening. The findings of the reported studies show that the SCQ is a fairly effective screening measure for ASD; however, future research on the psychometric properties of the SCQ is warranted due to some of the limitations addressed above. Until further psychometric data is available, scores from the SCQ should be interpreted with caution.

Research Design and Methods

Study 1: Improved Content Validity

GoalContent validity is the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose (Haynes, Richard, & Kubany, 1995, p. 238). As the SCQ is currently constructed, there are several problems with content validity. As previously reviewed, the SCQ items were created based on algorithm items of the ADI-R. The author has anecdotally noted that many parents have difficulty accurately reporting on the specific aspects of their childs behavior that SCQ items probe. Because items are based on the ADI-R, which is a clinician-administered semi-structured interview, some items are ambiguous or can be interpreted in different ways. For example, question 13 on the Lifetime Form of the SCQ states, Does she/he ever have any interests that are unusual in their intensity but otherwise appropriate for her/his age group (e.g., trains or dinosaurs)? Many parents question whether age group refers to the childs chronological or developmental age. Clarifying the existing items and adding more specific behavior-based items could improve the content validity of this measure by increasing the representativeness of the construct of ASD. Procedure

Haynes and colleagues (1995) suggest guidelines for building content validity into a measure. First, these authors suggest operationalizing the domain under study. In order to re-operationalize the construct of ASD for the improved SCQ, a comprehensive review of all literature surrounding autism diagnosis and diagnostic tools will be completed. Based on this review, any current information and data regarding changes to autism diagnostic criteria will be considered. Haynes and colleagues (1995) also suggest using population and expert sampling in order to generate new items. Several panel discussions will be conducted with family member of individuals with ASD. These population-based panels will help to inform the author about what types of behaviors and characteristics relevant to a diagnosis of ASD caregivers are able to easily observe. Expert panels will be formed consisting of both researchers and clinicians who have expertise in the field of ASD. These panels will provide a forum for the authors to discuss current changes in diagnostic conceptualization.

Based on the results of the literature review of autism diagnosis as well as information gathered from both the population-based and expert panels, the author will create a new version of the SCQ with items representing the both broad construct of ASD as well as items tapping behavior specific abnormalities or deficits. In addition, the existing items of the SCQ will be compared against the components of the carefully defined domain of ASD. The new list of generated items will be examined to ensure that all relevant and defining aspects of the disorder are accounted for by SCQ items. Also, items will be examined to ensure that non-ASD related symtomatology or behaviors are excluded from items. The format of the SCQ (i.e., caregiver report, yes/no answer) will remain the same, however many test-items may be added at this initial stage. This new version of the SCQ will be subject to multiple data analysis procedures.

Data AnalysisThere will be several ways that both face and content validity will be analyzed. First, as suggested by Haynes (1995), multiple judges of content validity will be recruited from the population-based and expert panels. Approximately 20 judges (10 from population and 10 experts) will be asked to rate multiple aspects of the new SCQ using a 5-point evaluation scale. The judges will evaluate item content areas such as relevance, representativeness, specificity, and clarity. They will also be asked to assess the face validity of the measure. In particular, they will rate whether the measure appears to assess what it is intended to assess (i.e., children at risk for a diagnosis of ASD). Following the completion of the evaluator ratings, the author will use this descriptive data to identify problematic SCQ items. These items will be re-written or discarded based on the judgments of the evaluators. This process will be repeated until the author is confident, based on the evaluation data, that the SCQ has accurately assessed the domain of ASD and has an adequate balance of appropriate content. This new version of the SCQ will be called the SCQ-R.Study 2: Improved Reliability and Validity of the SCQ-RGoalThe goal of Study 2 is to evaluate the psychometric characteristics of the SCQ-R. In order to evaluate the psychometric characteristics of scores of the SCQ-R, the measure will be given to approximately 1000 caregivers of children with ASD to complete. One of the aims of this study is to collect data from a diverse sample of children with ASD and their caregivers. Data on item and total scores will be collected and reliability and validity calculations will be computed.

Sample Participants

This research proposal aims to collect SCQ-R data from a sample of 1000 children with ASD. Because this large sample would be hard to access in one area, several geographic locations will be used to collect data. Four research group sites will be chosen based on their access to a variety of clinical populations and geographic location. See table 2 for a description and geographic breakdown of the sample. All samples will be taken from urban areas in which a major University or research laboratory is located. This is one limitation of the study design; however efforts will be made by each research group to recruit from outside urban areas. One advantage to sampling from urban and surrounding rural areas is that more ethnic groups are typically represented.

Currently, it is thought that ASD occurs in all ethnic groups and across all cultures (Dyches, 2004). Sample demographics will aim to match those of the current United States census data. The target age group of children, 2-12 years old, will match the census data in terms of racial profile. Gender and socio-economic data will be collected but not controlled for. The sample will be split into two groups comprised of children with ASD diagnoses and non-ASD diagnoses. Eleven diagnostic groups will be represented in the overall sample. See Table 3 for a review. Diagnoses of the ASD sample will be based upon evaluation data from the ADOS, ADI-R, and expert clinical judgment based on DSM-IV criteria. Diagnoses of the non-spectrum group will be based upon prior psychiatric or medical records. Human Subjects IssuesA formal consent form will be approved by each sites Internal Review Board. Caregivers for participants will be required to give their written consent for their children to participate in the study. All child participants will also be required to give their written assent to participate. Caregivers must also complete a consent to obtain medical information so that any medical or diagnostic history for child participants can be gathered and processed.

There are no foreseeable risks to participating in this study. Participants will be informed that they may terminate their participation at any time with no penalty. There are also no direct benefits to participating in this study, although participants will be monetarily compensated for their time. Caregivers will also be provided with diagnostic reports from their childs assessment. Several measures will be taken to protect the confidentiality of participants. Paper data will be stored in locked cabinets and will be identifiable only by the participants study identification number. There will be no other identifying information in the file (i.e., name, gender, date of birth). All electronic data will be double password encrypted. No data will be released to parties outside the study unless the proper consents are completed.

Recruitment

Although recruitment procedures will vary from site to site, each group will aim to be as consistent as possible. Information about the study will be distributed at area clinics, hospitals and parent groups. Advocacy and service organizations will also be notified and educated about the aims and needs of the study. Some subjects from past research projects conducted at each site will be contacted and asked to participate. Participants who complete the study will receive $30 and travel expenses.Measures

Autism Diagnostic Interview-Revised (ADI-R; Rutter, LeCouteur, & Lord, 2003)The ADI-R is a standardized, semi-structured, investigator-based interview for caregivers of individuals suspected of having an autism spectrum disorder. The extended interview is designed to elicit information related to language and communication, social interaction, and repetitive behaviors and restricted interests. The ADI-R must be given by an experienced clinical interviewer and the informant must be familiar with the developmental history and current daily functioning of the individual being evaluated. The ADI-R is appropriate for any individual with a developmental level of at least 24 months. Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, & LeCouteur, 1994)The ADOS is a semi-structured assessment of social interaction, communication, play and imaginative use of materials for individuals suspected of having an autism spectrum disorder. Planned social presses are designed to elicit a range of social interactions and responses. The goal of the ADOS is to elicit a range of spontaneous behaviors in a standardized context. Four separate modules of the ADOS, each taking 30 minutes to 1 hour to administer, allow for the observation and quantification of behavior across a wide range of developmental levels. The examiner chooses the appropriate module based on the expressive language level of the individual in order to make judgments about social and communicative abilities independent from the effects of language delay.

Social Communication Questionnaire-Revised (SCQ; Rutter, Bailey, & Lord, 2003)The Social Communication Questionnaire, based on the Autism Screening Questionnaire (Berument et al., 1999), is a parent-report questionnaire used to screen for symptoms associated with autism spectrum disorder. The Lifetime Form provides a total score which is interpreted with reference to validated cut-off scores. Scores exceeding the cut-off score indicate that an individual may suffer from an autism spectrum disorder and suggests that a more comprehensive diagnostic and psychological assessment should be completed. The Current Form offers scores are related to symptomatology in the past three months and may be useful in tracking symptom change and treatment outcome; however it will not be used in this study. Social Reciprocity Scale (SRS; Constantino & Gruber, 2005)The SRS is a 65-item questionnaire completed by the parent/caregiver, which assesses recent functioning in five categories: social awareness, social cognition, social communication, social motivation, and autistic mannerisms. The SRS has been shown to be a sensitive screener for ASD in both clinical and educational settings. In addition, the SRS has been successfully used as a tool to identify sub-clinical symptoms associated with ASD as well as a measure of treatment progress.

Scales of Adaptive Behavior-Revised (SIB-R; Bruininks, Woodcock, Weatherman, & Hill, 1996)

The SIB-R is a standardized caregiver-report interview that assesses adaptive functioning. Behaviors across four domains (Motor, Social Interaction and Communication, Personal Living, and Community Living) are measured. The SIB-R is designed to assess skills needed for functioning independently across a variety of settings (i.e., home, school, social, work, and community) and measures social and adaptive behavior problems that may limit an individuals adjustment across these settings. The Broad Independence standard score provides a summary score of items across the four domains.

Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999)

The WASI is an abbreviated form of the Wechsler Adult Intelligence Scale- Third Edition and the Wechsler Intelligence Scale for Children- Third Edition. It was designed to provide a brief and reliable measure of intellectual abilities of individuals aged 6 to 89 years. The test consists of four separate subtests including Vocabulary, Similarities, Block Design, and Matrix Reasoning, which yields standardized, normed Verbal, Performance, and Full-Scale IQ scores. Procedure

Once families of children with ASD are recruited, they will be asked to come to one of the four geographic sites for a diagnostic evaluation. Children will be administered the ADOS and caregivers of the children will be administered the ADI-R. Diagnoses will be made based on the results of the ADOS, ADI-R, and expert clinical judgment of the diagnosticians based on DSM-IV criteria. All diagnostic measures will be videotaped for later review. Twenty percent of diagnostic interviews will be reviewed by a second clinician to ensure clinical consensus. All caregivers will also complete the SCQ-R. Following the diagnostic visit, inclusion decisions will be made. SCQ-R data for included cases will be entered into the study database.

Non-ASD families will be recruited and asked to send medical records confirming the childs diagnosis. Medical records will be reviewed and those cases for which diagnosis is clear will be included in the study. Caregivers of included children will be sent the SCQ-R via mail with return postage included.

Data Analysis

Test-retest Reliability

A random sample of 250 caregivers will be chosen from the each of the four sites in order to investigate the test-retest reliability of the SCQ-R. Test-retest reliability of the SCQ-R will assess whether the scores of the SCQ-R remain stable over time. This sample of caregivers will participate in two administrations of the SCQ-R within a window of 4 weeks. This short time frame was chosen to reduce the chance that differences in scores from time one to time two would be affected by child maturation and any significant change-producing experiences within the family. A shorter time frame was considered however, the 4-week period was chosen to minimize carryover or practice effects from the first assessment time. The second measure will be sent to caregivers via postal mail. The caregivers will be asked to complete the self-report measure and return it to the nearest site. Differences in assessment conditions should not affect the data as the measure is self-report and testing conditions should not have a significant influence on the reliability of the scores. Using the data from the random sample of 250 caregivers, test-retest reliability coefficients will be calculated and reported. Because the test-retest window is short and the SCQ-R aims to assess stable symptoms of ASD, it is expected that the correlation will be high.

Internal Consistency Reliability

Internal consistency reliability will be assessed using data from the initial 1000 caregivers who complete the SCQ-R. Internal consistency reliability is the extent to which different groupings of items produce consistent scores. There are several factors that affect the size of internal consistency reliability correlations. First, content homogeneity of test items influence the magnitude of internal consistency correlations. The content of items should reflect homogenous symptoms and behaviors associated with ASD. Second, the nature of the construct being assessed affects the magnitude of the internal consistency correlations. ASD is thought to be characterized by a triad of deficits which may be reflected in the multidimensional nature of the SCQ-R. Third, the length, or how many items a test is composed of affects the magnitude of the correlation. The SCQ-R will be a 40-item measure which, because of the principle of aggregation, should increase reliability. Taken together, it is hypothesized that the internal consistency of the SCQ-R will be moderate to high. The SCQ-R is a self-report measure answered in a dichotomous yes/no format. Thus, internal consistency reliability will be analyzed using the KR-20 statistic.

Other forms of reliabilityInter-rater reliability will not be assessed in this study, as the SCQ-R is a caregiver report measure. Further studies investigating the inter-rater reliability between primary caregivers may be warranted in the future. Alternate form reliability will not be assessed as there is no foreseeable need to create an additional form of the measure at this time. Finally, split-half reliability will not be calculated as it is not conceptually justified to calculate the extent to which scores on one half of the measure are consistent with scores on another half. Internal consistency reliability should suffice for the demonstration of item consistency.Construct Validity: Convergent, Discriminant, and Factorial Validity

Construct validity concerns whether a given measure, or operational definition, actually assesses the underlying conceptual variable, or construct, that the measure is intended to represent (Bryant, 2000, p. 111). When scores on a target measure relate to scores on other measures in the hypothesized way, it is generally accepted that scores mean what they are intended to mean. Convergent validity is one type of construct validity and addresses whether scores on a measure relate positively to scores on a measure of the same construct. A measure is said to have good convergent validity when positive and moderate to high correlations are found.

To address the convergent validity of the SCQ-R, scores from the SCQ-R will be correlated with scores from a measure of the same construct, the Social Reciprocity Scale (SRS). Both of these measures are caregiver-report measures that aim to tap the severity of ASD symptomatology. Both tools have been identified as screeners for the disorder and measures of symptom change over time. A subset of 250 caregivers will be randomly chosen from all four sites and administered the SRS. Evidence for the convergent validity of the SCQ-R will be demonstrated by moderate to high correlations with the SRS. It would be ideal to use a measure assessed by another method aside from self-report, as suggested by Campbell and Fiske (1959). However, as the SCQ-R is designed to be a screening measure for a possible diagnosis of ASD, caregivers of children typically provide the most accurate and thorough information regarding daily behavior.

Discriminant validity is another type of construct validity defined by Bryant (2000) as the degree to which multiple measures of different concepts are distinct (p. 114). In the assessment of discriminant validity, scores on a target measure are correlated with scores on a measure of a supposedly theoretically unrelated construct. Ideally, a measure is said to have good discriminant validity if correlations between two theoretically unrelated measures are very low or zero.

Discriminant validity of the SCQ-R will be assessed using the Scales of Adaptive Behavior-Revised (SIB-R) and the Wecshler Abbreviated Scale of Intelligence (WASI). Because individuals diagnosed with ASD demonstrate broad levels of adaptive and intellectual functioning, ASD symptomatology as assessed by a screening measure such as the SCQ-R must remain independent of other deficits. In order to assess the discriminative validity of the SCQ-R, a subset of 250 children and caregivers will be randomly selected from each of the four sites. Children will be administered the WASI and caregivers will be administered the SIB-R. Evidence for good discriminative validity of the SCQ-R will be demonstrated by a low to zero correlation with scores from the SIB-R and WASI.

A third type of construct validity, factorial validity, assesses whether a measure is organized the same way as the theoretical construct that the measure purports to assess. Exploratory factor analysis is used to identify underlying dimensions of a domain of functioning, as assessed by a particular measuring instrument (Floyd & Widaman, 1995, p. 286). In this approach, the measure is factor analyzed to reduce items to fewer subscales and determine the dimensionality of the construct as assessed by the measure.

SCQ-R data from the whole sample of 1000 caregivers will undergo an exploratory factor analysis. It is hypothesized that three factors relating to language deficits, social abnormalities, and stereotyped behaviors and restricted interests will emerge from the factor analysis. A principal axis extraction method will be used as it is predicted that less than 100% of the variance in items will be explained due to error. In addition, an oblique (oblimen) factor rotation will be chosen for the analysis due to the prediction that the factors may be overlapping (i.e., the factors have shared variance). In order to determine which factors will be kept, the Kaiser rule and scree plot methods will be utilized. Factor loadings and residuals will be examined to determine which items load on which factors.

Criterion-Related Validity

Criterion-related validity refers to how accurately an instrument predicts a well-accepted indicator of a given concept, or a criterion. If a test measures what it is supposed to measure, then it should be useful in predicting future, present, and past outcomes of interest (Bryant, 2000, p. 106). One type of criterion-related validity, concurrent validity, is the test of how strongly a target measure correlates with a criterion measure, or gold standard, given at the same time. It is hoped that if the SCQ-R demonstrates good psychometric characteristics, it can be used in place of the ADI-R as a screener for the disorder. Because the ADI-R and the ADOS are time-consuming measures that require significant clinical skill to administer and score, the SCQ-R would be a very useful tool if it could replace these measures for the screening of ASD in children. To assess the concurrent validity of the SCQ-R, scores from the ADI-R, a gold standard diagnostic measure for ASD, will be correlated with the SCQ-R. Good concurrent validity of the SCQ-R would be demonstrated with high correlations between SCQ-R and ADI-R scores.

Further criterion-related validity will be assessed by examining the hit rate, sensitivity, and specificity of the SCQ-R. Hit rate is the overall rate of detection in a sample. Streiner (2003) defines sensitivity as the proportion of people who have the attribute who are detected by the test (p. 210). In contrast, specificity is defined as the proportion of people without the attribute who are correctly labeled by the test (Streiner, 2000, p. 211). Overall, these diagnostic efficiency characteristics (Streiner, 2000) are a measure of how well an instrument makes correct and incorrect diagnostic decisions.

In order to evaluate these diagnostic efficiency characteristics for the SCQ-R, data from all 1000 caregivers will be used. Dichotomous diagnostic outcome (i.e., ASD diagnosis or non-ASD diagnosis) will be determined based on data from the ADOS, ADI-R, and expert clinical judgment. A range of cut-off scores with the associated sensitivity and specificity scores will be evaluated. The cutoff score that yields that maximum sensitivity and specificity of the SCQ-R will be chosen. Children whose SCQ-R scores meet or exceed this determined cut-off score will be recommended for further evaluation.

Other forms of Validity

Face and content validity was assessed in study one. The discriminative validity of the SCQ-R will not be assessed as this measure of validity does not apply to this measure. Because the SCQ-R is intended to be a screening measure for ASD, converget, discriminant, and criterion-related validity are more applicable to the measure.

[IS A DISCUSSION NEEDED FOR THIS GRANT PROPOSAL?]References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders- Text Revision (4th ed.). Washington, DC: Author.Berument, S.K., Rutter, M., Lord, C., Pickles, A., & Bailey, A. (1999). Autism Screening

Questionnaire: Diagnostic validity. The British Journal of Psychiatry, 175, 444-451.

Bruininks, R.H., Woodcock, R.W., Weatherman, R.F., & Hill, B.K. (1996). Scales of

Independent Behavior-Revised. Comprehensive Manual. Itasca, IL: Riverside

Publishing.Bryant, F. B. (2000). Assessing the validity of measurement. In L. G. Grimm & P. R. Yarnold

(Eds.). Reading and understanding MORE multivariate statistics (pp. 99-146).

Washington, DC: American Psychological Association.

Campbell, D.T., & Fiske, D. (1959). Convergent and discriminant validation by the multi-trait-multimethod matrix. Psychological Bulletin, 56, 81-105.Center for Disease Control: Autism Information Center-Frequently Asked Questions.

(2007). Retrieved October 21, 2008, from http://www.cdc.gov/ncbddd/autism/faq_prevalence.htm.Constantino, J. M., & Gruber, C. P. (2005). Social responsiveness scale (SRS). Los Angeles, CA: Western Psychological Services.Dyches, T.T., Wilder, L.K., Sudweeks, R.R., Obiakor, F.E. & Algozzine, B. (2004). Mutli-

cultural Issues in Autism. Journal of Autism and Developmental Disorders, 34(2), 211-222.

Filipek, P.A., Accardo, P.J., Baranek, G.T., Cook, Jr. E.H., Dawson, G., et al. (1999). The

screening and diagnosis of autistic spectrum disorders. Journal of Autism and Developmental Disorders, 29(6), 439-590.Floyd, F., & Widaman, K. F. (1995). Factor analysis on the development and refinement of

clinical assessment instruments. Psychological Assessment, 42, 286-299.

Haynes, S. N., Richard, D. C. S., & Kubany, E. (1995). Content validity in psychological

assessment: A functional approach to concepts and methods. Psychological Assessment,

42, 238-247.Lord, C., Risi, S., Lambrecht, L., Cook, Jr., E.H., Leventhal, B.L., DiLavore, P.C., et al. (2000).

The Autism Diagnostic Observation Schedule-Generic: A standard measure of social communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205-223.

Lord, C., Rutter, M., & LeCouteur, A. (1994). Autism Diagnostic Interview-Revised: A revised

version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24, 659-685.

Rutter, M. (2005). Incidence of autism spectrum disorders: Changes over time and their meaning. Acta Paediatrica, 94, 2-15.

Rutter, M., Bailey, A., & Lord, C. (2003). Social Communication Questionnaire-WPS

SCQ-WPS). Los Angeles, CA: Western Psychological Services.

Rutter, M., LeCouteur, A., & Lord, C. (2003). Autism Diagnostic Interview-Revised (ADI-R-WPS). Los Angeles, CA: Western Psychological Services.Streiner, D. L. (2003). Diagnosing tests: Using and misusing diagnostic and screening tests.

Journal of Personality Assessment, 81, 209-219.Wechsler, D. (1999). Wechsler Abbreviated Scale of Intelligence (WASI). San Antonio: Pearson Assessment.Table 1. SCQ norming sampleDiagnostic GroupNumber of subjects

PDD GroupAutism83

Atypical Autism49

Asperger syndrome16

Fragile X syndrome (no autism)7

Rett syndrome5

Non-PDD GroupConduct Disorder10

Developmental Language Disorder7

Mental Retardation15

Other psychiatric Diagnoses8

Table 2. Sample groups by location

Research GroupGeographic LocationExpected Sample Number

ISouthwest250

IISoutheast250

IIINorthwest250

IVNortheast250

Table 3. Clinical sample breakdown

ASD sampleAutism

Pervasive Developmental Disorder

Asperger Syndrome

Atypical Autism

Rett syndrome

Fragile X syndrome

Non-ASD sampleAttention Deficit Hyperactivity Disorder

Language Delay/Impairment

Conduct Disorder

Non-specific Mental Retardation

Other Clinical Diagnoses (affective disorders)