A Statewide Tiered System forScreening and Diagnosis of AutismSpectrum DisorderRebecca McNally Keehn, PhD, HSPP,a Mary Ciccarelli, MD,a Dorota Szczepaniak, MD,a Angela Tomlin, PhD,a Thomas Lock, MD,a

Nancy Swigonski, MD, MPH, MBAa,b

abstractAlthough autism spectrum disorder (ASD) can be reliably detected in thesecond year of life, the average age of diagnosis is 4 to 5 years. Limitations inaccess to timely ASD diagnostic evaluations delay enrollment in interventionsknown to improve developmental outcomes. As such, developing and testingstreamlined methods for ASD diagnosis is a public health and researchpriority. In this report, we describe the Early Autism Evaluation (EAE) Hubsystem, a statewide initiative for ASD screening and diagnosis in the primarycare setting. Development of the EAE Hub system involved geographicallytargeted provision of developmental screening technical assistance toprimary care, community outreach, and training primary care clinicians inASD evaluation. At the EAE Hubs, a standard clinical pathway wasimplemented for evaluation of children, ages 18 to 48 months, at risk for ASD.From 2012 to 2018, 2076 children were evaluated (mean age: 30 months;median evaluation wait time: 62 days), and 33% of children receiveda diagnosis of ASD. Our findings suggest that developing a tiered system ofdevelopmental screening and early ASD evaluation is feasible in a geographicregion facing health care access problems. Through targeted delivery ofeducation, outreach, and intensive practice-based training, large numbers ofyoung children at risk for ASD can be identified, referred, and evaluated in thelocal primary care setting. The EAE Hub model has potential fordissemination to other states facing similar neurodevelopmental health caresystem burdens. Implementation lessons learned and key system successes,challenges, and future directions are reviewed.

Autism spectrum disorder (ASD) isa complex neurodevelopmentaldisability characterized by impairmentsin social communication and thepresence of restricted and repetitivebehaviors affecting 1 in 54 children1

with lifetime costs exceeding $2.4million.2 Measurable behavioralsymptoms emerge in the first year oflife,3–5 and the diagnostic phenotypebecomes largely stable starting at 14months.6 Yet, nationally, the averageage of ASD diagnosis is 4 to 5 years,7,8

with children from lower income,

minority, and rural backgroundslagging further behind.9–11 A shortageof expert evaluators, time-intensiveevaluations, reimbursementconstraints, and provider hesitancy12

contribute to delays in referral and longevaluation wait times. The significantdelay between the emergence of ASDsymptoms and diagnosis means thatyoung children are missingopportunities for intervention at thetime of optimal neuroplasticity.13

Accordingly, developing and testingstreamlined methods for early ASD

aDepartment of Pediatrics, Indiana University School ofMedicine, Indianapolis, Indiana; and bDepartment of HealthPolicy and Management, Richard M. Fairbanks School ofPublic Health, Indiana University–Purdue UniversityIndianapolis, Indianapolis, Indiana

Drs Swigonski, Ciccarelli, and Lock conceptualizedand designed the study, contributed to designing thedata collection instruments, data collection, analysis,and interpretation, and reviewed and revised themanuscript for important intellectual content; DrMcNally Keehn contributed to designing the datacollection instruments, led the data analysis andinterpretation efforts, drafted the initial manuscript,and reviewed and revised the manuscript; DrsTomlin and Szczepaniak conceptualized and designedthe study and reviewed and revised the manuscriptfor important intellectual content; and all authorsapproved the final manuscript as submitted andagree to be accountable for all aspects of the work.

DOI: https://doi.org/10.1542/peds.2019-3876

Accepted for publication Apr 24, 2020

Address correspondence to Rebecca McNally Keehn,PhD, HSPP, Department of Pediatrics, IndianaUniversity School of Medicine, Health Informationand Translational Sciences Building, 410 West 10thStreet, Room 2027, Indianapolis, IN 46202.E-mail: mcnallyr@iu.edu

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online,1098-4275).

Copyright © 2020 by the American Academy ofPediatrics

FINANCIAL DISCLOSURE: The authors have indicatedthey have no financial relationships relevant to thisarticle to disclose.

To cite: McNally Keehn R, Ciccarelli M,Szczepaniak D, et al. A Statewide TieredSystem for Screening and Diagnosis of AutismSpectrum Disorder. Pediatrics. 2020;146(2):e20193876

PEDIATRICS Volume 146, number 2, August 2020:e20193876 SPECIAL ARTICLE by guest on August 21, 2020www.aappublications.org/newsDownloaded from

diagnosis is a public health andresearch priority.7,14

One important but recently debated15

method for early ASD detection isuniversal screening at 18 and24 months of age. Although theAmerican Academy of Pediatricsrecommends both universaldevelopmental16 and ASD17

screening, the US Preventive ServicesTask Force found insufficientsupportive evidence.18 Despite variedresults regarding the accuracy of ASDscreening,19–21 evidence indicatesthat the mean time to diagnosis issignificantly shorter for those who doscreen positive for ASD,19

highlighting the importance ofmaintaining this standard until morereliable measures are developed.

A second strategy for decreasing theage of ASD diagnosis is to improveaccess to diagnostic evaluations. Thefield has seen an emergence ofpromising research on trainingprimary care providers (PCPs),22–25

embedding behavioral healthproviders in primary care,26,27 andusing telemedicine-based diagnosticprocedures.28,29 Many studies employan evaluation model in whichdiagnosis is based on developmentalhistory as well as administration ofthe Screening Tool for Autism inToddlers and Young Children(STAT),30 a level 2 ASD screeningmeasure. Further research is neededto determine the feasibility of scalingthis approach to larger systems.

To address the significantneurodevelopmental needs of youngchildren across the state of Indiana,we developed an innovative tieredsystem of developmental screeningand diagnostic evaluation. Our goalwas to improve access to early ASDevaluation in children’s localcommunities and support enrollmentinto evidence-based interventions. Inthis report, we describe thedevelopment and scale-up of thestatewide Early Autism Evaluation(EAE) Hub system as well as

outcomes regarding 6 years of systemimplementation and sustainability.Lessons learned and key systemsuccesses, challenges, and futuredirections are offered for otherregions that may wish to adopt andexpand the EAE Hub model.

SETTING

At the time of initiation of the EAEHub system, Indiana lagged behindthe national average in the number ofchildren receiving standarddevelopmental screening; hada higher number of children at riskfor developmental, behavioral, orsocial delays;31 and had manycounties designated as MedicallyUnderserved Areas32 (seeSupplemental Table 4). Reliable state-level data on the average age of ASDdiagnosis in Indiana do not exist.However, an internal needsassessment indicated that ASD anddevelopmental delay were the 2 mostprevalent diagnoses served in theneurodevelopmental outpatientclinics of the state’s largest pediatrichospital and that most diagnoseswere made after children entered thepublic school system. Furthermore,this assessment revealed that, similarto nationally reported wait times of 6to 12 months,33,34 Indiana childrenwere waiting an average of 9 to12 months from referral toevaluation.

EAE HUB SYSTEM MODEL

The guiding framework of the EAEHub system is composed of 3 tiers ofservice: (1) children receive standarddevelopmental surveillance andscreening and ASD screening atprimary care well visits; (2) children,ages 18 to 48 months, identified as atrisk for ASD are referred to a localEAE Hub for ASD evaluation andcounseling on next steprecommendations; and (3) childrenwith complex or equivocal symptompresentation are referred forcomprehensive ASD evaluation at

a specialty diagnostic center. Aframework of quality improvement,coordination of care, communityengagement, and plannedcomanagement with the referringPCP overlays the system. The primaryEAE Hub team included an executivedirector (ie, academic pediatrician),project coordinator, and practiceliaisons. Notably, the team included 2parents of children withneurodevelopmental disabilities(including ASD), promoting theimportance of family-professional-community partnership in this effort.The development and scale-up of theEAE Hub system was funded bya combination of federal and stategrants, philanthropy, and individualcontracts with EAE Hub institutions(see Supplemental Information forfurther information).

Developmental Screening TechnicalAssistance and Outreach

Developmental screening technicalassistance to pediatric and familymedicine primary care practices wassequentially targeted aroundgeographic regions as each EAE Hubwas developed. A practice liaison andpediatrician visited practices toprovide education on (1)standardized developmental and ASDscreening procedures followingAmerican Academy of Pediatricspolicy;16,17 (2) training on the Agesand Stages Questionnaires, ThirdEdition (ASQ-3)35 and ModifiedChecklist for Autism in Toddlers,Revised with Follow-up (MCHAT-R/F)36 as well as kits at no cost; and (3)referral procedures for the local EAEHub and community services andresources. Follow-up technicalassistance occurred as needed.Geographically focused outreach tocommunity organizations includingearly intervention agencies, schoolcorporations, advocacy groups, andregional representatives of stateagencies was conducted to provideeducation on the EAE Hub system anddevelop partnerships to supportchildren and families.

2 MCNALLY KEEHN et al by guest on August 21, 2020www.aappublications.org/newsDownloaded from

EAE Hub System Development

The EAE Hub leadership teamreceived individualized and intensivetraining from the developers of theScreening Tools and ReferralTraining-Evaluation and Diagnosis(START-ED) model.25 The objectivesof START-ED are to provide primarycare pediatricians with a functionaland streamlined framework andassessment tools for the accuratediagnosis of young children with ASD.Included in the training were bothdidactic education on ASD evaluationand certification in administrationand scoring of the STAT, selectedbecause of its utility in theassessment of toddlers in the primarycare setting. This training was used toprepare the EAE Hub leadership teamto adapt the START-ED model for thedevelopment of the EAE Hub trainingcurriculum and clinical pathway.

The first EAE Hub site was piloted atan academic health center–affiliatedpediatric primary care clinic, allowingfor refinement of the model andtraining curricula. Additional EAEHub sites were selected on the basisof a 2-step process including (1) ananalysis of population distribution totarget geographic regions and (2)selection of pediatric primary carepractices in targeted regions withknown pediatric champions whowere actively engaged in earlychildhood initiatives. Given thegeneral assumption that pediatricianshave more formal expertise andexperience in atypical child behaviorand development, other types ofprimary care practices (eg, familymedicine) were not recruited as EAEHub sites.

The goal was for each EAE Hub to bea clinically and administratively self-sustaining site within the system. EAEHub sites ranged from large healthsystems to private pediatric practices,with commitment from theirgoverning leadership to providingthis service in their communities.Individual EAE Hubs negotiated

evaluation capacity, payment andrevenue, office space and supportstaff needs and related issues withtheir home organization. Althoughthere was no formal top-downoversight by the EAE Hub leadershipteam, consultation and ongoingsupport was provided to sitesthrough individualized technicalassistance and a monthly learningcollaborative webinar. Thecollaborative focused on didactictraining, case presentations, andpractice-based quality improvementdiscussions. An annual meeting washeld to review quality improvementdata, share practice updates, assesssystem needs and goals, and fosterrelationships to supportsustainability.

Each EAE Hub signed a memorandumof understanding to document formalcollaboration and agreement to (1)develop a clinical team, includinga pediatrician or nurse practitioner(NP) and nurse or medical assistant,ideally with the inclusion of anadministrative leader and carecoordinator to support follow-upcare; (2) participate in EAE Hubtraining; (3) implement the standardEAE Hub clinical pathway (seeTable 1); (4) collect quality indicatordata; and (5) participate in themonthly learning collaborative andannual meeting.

EAE Hub Training Curriculum

Each EAE Hub, including cliniciansand staff, participated in an on-sitemultiday intensive training on ASDevaluation. Training was provided byacademic faculty in generalpediatrics, developmental pediatrics,child psychology and/or psychiatry,and quality improvement science.Included in the didactic curriculumwere education on developmentalscreening, structured developmentalhistory and interviewing techniques(including the assessment ofDiagnostic and Statistical Manual ofMental Disorders, Fifth Edition [DSM-5] ASD symptoms), medical andpsychological differential diagnosisand common comorbid concerns,communication skills for delivery ofdiagnosis, and current evidenceregarding ASD interventions. Trainingon billing and coding and practicequality improvement was provided toEAE Hub clinicians and pertinentpractice staff. Regional communityagencies were invited to share localresources, and families of childrenwith neurodevelopmental disabilitiesjoined the training as “faculty for theday” to share their experiences withdiagnosis and navigating systems andservices. Clinical practicum trainingincluded in vivo practice andsupervision on all steps of the clinicalpathway for up to 6 evaluations ofchildren with (1) typical

TABLE 1 Components of the EAE Hub Standard Clinical Pathway: Evaluation Procedures and ToolsImplemented During the EAE Hub Evaluation

Evaluation Procedures

Review and/or administration of ASQ-3 and MCHAT-R/FDiagnostic interview, including assessment of DSM-5 ASD criteria and medical history, with

caregiver(s)Physical examinationAdministration of STATIntegration of data including screening measures, developmental history and DSM-5 ASD interview,

and STAT results to formulate a clinical diagnosisDiagnostic feedback with caregiver(s), including the sharing of clinical recommendations and local

resourcesDissemination of clinical evaluation report to the PCP, including the recommended next steps for

care management; further consultative follow-up to the PCP is provided as needed and requested

All evaluation procedures are conducted by the EAE Hub clinician (and team support staff) unless otherwise noted. EAEHub teams were trained to administer ASQ-3 and MCHAT-R/F as part of the evaluation process; however, if these measureswere completed within 3 months of the EAE Hub evaluation and provided by the referring PCP, they were not alwaysrepeated at the time of evaluation.

PEDIATRICS Volume 146, number 2, August 2020 3 by guest on August 21, 2020www.aappublications.org/newsDownloaded from

development, (2) confirmed diagnosisof ASD, and (3) referral concern forASD. Training faculty providedlearners with written feedback,including ratings of performanceduring observed practicum sessions.Measures of trainee satisfaction wereused to guide revisions of thecurriculum over time.

EAE Hub Clinical Pathway

The EAE Hub model was developedfollowing the principles of the START-ED model,25 whereby clinicians areprovided with training on a standardclinical evaluation protocol andassessment tools for diagnosis of ASDin toddlers. In contrast with standardcomprehensive ASD evaluation (ie,which often includes labor-, training-,and cost-intensive diagnostic toolssuch as Autism Diagnostic ObservationSchedule, Second Edition,37 andAutism Diagnostic Interview-Revised38), the EAE Hub clinicalpathway specifies a brief evaluationprotocol designed to be completed ina 90-minute primary care office visit.Evaluation components includea review of ASQ-3 and MCHAT-R/F,diagnostic interview to solicit DSM-5ASD symptoms and medical history,physical examination, andadministration of the STAT (seeTable 1). The STAT, a level 2screening tool originally developedfor use in children ages 24 to 35months, has been shown to have goodpsychometric properties39 (ie,sensitivity = 1.0; specificity = 0.85;positive predictive value = 0.86;negative predicative value = 0.92).Additional research has revealedpromising utility for an extended agerange of 14 to 47 months.26,40,41 AtEAE Hub system initiation, an agerange of 18 to 42 months wastargeted. However, over time, theage range was expanded up to48 months on the basis of availableSTAT guidelines (eg, includingthe use of alternative age-basedscoring procedures39–42) as wellas clinician feedback regardingcomfort and desire to serve a broader

group of children for which thestandard clinical pathway wasappropriate.

EAE Hub Data Collection

EAE Hubs collected and submittedde-identified data for each evaluationvia standardized paper-based visitsummary sheets or direct entry intoan online database. To minimize datacollection burden on EAE Hubs andensure Health Insurance Portabilityand Accountability Act of 1996compliance, individual demographicinformation was not collected (seeSupplemental Information andSupplemental Table 4 for county- andstate-level demographic information).Data were stored in a secure databaseand analysis was completed with IBMSPSS Statistics, version 26 (IBM SPSSStatistics, IBM Corporation).

EAE HUB SYSTEM OUTCOMES:2012–2018

From 2012 to 2018, the EAE teamprovided technical assistance ondevelopmental screening to 193primary care practices composed of798 clinicians (ie, 82% physicians;17% NPs) and their staff. Outreachefforts also included presentations to136 community organizations,including early intervention agencies(n = 31), schools (n = 38), autismintervention agencies (n = 7), andlocal community organizations (n =60). Medical presentations weredelivered at 73 events (see Fig 1A).Education and outreach efforts wereconducted in 76% of Indiana counties(see Fig 2).

Twelve EAE Hubs were developed inpediatric primary care practicesacross the state of Indiana (see Fig 2),

FIGURE 1A, Developmental screening technical assistance, community outreach, and medical educationefforts (2012–2018). B, EAE Hub completed evaluations (2012–2018). a Indicates 1 EAE Hub wasinitiated. b Indicates 5 EAE Hubs were initiated. c Indicates 3 EAE Hubs were initiated. d Indicates 2 EAEHubs were initiated.

4 MCNALLY KEEHN et al by guest on August 21, 2020www.aappublications.org/newsDownloaded from

representing 8 health systems. EAEHub training was delivered to 90individuals, including 30 clinicians(ie, 20 physicians, 10 NPs) and 53support staff (ie, administrators,medical assistants, billing specialists,and social workers). Over 6 years,there was a 92% Hub retention rate;1 EAE Hub exited the collaborativebecause of personnel turnover.Engaging the EAE Hub teams inpartnership, intensive training, andmonthly continuing education hassupported excellent retention of Hubsites and sustainability of the system.

Over 6 years, a total 2076 childrenwere evaluated across the EAE Hubsystem. Of 706 PCPs making referrals,36% had received developmentalscreening technical assistance fromthe EAE Hub team, suggesting thateducational efforts spread beyondthose who received direct technicalassistance. EAE Hub evaluationsincreased over time as Hubs becameestablished in communities and moresites were added across the system(see Fig 1B). By 2018, ∼72% ofexpected ASD diagnoses in their

respective regions and 15% ofexpected ASD diagnoses statewidewere made at EAE Hubs (seeSupplemental Information andSupplemental Fig 3).

Thirty-three percent of the childrenevaluated received a diagnosis of ASD(see Table 2). In 8% of cases,evaluation results were equivocal,and the EAE Hub clinician was unableto make a definitive determination ofASD diagnostic status. Although theEAE Hub model mandated referral toa specialty diagnostic center forequivocal cases, limitations in ourcapacity for follow-up data collectionprecluded confirmation that thesechildren received further evaluation,and, thus, definitive diagnostic statusfor this group of children is unknown.

Of all children evaluated (ie,regardless of ASD status), 72% metdiagnostic criteria for globaldevelopmental delay (GDD), definedas delays in $2 developmentaldomains on the basis of ASQ-3 and/orclinical judgment. Sixteen percent ofchildren met neither criteria for ASDnor GDD; 89% of these children wereidentified as having one or moredevelopmental, behavioral, or medicalconcern warranting follow-up orintervention (see SupplementalTable 5). Together, these findingssuggest that even those not diagnosedwith ASD were likely to benefit fromevaluation.

Across the EAE Hubs, the mean age atevaluation was 30 months (seeTable 3), significantly less than thenational average of 4 to 5 years ofage8 and consistent with existingreports of community-baseddiagnostic models.24–27 Compared tohistorical 9- to 12-month wait timesestimated across tertiary outpatientclinics, the median latency fromreferral to EAE Hub evaluation (ie,wait time) was 62 days. This findingof improved access throughimplementation of ASD evaluation inthe primary care setting has beenfound across several smallerstudies.26–29 Decreasing wait timesfor evaluation services provided inchildren’s local communities has theimportant potential of increasingaccess to early intervention andsupportive services.

To determine if frequency of ASDdiagnostic outcome, age at evaluation,and wait time differed across EAEHub sites, a series of exploratoryanalyses were conducted. For theseanalyses, 11 of 12 EAE Hubs wereincluded; 1 EAE Hub developed in2018 was excluded from analysisgiven the small number of evaluationsconducted (n = 4). The frequency ofASD diagnosis varied significantly(P , .001) across site (see Table 2).There was also a significantdifference in age at evaluation (P ,.001) and wait time (P , .001) acrosssites (see Table 3). Wait time at theEAE Hub within the Indiana

FIGURE 2Gray shaded areas depict Indiana counties inwhich developmental screening technical as-sistance, community outreach, and/or medicaleducation outreach occurred. Red stars depictthe location of EAE Hubs.

TABLE 2 Number and Percentage of Children Diagnosed With ASD in the EAE Hubs

ASD Diagnosis No. (%)

Total EAE Hub evaluations (N = 2076) 691 (33.3)Hub 1 (n = 429) 128 (29.8)Hub 2 (n = 86) 18 (20.9)Hub 3 (n = 184) 36 (19.6)Hub 4 (n = 85) 39 (38.2)Hub 5 (n = 151) 49 (32.5)Hub 6 (n = 464) 169 (36.4)Hub 7 (n = 389) 147 (37.8)Hub 8 (n = 150) 57 (38.0)Hub 9 (n = 25) 11 (44.0)Hub 10 (n = 31) 8 (25.8)Hub 11 (n = 61) 26 (42.6)

Percent is based on the total number of children evaluated in the EAE Hubs from 2012 to 2018 (N = 2076).

PEDIATRICS Volume 146, number 2, August 2020 5 by guest on August 21, 2020www.aappublications.org/newsDownloaded from

University School of Medicine’sacademic outpatient clinic (ie, Hub 6)was significantly longer than theaverage at all other EAE Hubscombined (P , .001).

Findings regarding differences in thenumber of children diagnosed withASD, age at evaluation, and wait timeacross EAE Hub sites are notable.Previous research has documentedthe impact of health care providerknowledge and behavior on referralsfor ASD evaluation.43–47 It is likelythat referring PCPs may have variableexperience with the heterogeneousASD phenotype as well as differentthresholds of concern that promptreferral, both contributing todifferences in age and diagnosticprofiles of children evaluated in theEAE Hubs. Additionally, somereferring PCPs may use the EAE Hubsystem more broadly (ie, for childrenwithout clear ASD symptoms but withother developmental and/orbehavioral concerns), thus skewingthe number of children diagnosed

with ASD at some sites. Hub-specificfactors such as catchment areapopulation size and site capacity arelikely to account for variablewait times.

Although the development of the EAEHub system represents a significantadvance in improving access to timelyASD evaluation, such statewideefforts have many challenges, andinterpretation of our outcomes mustbe considered in the context ofseveral limitations. First, although ourclinical pathway was developed froma well-accepted evaluation protocol25

and involved intensive practice-basedtraining and performance feedbackfrom experts, there was noindependent ASD evaluation fromwhich to evaluate diagnostic accuracyor determine diagnosis for those withequivocal diagnosis. We must alsounderstand how child and familysociocultural factors affect access andoutcomes through collection ofdemographic data. Together, theseefforts will be critical in further

evaluating the validity of sucha statewide system. In addition,although more young children arenow receiving ASD screening andevaluation in their local communities,we did not have reliable statewidebaseline measures from which toevaluate system impact. We alsocannot draw conclusions regardingthe rate of referral for evaluation forthose children who screen positivefor ASD or determine if an earlierdiagnosis results in earlier entry intointervention and how this may impactchild outcome.

LESSONS LEARNED FROM EAE HUBSYSTEM IMPLEMENTATION

To our knowledge, this is the largestpublished report on the developmentand implementation of statewidesystem for early ASD screening anddiagnosis to date. We offer lessonslearned and key system successes,challenges, and future directions forother regions facing similarneurodevelopmental access issuesthat may wish to adopt and expandthe EAE Hub model.

Develop a CommittedInterdisciplinary Planning andLeadership Team

A key ingredient for EAE Hub systemsuccess was committedinterdisciplinary planning andongoing leadership. The Departmentof Pediatrics prioritized this pediatricpublic health need and dedicatedtime, resources, and faculty expertiseto this effort. An internal needsassessment drove systemdevelopment planning, and aninterdisciplinary team ofsubspecialists, family advocates, andgeneral pediatricians met weekly (at7 AM) to carefully construct anddebate a statewide approach toimproving access to ASD evaluation.Faculty committed effort above andbeyond their existing clinical andacademic duties to participate inbroad leadership workgroups that

TABLE 3 Age at Evaluation and Wait Time for EAE Hub Evaluations

Mean Median Range SD

Age, moAll EAE Hubs (N = 2059) 30.3 30.0 34 6.7Hub 1 (n = 428) 29.7 30.0 26 6.8Hub 2 (n = 84) 28.5 28.0 24 6.9Hub 3 (n = 183) 30.7 30.0 34 7.1Hub 4 (n = 102) 30.2 30.0 24 6.8Hub 5 (n = 151) 30.5 30.0 28 7.1Hub 6 (n = 464) 30.2 30.0 24 5.4Hub 7 (n = 389) 32.0 32.0 28 6.6Hub 8 (n = 150) 29.5 29.0 30 7.2Hub 9 (n = 22) 30.1 30.5 22 6.5Hub 10 (n = 31) 27.5 26.0 23 5.6Hub 11 (n = 55) 28.96 29.0 23 6.3

Wait time, dAll EAE Hubs (N = 1674) 76.82 62.0 341 55.9Hub 1 (n = 403) 65.1 54.0 324 51.4Hub 2 (n = 13) 30.4 27.0 71 17.4Hub 3 (n = 94) 37.7 31.5 220 31.0Hub 4 (n = 43) 61.6 40.0 316 60.5Hub 5 (n = 142) 62.8 57.0 184 32.1Hub 6 (n = 410) 122.9 123.5 337 61.6Hub 7 (n = 366) 72.5 77.0 200 35.9Hub 8 (n = 143) 42.8 30.0 316 47.0Hub 9 (n = 16) 91.3 74.0 145 46.5Hub 10 (n = 29) 48.9 44.0 183 48.9Hub 11 (n = 15) 32.13 29.0 49 12.7

Total EAE Hub evaluations (2012–2018) = 2076; the data reported above reflect some missing values for both age and waittime. It should be noted that wait times were not consistently reported by all EAE Hubs for the duration of the project.

6 MCNALLY KEEHN et al by guest on August 21, 2020www.aappublications.org/newsDownloaded from

determined the scope, process, andfunding for system development.

Invest in Local Partnerships atSystem Start-up

From the beginning, we aimed tocultivate strong partnerships withEAE Hub clinicians and theirorganizations, and these relationshipshave been the foundation ofsustainability. Over 6 years ofimplementation, one of our mostsignificant insights has been theimportance of identifying a pediatricchampion (typically a medical doctorin a leadership role) at each Hub site.This champion served to coordinatesite-specific EAE Hub services andadvocate for the importance of thesystem at the organization andcommunity level. In addition, thesechampions were invaluable in fuelingconnections between the EAE Hub,our central leadership team, and localcommunity advocates andorganizations. Having relationalconnections in and acrosscommunities is necessary to mosteffectively support children and theirfamilies in accessing needed services.

Maintain System Engagement andMomentum

A crucial ingredient to systemengagement and sustainability wasfostering collaboration through thelearning collaborative. Through thesewebinars, our central leadership teamfocused on nurturing shared prideand ownership of the system amongall EAE Hub teams. Sharing qualityimprovement data underscored thesignificant impact that each team andour collective system made. Thewebinars also allowed for regularproblem solving of issues such aschallenging clinical cases, insurancereimbursement, and servicenavigation. The central leadershipteam was able to keep a pulse onsystem quality and managementissues that required follow-up.

Develop an Infrastructure forTraining and Personnel Management

Providing training in ASD evaluationand ongoing maintenance of skills toa large group of PCPs requiressignificant investment. The selectionof faculty with requisite expertise andfunding of their time and travel to theEAE Hubs to provide on-site trainingproved challenging. Over time, EAEHub clinicians and staff, includingthose involved in supporting service,billing, and quality improvementefforts, retired from or left theirinstitutions. Turnover in personnelcreated disruption in systemoperations and capacity as well asdemands for training new teammembers. Periodic formal continuingeducation and reevaluation ofdiagnostic accuracy and fidelity to theEAE Hub model is critical for qualityassurance. Given time and fundingconstraints, we were not able toinvest in these important efforts fromsystem inception, although wesuggest that others who undertakeadaptation of this model strive tobuild and fund this infrastructurefrom the start.

Develop Processes to MeasureImpact

The development of initial systemoutcome measures was focusedaround goals of decreasing evaluationwait times and lowering the age ofASD diagnosis. Yet there are criticaldownstream impacts that must bemeasured to further understand thesignificance of this statewide system.For example, understanding whetherearly diagnosis leads to cascadingeffects (including earlier enrollmentin intervention, improved childoutcomes, reduced burden on theeducational system, and loweredlifetime costs) will be critical tofurther system dissemination andfunding. Collaboration with statewideagencies (including Birth-to-Threeprograms, public school systems,intervention agencies, and health carefinancing organizations) is one

method for systematically collectingthese types of data.

Support Fiscal Sustainability

There has been inconsistentinsurance reimbursement for ASDevaluation services, which strainsindividual EAE Hub organizations. Inaddition, payment for 90-minute EAEHub evaluations is often lower thanwhat would be provided for a highernumber of routine office visits.Clinician productivity andreimbursement requirements vary byorganization type, and, althoughprimary care clinics set within largerhealth networks may be able to bearsome financial burden in support ofaddressing a critical pediatric need,this is unlikely to be the case forsmaller practices.

An additional hurdle has beennavigating significant changes in whatinsurers deem a valid ASD evaluation.For example, some insurers are nowmandating inclusion of specificassessment tools (ie, AutismDiagnostic Observation Schedule,Second Edition) to authorize ASDintervention services. Yet the use ofthese tools requires expertdiagnosticians and, thus, contributesto problems with access. Our centralleadership team has worked directlywith Medicaid and other insurers toprovide education about the EAE Hubsystem and developed standarddocumentation regarding the ASDclinical pathway, including evidencefor how the model is aligned withrecommended standards forevaluation. Standardization of systemprocesses, including adherence tospecified formats for evaluationreports and insurance appeals, aswell as deepening partnerships withinsurers is likely to benefit thiscontinued effort.

Avoid Excess Burden on EAE Hubs

In the current health care climate, thetime and capacity of primary careclinicians and their teams arecontinuously stretched. For most EAE

PEDIATRICS Volume 146, number 2, August 2020 7 by guest on August 21, 2020www.aappublications.org/newsDownloaded from

Hub clinicians, efforts related tosystem participation (includingservice delivery) account for ,10%of their practice, and, as such, theymust balance demands from theirmany competing roles. Participationin this statewide effort without anydirect funding (eg, for additionalsupport staff or indirect costsassociated with office space, patientbilling, clinician and staff trainingtime, and data collection efforts)creates a significant burden onindividual EAE Hub clinicians andtheir organizations. One potentialavenue to reduce burden may be todevelop a shared infrastructure ofsupport with a state department ofhealth or similar agency. Thispartnership could potentiallyallow for the use of funding tosupport the work of individual EAEHubs and build capacity for thecollection of comprehensivelongitudinal outcome data to evaluatesystem impact. These efforts must bepriorities for ensuring sustainability,advancing rigorous systemevaluation, and improving pediatricpopulation health.

CONCLUSIONS

Developing a tiered system ofdevelopmental screening and earlyASD evaluation is feasible in

a geographic region facing significanthealth care access problems. Throughtargeted delivery of developmentalscreening technical assistance,community outreach, medicaleducation, and intensive practice-based training, large numbers ofyoung children at risk for ASD can beidentified and evaluated in the localprimary care setting. Althoughfurther rigorous testing of the EAEHub system is warranted, ourfindings suggest that this model haspotential for further expansion anddissemination to other states facingsimilar neurodevelopmental healthcare system burdens. Futuredirections must include evaluation ofdiagnostic accuracy of the system, aneffort that is in progress, as well asmeasurement of provider and familysatisfaction, child interventionenrollment and outcomes, and cost ofimplementation.

ACKNOWLEDGMENTS

We thank Angela Paxton, MaryDelaney, Mary Jo Paladino andMaureen McAteer, DO, for theirinvaluable contributions to thisproject. We are grateful to theclinicians and supporting staff at eachof the EAE Hubs for theircollaboration, service to children andfamilies, and submission of data.

Without local pediatric championswho are committed to providingIndiana’s children with this criticalcommunity-based service, theimplementation of the EAE Hubswould not have been possible.Finally, we thank the IndianaChapter of the American Academyof Pediatrics for their partnershipin this work.

ABBREVIATIONS

ASD: autism spectrum disorderASQ-3: Ages and Stages Question-

naire, Third EditionDSM-5: Diagnostic and Statistical

Manual of Mental Disor-ders, Fifth Edition

EAE: Early Autism EvaluationGDD: global developmental delayMCHAT-R/F: Modified Checklist

for Autism in Tod-dlers, Revised withFollow-up

NP: nurse practitionerPCP: primary care providerSTART-ED: Screening Tools and

Referral Training-Evaluation andDiagnosis

STAT: Screening Tool for Autism inToddlers and YoungChildren

FUNDING: Supported by the Riley Children’s Foundation, Kiwanis Indiana Three Wishes Campaign, Linking Actions in Unmet Needs in Children’s Health, Indiana State

Department of Health (Community Integrated Systems of Service grant), and Early Childhood Comprehensive Systems Collaborative Innovation and Improvement

Network.

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

REFERENCES

1. Maenner MJ, Shaw KA, Baio J, et al.Prevalence of autism spectrumdisorder among children aged 8 years -autism and developmental disabilitiesmonitoring network, 11 sites, UnitedStates, 2016. [published correctionappears in MMWR Morb Mortal WklyRep. 2020;69:503]. MMWR SurveillSumm. 2020;69(4):1–12

2. Buescher AV, Cidav Z, Knapp M, MandellDS. Costs of autism spectrum disordersin the United Kingdom and the UnitedStates. JAMA Pediatr. 2014;168(8):721–728

3. Barbaro J, Dissanayake C. Autismspectrum disorders in infancyand toddlerhood: a review of the

evidence on early signs, earlyidentification tools, and earlydiagnosis. J Dev Behav Pediatr. 2009;30(5):447–459

4. Miller M, Iosif A-M, Hill M, Young GS,Schwichtenberg AJ, Ozonoff S.Response to name in infants developingautism spectrum disorder:

8 MCNALLY KEEHN et al by guest on August 21, 2020www.aappublications.org/newsDownloaded from

a prospective study. J Pediatr. 2017;183:141.e1-146.e1

5. Paterson SJ, Wolff JJ, Elison JT, et al;Infant Brain Imaging Study Network.The importance of temperament forunderstanding early manifestations ofautism spectrum disorder in high-riskinfants. J Autism Dev Disord. 2019;49(7):2849–2863

6. Pierce K, Gazestani VH, Bacon E, et al.Evaluation of the diagnostic stability ofthe early autism spectrum disorderphenotype in the general populationstarting at 12 months. [publishedcorrection appears in JAMA Pediatr.2019;173(8):801]. JAMA Pediatr. 2019;173(6):578–587

7. Zwaigenbaum L, Penner M. Autismspectrum disorder: advances indiagnosis and evaluation. BMJ. 2018;361:k1674

8. Baio J, Wiggins L, Christensen DL, et al.Prevalence of autism spectrumdisorder among children aged 8 years -autism and developmental disabilitiesmonitoring network, 11 sites, UnitedStates, 2014. [published correctionsappear in MMWR Morb Mortal WklyRep. 2018;67(19):564 and MMWR MorbMortal Wkly Rep. 2018;67(45):1280].MMWR Surveill Summ. 2018;67(6):1–23

9. Mandell DS, Listerud J, Levy SE, Pinto-Martin JA. Race differences in the ageat diagnosis among Medicaid-eligiblechildren with autism. J Am Acad ChildAdolesc Psychiatry. 2002;41(12):1447–1453

10. Mandell DS, Novak MM, Zubritsky CD.Factors associated with age ofdiagnosis among children with autismspectrum disorders. Pediatrics. 2005;116(6):1480–1486

11. Mandell DS, Wiggins LD, Carpenter LA,et al. Racial/ethnic disparities in theidentification of children with autismspectrum disorders. Am J PublicHealth. 2009;99(3):493–498

12. Gordon-Lipkin E, Foster J, Peacock G.Whittling down the wait time: exploringmodels to minimize the delay frominitial concern to diagnosis andtreatment of autism spectrum disorder.Pediatr Clin North Am. 2016;63(5):851–859

13. Courchesne E, Pramparo T, GazestaniVH, Lombardo MV, Pierce K, Lewis NE.

The ASD living biology: from cellproliferation to clinical phenotype. MolPsychiatry. 2019;24(1):88–107

14. Interagency Autism CoordinatingCommittee. 2016–2017 InteragencyAutism Coordinating Committeestrategic plan for autism spectrumdisorder. 2017. Available at: https://iacc.hhs.gov/publications/strategic-plan/2017/strategic_plan_2017.pdf. AccessedApril 8, 2020

15. Broder-Fingert S, Feinberg E, SilversteinM. Improving screening for autismspectrum disorder: is it time forsomething new? Pediatrics. 2018;141(6):e20180965

16. Lipkin PH, Macias MM; Council onChildren with Disabilities, Section onDevelopmental and BehavioralPediatrics. Promoting optimaldevelopment: identifying infants andyoung children with developmentaldisorders through developmentalsurveillance and screening. Pediatrics.2020;145(1):e20193449

17. Hyman SL, Levy SE, Myers SM; Councilon Children with Disabilities, Section onDevelopmental and BehavioralPediatrics. Identification, evaluation,and management of children withautism spectrum disorder. Pediatrics.2020;145(1):e20193447

18. Siu AL, Bibbins-Domingo K, GrossmanDC, et al; US Preventive Services TaskForce (USPSTF). Screening for autismspectrum disorder in young children:US Preventive Services Task Forcerecommendation statement. JAMA.2016;315(7):691–696

19. Guthrie W, Wallis K, Bennett A, et al.Accuracy of autism screening in a largepediatric network. Pediatrics. 2019;144(4):e20183963

20. Stenberg N, Bresnahan M, Gunnes N,et al. Identifying children with autismspectrum disorder at 18 months ina general population sample. PaediatrPerinat Epidemiol. 2014;28(3):255–262

21. Toh T-H, Tan VW-Y, Lau PS-T, Kiyu A.Accuracy of Modified Checklist forAutism in Toddlers (M-CHAT) indetecting autism and otherdevelopmental disorders in communityclinics. J Autism Dev Disord. 2018;48(1):28–35

22. Mazurek MO, Brown R, Curran A, Sohl K.ECHO autism. Clin Pediatr (Phila). 2017;56(3):247–256

23. Mazurek MO, Curran A, Burnette C, SohlK. ECHO autism STAT: accelerating earlyaccess to autism diagnosis. J AutismDev Disord. 2019;49(1):127–137

24. Swanson AR, Warren ZE, Stone WL,Vehorn AC, Dohrmann E, Humberd Q.The diagnosis of autism in communitypediatric settings: does advancedtraining facilitate practice change?Autism. 2014;18(5):555–561

25. Warren Z, Stone W, Humberd Q. Atraining model for the diagnosis ofautism in community pediatric practice.J Dev Behav Pediatr. 2009;30(5):442–446

26. Hine JF, Allin J, Allman A, et al.Increasing access to autism spectrumdisorder diagnostic consultation inrural and underserved communities:streamlined evaluation within primarycare. J Dev Behav Pediatr. 2019;41(1):16–22

27. Hine JF, Herrington CG, Rothman AM,et al. Embedding autism spectrumdisorder diagnosis within the medicalhome: decreasing wait times throughstreamlined assessment. J Autism DevDisord. 2018;48(8):2846–2853

28. Juárez AP, Weitlauf AS, Nicholson A,et al. Early identification of ASD throughtelemedicine: potential value forunderserved populations. J Autism DevDisord. 2018;48(8):2601–2610

29. Stainbrook JA, Weitlauf AS, Juárez AP,et al. Measuring the service systemimpact of a novel telediagnostic serviceprogram for young children withautism spectrum disorder. Autism.2019;23(4):1051–1056

30. Stone WL, Coonrod EE, Ousley OY. Briefreport: screening tool for autism intwo-year-olds (STAT): development andpreliminary data. J Autism Dev Disord.2000;30(6):607–612

31. Data Resource Center for Child &Adolescent Health. National survey ofchildren with special health care needs:NS-CSHCN 2009/10. Data query from theChild and Adolescent HealthMeasurement Initiative. Available at:www.childhealthdata.org. AccessedApril 8, 2020

32. Indiana State Department of Health.Indiana Medically Underserved Areas

PEDIATRICS Volume 146, number 2, August 2020 9 by guest on August 21, 2020www.aappublications.org/newsDownloaded from

and Populations: April, 2014. Availableat: https://www.in.gov/isdh/files/mua_0414.pdf. Accessed January 21, 2020

33. Austin J, Manning-Courtney P, JohnsonML, et al. Improving access to care atautism treatment centers: a systemanalysis approach. Pediatrics. 2016;137(suppl 2):S149–S157

34. Penner M, Anagnostou E, Ungar WJ.Practice patterns and determinants ofwait time for autism spectrum disorderdiagnosis in Canada. Mol Autism. 2018;9:16

35. Squires J, Bricker D, Twombly E, et al.Ages and Stages Questionnaire, 3rd ed.Baltimore, MD: Brookes Publishing;2009

36. Robins DL, Casagrande K, Barton M,Chen C-M, Dumont-Mathieu T, Fein D.Validation of the Modified Checklist forAutism in Toddlers, Revised with Follow-up (M-CHAT-R/F). Pediatrics. 2014;133(1):37–45

37. Lord C, Rutter M, Dilavore P, Risi S,Gotham K, Bishop S. Autism DiagosticObservation Schedule, 2nd ed.

Torrence, CA: Western PsychologicalServices; 2012

38. Rutter M, Lecouteur A, Lord C. AutismDiagnostic Interview-Revised. Torrance,CA: Western Psychological Services;2003

39. Stone WL, Coonrod EE, Turner LM,Pozdol SL. Psychometric properties ofthe STAT for early autism screening.J Autism Dev Disord. 2004;34(6):691–701

40. Stone WL, McMahon CR, Henderson LM.Use of the Screening Tool for Autism inTwo-year-olds (STAT) for children under24 months: an exploratory study.Autism. 2008;12(5):557–573

41. Foss-Feig JH, Stone W. Use of the STATas an autism screen for children 36–47months. In: International Meeting forAutism Research; May 4, 2007; Seattle,WA

42. Stone W, Ousley OY. Screening Tool forAutism in Toddlers and Young Children:User’s Manual. Nashville, TN: Treatmentand Research Institute for AutismSpectrum Disorders; 2008

43. Carbone PS, Murphy NA, Norlin C, AzorV, Sheng X, Young PC. Parent andpediatrician perspectives regarding theprimary care of children with autismspectrum disorders. J Autism DevDisord. 2013;43(4):964–972

44. Golnik A, Ireland M, Borowsky IW.Medical homes for children withautism: a physician survey. Pediatrics.2009;123(3):966–971

45. Guerrero AD, Rodriguez MA, Flores G.Disparities in provider elicitation ofparents’ developmental concerns for USchildren. Pediatrics. 2011;128(5):901–909

46. Rea KE, Armstrong-Brine M, Ramirez L,Stancin T. Ethnic disparities in autismspectrum disorder screening andreferral: implications for pediatricpractice. J Dev Behav Pediatr. 2019;40(7):493–500

47. Zuckerman KE, Lindly OJ, Sinche BK.Parental concerns, provider response,and timeliness of autism spectrumdisorder diagnosis. J Pediatr. 2015;166(6):1431.e1-1439.e1

10 MCNALLY KEEHN et al by guest on August 21, 2020www.aappublications.org/newsDownloaded from

originally published online July 6, 2020; Pediatrics Thomas Lock and Nancy Swigonski

Rebecca McNally Keehn, Mary Ciccarelli, Dorota Szczepaniak, Angela Tomlin,Disorder

A Statewide Tiered System for Screening and Diagnosis of Autism Spectrum

ServicesUpdated Information &

019-3876http://pediatrics.aappublications.org/content/early/2020/07/02/peds.2including high resolution figures, can be found at:

References

019-3876#BIBLhttp://pediatrics.aappublications.org/content/early/2020/07/02/peds.2This article cites 39 articles, 10 of which you can access for free at:

Subspecialty Collections

http://www.aappublications.org/cgi/collection/autism:asd_subAutism/ASDal_issues_subhttp://www.aappublications.org/cgi/collection/development:behaviorDevelopmental/Behavioral Pediatrics_subhttp://www.aappublications.org/cgi/collection/community_pediatricsCommunity Pediatricsfollowing collection(s): This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtmlin its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprintshttp://www.aappublications.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

by guest on August 21, 2020www.aappublications.org/newsDownloaded from

originally published online July 6, 2020; Pediatrics Thomas Lock and Nancy Swigonski

Rebecca McNally Keehn, Mary Ciccarelli, Dorota Szczepaniak, Angela Tomlin,Disorder

A Statewide Tiered System for Screening and Diagnosis of Autism Spectrum

http://pediatrics.aappublications.org/content/early/2020/07/02/peds.2019-3876located on the World Wide Web at:

The online version of this article, along with updated information and services, is

http://pediatrics.aappublications.org/content/suppl/2020/07/02/peds.2019-3876.DCSupplementalData Supplement at:

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397. the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2020has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on August 21, 2020www.aappublications.org/newsDownloaded from