National Quality Measures for Child Mental Health Care:Background, Progress, and Next Steps

abstractOBJECTIVE: To review recent health policies related to measuring childhealth care quality, the selection processes of national child healthquality measures, the nationally recommended quality measures forchild mental health care and their evidence strength, the progressmade toward developing new measures, and early lessons learnedfrom these national efforts.

METHODS:Methods used included description of the selection processof child health care quality measures from 2 independent national ini-tiatives, the recommended quality measures for child mental healthcare, and the strength of scientific evidence supporting them.

RESULTS: Of the child health quality measures recommended or en-dorsed during these national initiatives, only 9 unique measures wererelated to child mental health.

CONCLUSIONS: The development of new child mental health qualitymeasures poses methodologic challenges that will require a paradigmshift to align research with its accelerated pace. Pediatrics 2013;131:S38–S49

AUTHORS: Bonnie T. Zima, MD, MPH,a J. Michael Murphy,EdD,b Sarah Hudson Scholle, MPH, DrPH,c Kimberly EatonHoagwood, PhD,d Ramesh C. Sachdeva, MD, PhD,e RitaMangione-Smith, MD, MPH,f Donna Woods, EdM, PhD,g

Hayley S. Kamin, BA,h and Michael Jellinek, MDi

aUCLA Center for Health Services & Society, UCLA Semel Institutefor Neuroscience and Human Behavior, University of California atLos Angeles.; bMassachusetts General Hospital, Harvard MedicalSchool, Boston, Massachusetts; cNational Committee for QualityAssurance, Washington, DC; dDepartment of Child Psychiatry, NewYork University Medical Center, New York, New York; eMedicalCollege of Wisconsin, Children’s Hospital and Health System,Milwaukee, Wisconsin; fUniversity of Washington, SeattleChildren’s Research Institute, Center for Child Health, Behavior,and Development, Seattle, Washington; gInstitute for HealthcareStudies, Feinberg School of Medicine, Northwestern University,Chicago, Illinois; hDepartment of Psychology, University of Florida,Gainesville, Florida; and iPartners Healthcare, Harvard MedicalSchool, Boston, Massachusetts

KEY WORDSADHD, child mental health, clinical validity, depression, qualityimprovement research, quality measures

ABBREVIATIONSAAP—American Academy of PediatricsADHD—attention-deficit/hyperactivity disorderAHRQ—Agency for Healthcare Research and QualityCEBM—Centre for Evidence-Based MedicineCHIP—Children’s Health Insurance ProgramCHIPRA—Children’s Health Insurance Program ReauthorizationAct of 2009CMS—Centers for Medicare & Medicaid ServicesCOE4CCN—Center of Excellence on Quality of Care Measures forChildren with Complex NeedsDSM-IV—Diagnostic and Statistical Manual of Mental Disorders,Fourth EditionMDD—major depressive disorderNCINQ—National Collaborative for Innovation in Quality Measure-mentNQF—National Quality ForumNQS—National Quality StrategyPI—Principal InvestigatorPMCoE—Pediatric Measurement Center of ExcellenceRCT—randomized controlled trialSNAC—Subcommittee on Children’s Healthcare Quality Measuresfor Medicaid and Children’s Health Insurance Programs

(Continued on last page)

S38 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

Recent health policies have acceleratedthe development and use of qualitymeasures for children receiving pub-licly funded care.1,2 In response, a leg-islatively mandated national committeeand a nonprofit organization system-atically rated large pools of qualitymeasures and recommended a limitednumber to monitor the quality of carereceived by US children. Although theseinitiatives were independent and useddifferent approaches to select and ratechild health care quality measures,each recommended few measures re-lated to child mental health care.3,4 Thisgap is of public health significancebecause improving the quality of childmental health care is a longstandingnational priority,5–9 and there is sub-stantial room for improvement in men-tal health care for both private andpublicly insured populations.10–18

This article reviews the following: re-cent relevant health policy initiatives;the selection of national child healthquality measures; existing nationalstandards for child mental health care,including the strength of the evidencesupporting them; an update on de-velopment of new quality measuresrelated to child mental health care; andearly lessons learned from these na-tional efforts.

BACKGROUND

The Children’s Health Insurance Pro-gram Reauthorization Act of 2009(CHIPRA) called for identification, re-finement, and development of childhealth care quality measures for vol-untary use in Medicaid and Children’sHealth Insurance Programs (CHIP).19

Developed under the auspices of theAgency for Healthcare Research andQuality (AHRQ), an initial core set of24 quality measures was submitted tothe Secretary of the US Department ofHealth and Human Services on January1, 2010. For the subsequent PediatricQuality Measures Program, $55 million

was made available to support 7 Cen-ters of Excellence in 2010 to developnewmeasures and refine existing onesfor potential core set enhancements inJanuary 2013, 2014, and 2015.20

Under the leadership of the Centers forMedicare & Medicaid Services (CMS),CHIPRA also funded 10 five-year dem-onstration projects to states at an es-timated total cost of $100 million inFebruary 2010; 7 of them propose todevelop, test, evaluate, and/or reportadherence to quality measures.21 Out-reach and technical assistance effortsto the states to report on adherenceto 12 of the 24 measures in the initialcore set began in 2011.22 The use ofthe measures is likely to be sustainedthrough financial incentives to coll-ect and report on adherence rates re-garding quality indicators througha matching Federal Medical AssistancePercentage that is part of the AmericanRecovery and Reinvestment Act of2009.23 Eligible providers will receivethese payments for demonstrating“meaningful use” of quality measuresunder the Electronic Health RecordsIncentive Program and are anticipatedto be given the capacity to benchmarktheir own performance against aggre-gated data.23 Together, these activitiesare envisioned to be “the first stepstaken” to reach the goal of a quality-driven, evidence-based national sys-tem of child health care.22

Consistent with this vision, the NationalQuality Strategy (NQS) was established“to improve the delivery of health care,services, patient health outcomes, andpopulation health” for all Americans,as part of the 2010 Patient Protectionand Affordable Care Act.2,24,25 This is thefirst legislation to set national goals toimprove the quality of health care inpublic and private health care pro-grams. It will guide all US Departmentof Health and Human Services qualityimprovement programs and regula-tions, and set criteria to measure the

quality of health care to align with na-tional efforts for quality improve-ment.25 The 3 aims of the NQS are toimprove the overall quality of care,improve the health of the US pop-ulation, and reduce the cost of qualityhealth care.24 To adapt the NQS forbehavioral health care, the SubstanceAbuse and Mental Health Services Ad-ministration developed the BehavioralHealth Quality Framework that tailorsthe 6 national priority areas to behav-ioral health care, reinforcing how the3 aims of the NQS could be equallyapplied to the care of mental healthproblems.3

Contemporaneously, the National Qual-ity Forum (NQF) is a private, nonprofitorganization that was given federalfunding to conduct a parallel effort toidentify and endorse measures thatcould be used to assess the quality ofchildren’s health care. The NQF is ded-icated to improving the quality of UShealth care by: (1) building consensuson national priorities and goals forperformance improvement and work-ing in partnership to achieve them;(2) endorsing national consensus stand-ards for measuring and publicly report-ing on performance; and (3) promotingthe attainment of national goals througheducation and outreach programs.26 Aspart of their mission, the NQF orga-nized a standardized process to eval-uate and endorse voluntary consensusstandards for patient outcomes forchild health and mental health, andchild health candidate standards. Theprojects, undertaken between 2009 and2011, are known as the Patient Out-comes (Phase III): Child Health and ChildHealth Measures Projects. Althoughspecific approaches across these dif-ferent national initiatives varied, theyraised similar questions about how toaddress barriers that limit the feasi-bility of these quality measures, theacceptable threshold for sufficient sci-entific evidence for clinical validity, and

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S39 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

how to address methodologic limita-tions that could influence the inter-pretation of findings.

QUALITY MEASURE SELECTIONPROCESS

CHIPRA: Development of Initial CoreSet of Measures

In partnership with AHRQ and CMS, theinitial core measure set was identifiedby using an evidence-informed processthat integrated input fromabroadarrayof stakeholders and public comments.27

A multidisciplinary AHRQ National Ad-visory Council Subcommittee on Child-ren’s Healthcare Quality Measures forMedicaid and Children’s Health In-surance Programs (SNAC) was formedin May 2009. The SNAC was chargedwith establishing quality measure eval-uation criteria, identifying a strategyfor gathering measures, and applyingthe evaluation criteria to the measures.It comprised multiple stakeholders,including officials from publicly in-sured programs, national professionalorganizations, and child and familyadvocacy organizations, as well as na-tional experts in health care qualitymeasurement.28

Over a 4-month period, the SNAC held 2public meetings and undertook sub-stantialworkoutsideof thesemeetings.This work included assessing an initialset of quality measures in use by Me-dicaid and CHIP by using an adaptedversion of the Rand/UCLA modifiedDelphi method, identifying a process tosupplement these measures througha public call for nominations, and sub-sequently assessing the nominatedmeasures by using the same modifiedDelphi method. The Rand/UCLA appro-priatenessmethod isawell-establishedapproach that integrates scientific ev-idence with expert clinical judgment29;it has been successfully used to assessthe quality of outpatient general healthcare among children nationally.30 Ithas also been used to assess the

quality of mental health care statewideamong children receiving publicly fun-ded outpatient specialty mental healthcare.18 The process integrates a reviewof the evidence base for a proposedmeasure and 2 rounds of structuredexpert ratings. During this process, theSNAC assessed the validity, feasibility,and importance of 119 measures, ofwhich 12 were specific to child mentalhealth. For each measure, the SNACrated the level of scientific evidencesupporting the measure, feasibility ofimplementing the measure, and themeasure’s importance. When consid-ering importance, highest priority wasgiven to measures that were deemedactionable (by which the SNAC meantthe extent to which a publicly insuredprogram would likely be able to im-prove their performance) and likely tosubstantially reduce health care costs.

The initial modified Delphi process re-duced the pool of candidate measuresunder consideration to 70. During thesecond public meeting, a series ofprivate electronic votes were conductedto eliminate overlapping measures,merge conceptually similar measures,and prioritize the remaining pool toselect the final measures. The SNACrecommended 25 measures that werethen reviewed by the CHIPRA FederalQuality Workgroup, Medicaid and CHIPofficials, and other key stakeholders.From this process, 2 measures weredropped due to lack of field testing,including 1 pertaining to suicide riskassessment for children with majordepression. Details of the methods andadministrative review pathways be-fore final submission of the initial coreset of measures are described else-where.27,28,31

In addition to selecting measures, theSNACprovidedguidance to thePediatricQualityMeasuresProgram. It found thatmeasures lacked the capacity to stratifyadherence according to race/ethnicity,tribe, socioeconomic state, or special

healthcareneedstatus, characteristicscalled for in the CHIPRA legislation.32,33

Content gaps led to recommendationsfor newmeasures for substance abusecare and mental health treatment aswell as in several areas relevant tochild mental health: specialty care, in-patient care, availability of services,coordination of care, medical home,family experiences of care, and out-comes.27,31,34,35 Furthermore, the SNACstrongly encouraged new quality meas-ures to be aligned with the priorities ofstate Medicaid and CHIP agencies,36,37

providers, and parents.38,39

NQF: Endorsement of Child HealthQuality Measures

The NQF consensus development pro-cess involves 9main steps that typicallyoccurovera 12- to 18-month period. Thesteps are as follows: (1) call for intent;(2) call for nominations; (3) call forcandidate standards; (4) candidateconsensus standards review; (5) publicand member comment; (6) membervoting; (7) Consensus Standards Ap-proval Committee Decision; (8) boardratification; and (9) 30-day appeals.40

The review of the candidate standardsfor the aforementioned child health–related projects was conducted bysteering committees composed of childhealth and family advocates, healthcare system and provider professionalorganizations, clinicians, and healthcare quality measurement experts.After a set of standardized trainingsessions, the committee conducted adetailed review of the candidatestandards during an in-personmeetingwith follow-up as required by confer-ence call. Similar to the developmentof the CHIPRA initial core set, trans-parency was of high priority. Thesteering committee meeting was opento the public, member voting was doneopenly, information about the meetingwas posted on the NQF Web site, andtime for public comment was allocatedon the agenda.

S40 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

The measures were rated on 4 maincriteria: (1) importance tomeasureandreport the nominal topic; (2) scientificacceptability; (3) usability; and (4) fe-asibility. Within these 4 domains, thereviewer also rated subdomains tostandardize the rationale for the maincriterion rating. If the measure wasdeemed not to be important, the ratingstopped. The extent a measure met theremaining criteria was rated on a4-point scale (ie, completely, partially,minimally, not at all). During the votefor recommendation for endorsement,each reviewer personally weighed hisor her item ratings. Recommendationswere then classified as with or withoutconsensus by NQF staff. Details of therating criteria used for both initiativesare summarized in Table 1. The NQFcriteria are regularly updated, andmore rigorous criteria for scientificacceptability are being applied for the2012 Behavioral Health MeasuresEvaluation.41

RECOMMENDED OR ENDORSEDQUALITY MEASURES FOR CHILDMENTAL HEALTH CARE

Although the approaches varied, bothprocesses yielded relatively few childmental health qualitymeasures (Table 2).Of the 70 measures considered for theCHIPRA initial core set, 12 pertained tochildmental health care; of these, 3 wererecommended. Of the 101 candidatemeasures reviewed during the NQFprojects, 15 pertained to child mentalhealth care. Five of these overlappedwiththe 3 CHIPRA measures, 2 were the samemeasure for 2 different age groups ofteenagers, and 1 measured maternalmental health. Thus, there were 9 uniquemeasures of the quality of child mentalhealth care in CHIPRA and NQF combined.

For both initiatives, priority was placedon the development of a balanced set ofmeasures to build capacity to trackawidebreadthof quality care. For thesemeasures, the age ranges varied in the

specifications,suchthat1wasrestrictedto childrenaged0 to 5 years, 2 to ages13to 18 years, and 6 included all or mostchild age groups. The focus of concernalso ranged from specific to generalproblem areas. Two measures focusedon depression, 2 on attention-deficit/hyperactivity disorder (ADHD), 1 on riskybehaviors, 1 on suicidality, and 3 on gen-eral problem areas. Two of themeasuresinvolved monitoring, 3 called for screen-ing, and 4 required clinicians to makeassessments.

EVIDENCE STRENGTH FOR CHILDMENTAL HEALTH CARE QUALITYMEASURES

One potential next step for the creationof quality standards is to rate the em-pirical evidence that supports eachmeasure. The Oxford Centre for Evidence-based Medicine (CEBM)42 has put to-gether detailed methods for conductingthese kinds of ratings, and all of theCHIPRA measures were reviewed ac-cording to the CEBM standards.27 TheCEBM protocol involves assigning a let-ter grade of A (the best evidence) to D(the worst) for the quality of the evi-dence for a given measure based on thetypes of studies that have been con-ducted to validate its use as a standard.A letter grade of A corresponds to con-sistent level 1 studies (randomized con-trolled trials [RCTs]). A grade of Bcorresponds to consistent level 2 or 3studies or extrapolations from level 1studies, with level 2 studies defined asthose that include either systematicreviews of cohort studies or individualcohort studies (including low-qualityRCTs and “outcomes” research). Level3 studies are systematic reviews withhomogeneity of case-control studiesor an individual case-control study.A grade of C is given if there are onlylevel 4 studies or extrapolations fromlevel 2 or 3 studies, with level 4 definedas case series and poor-quality cohortand case-control studies. A grade of Dis given if the evidence is only of level 5

(expert opinion) or if the evidence isinconsistent or inconclusive.

As noted earlier, the quality of the evi-dence for the 3 CHIPRA measureshad been graded according to CEBMstandards. Although 1 of the CHIPRAmeasures received the low grade of D, 2measures were graded as B, but eventhese measures were noted to havelimitations in the quality of their evi-dence. Onemeasure hadbeenassessedin studies that did not specify age(CHIPRA #21: “Follow up after hospi-talization for mental illness”) and theother revealed “no data on whetherscreening using standardized tools ul-timately leads to better outcomes forthese children” (CHIPRA #8: “Screeningusing standardized screening tools forpotential delays in social and emo-tional development”).

Although NQF did not use CEBM stand-ards, there was a section on evidenceand all relevant studies on the NQF Website for each measure.26 For the pur-poses of the current article, we re-viewed the studies cited there andsupplemented this with a review ofstudies on the Web site of the stewardlisted for each measure. We also con-ducted a search by using Ovid andPubMed of studies published from 2001to 2011 with the 6 measure names asspecific and general search terms.

For only 2 of the measures did we findstudies suggesting higher than aD levelof evidence. The NQF summary for“Depression Screening by 13/18 yearsof age” (NQF # 1394 and 1515) notedthat this measure had been rated bythe US Preventive Services Task Forceas having a B level of evidence, cit-ing studies43,44 which reported thatscreening instruments both performedwell and increased the use of effectivetreatments and that use of the Pediat-ric Symptom Checklist was associatedwith increased rates of referral andimproved functioning for children afterintervention.45–48

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S41 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

TABLE 1 Assessment Criteria for Proposed Quality Measures by National Initiative

CHIPRA Initial Core Set27,69 NQF Patient Outcomes (Phase III) and Child HealthOutcome Measures40

Importancea

Importance (descending order) Importance to measure and report: the extent to which thespecific measure focus is important to making significantgains in health care quality (safety, timeliness, effectiveness,efficiency, equity, patient-centeredness) and improving healthoutcomes for a specific high-impact aspect of health carewhere there is variation in or overall poor performance.

• The measure should be actionable. State Medicaid and CHIP programs, managed care plans,and relevant health care organizations should have the ability to improve their performanceon the measure with implementation of quality improvement efforts

• High impact

• The cost to the nation for the area of care addressed by the measure should be substantial • Opportunities for improvement• Health care systems should clearly be accountable for the quality problem assessed by themeasure • Outcome or evidence to support measure focus• The extent of the quality problem addressed by the measure should be substantial (ie, significantproportion of the US child population should be affected by poor performance on the measure)

• There should be documented variation in performance on the measure• The measure should be representative of a class of quality problems (ie, a “sentinel measure” ofquality of care provided for preventive care, mental health care, or dental care)

• The measure should assess an aspect of health care in which there are known disparities• The measure should contribute to a final core set that represents a balance portfolio ofmeasures and is consistent with the intent of the legislation

• Improving performance on measures included in the core set should have the potential totransform care for our nation’s children

Scientific acceptabilityb

Validity: the degree to which a quality measure is associated with what it purports to measure Scientific acceptability of measure properties• It meets criteria for scientific soundness, defined as adequate scientific evidence or, whereevidence is insufficient, expert professional consensus to support the relation betweenstructure and process, structure and outcome, or process and outcome

• Precisely specified

• The measure itself is valid; that is, it should truly assess what it purports to measure • Reliability testing• Validity testing• Exclusions justified• Risk adjustment for outcomes/resource use measures• Identification of meaningful difference in performance• Comparability of multiple data sources/methods• Disparities in care

Feasibilityc

Feasibility: the degree to which the measure is free from random error Feasibility: the extent to which the required data are readilyavailable, retrievable without undue burden, and can beimplemented for performance measurement

• The data necessary to score the measure are available to state Medicaid and CHIPprograms

• Detailed specifications are available for the measure• Estimates of adherence to the measure based on available data sources are likely tobe reliable and unbiased. This allows for meaningful comparisons across states, programs,individual providers or institutional providers

Usabilityd

Usability: the extent to which intended audiences (eg,consumers, purchasers, providers, policy makers) canunderstand the results of the measure and are likely to findthem useful for decision-making

• Meaningful, understandable, and useful information• Relation to other NQF-endorsed measures (harmonization,distinctive, or additive value)

• Data generated as a byproduct of care processes• Electronic sources• Exclusions• Susceptibility to inaccuracies, errors, or unintendedconsequences

• Data collection strategy/implementationa CHIPRA rating: 7–9 = definitely important and meets several of the criteria; 4–2 = uncertain level of importance and meets some of the criteria but fails to meet some of the criteria givenhigher weight (1–4); 1–3 = fails to meet most of the criteria; CHIPRA median pass score: $4; NQF rating: yes/no (must pass).b CHIPRA rating: 7–9 = scientifically sound and the measure itself is definitely valid (ie, sufficient evidence); 4–2 = uncertain scientific soundness (ie, insufficient evidence) and the measureitself has uncertain validity; 1–3 = not scientifically sound and the measure itself is not valid; CHIPRA median pass score: $7; NQF rating: completely, partially, minimally, not at all.c CHIPRA rating: 7–9 = definitely feasible; 4–2 = uncertain feasibility; 1–3 = not feasible; CHIPRA median pass score: $4; NQF rating: completely, partially, minimally, not at all.d NQF rating: completely, partially, minimally, not at all.

S42 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

TABLE 2 Recommended and/or Endorsed Child Mental Health Quality Measures

Measure (Source, Number) EvidenceGradea,b,c

Steward Description

1a. Follow-up care for children prescribed ADHDmedication (CHIPRA-20; NQF-108)

D National Committee for Quality Assurance Percentage of children 6–12 years of age as of theindex prescription episode start date with anambulatory prescription dispensed for andADHD medication and who had 1 follow-up visitwith a practitioner with prescribing authorityduring the 30-day initiation phase

1b. Management of ADHD in primary care forschool-aged children and adolescents (NQF-107)

NR Institute for Clinical Systems Improvement Percentage of patients treated with psycho-stimulant medication for the diagnosis of ADHDwhose medical record contains documentationof a follow-up visit at least twice a year

2. Follow-up after hospitalization for mentalillness (CHIPRA-21; NQF-576)

B National Committee for Quality Assurance Percentage of discharges for members aged $6years who were hospitalized for treatment ofselected mental health disorders and who hadan outpatient visit, an intensive outpatientencounter, or partial hospitalization witha mental health practitioner. Two rates arereported. Rate 1 was the percentage ofmembers who received follow-up within 30days of discharge. Rate 2 was the percentage ofmembers who received follow-up within 7 daysof discharge

3a. Developmental screening in the first 3 years oflife (CHIPRA-8; NQF-1448)

B National Committee for Quality Assurance,The Children and Adolescent HealthMeasurement Initiative

Percentage of children screened for risk ofdevelopmental, behavioral, and social delays byusing a standardized screening tool in thefirst 3yearsof life. This isameasureofscreening in thefirst 3 years of life that includes 3 age-specificindicators assessing whether children arescreenedby12months, 24months,or36monthsof age

3b. Developmental screening by 2 years of age(NQF-1399)

NR National Committee for Quality Assurance Percentage of children who turned 2 years oldduring the measurement year who hada developmental screening performed between12 and 24 months of age

4. Pediatric Symptom Checklist (NQF-722) NR Massachusetts General Hospital The Pediatric Symptom Checklist is a brief parent-report questionnaire that is used to measureoverall psychosocial functioning in childrenaged 4 to 16 years

5a. Depression screening by 13 years of age (NQF-1394)

NR National Committee for Quality Assurance Percentage of adolescents who turn 13 yearsof age in the measurement year who hada screening for depression by usinga standardized tool

5b. Depression screening by 18 years of age (NQF-1515)

NR National Committee for Quality Assurance Percentageofadolescentswho turn18yearsof agein the measurement year who had a screeningfor depression by using a standardized tool.

6a. Risky behavior assessment by age 13 years(NQF-1406)

NR National Committee for Quality Assurance Percentage of children with documentation ofa risk assessment or counseling for riskybehaviors by the age of 13 years. Four rates arereported: risk assessment or counseling foralcohol use, risk assessment or counseling fortobacco use, risk assessment or counseling forother substance abuse, and risk assessment orcounseling for sexual activity

6b. Risky behavior assessment by age 18 years(NQF-1515)

NR National Committee for Quality Assurance Percentage of children with documentation ofassessment or counseling for risky behavior.Four rates are reported: assessment orcounseling for alcohol use, tobacco use, othersubstance use, and sexual activity

7. Suicide risk assessment (NQF-1365) NR American Medical Association Percentage of patient visits for thosepatients aged6 through 17 yearswith a diagnosis of MDDwithan assessment for suicide risk

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S43 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

Overall, the evidence strength sup-porting the childmental health qualitymeasures was variable. None of themeasures was supported by researchusing RCTs to examine the relationshipbetween adherence and outcomes thatwere meaningful to “decision makers”(ie, parents, providers, payers)49 or im-pact on health.50 Such a research gapis consistent with adult mental healthand substance abuse care qualitymeasures.51

IMPROVING QUALITY MEASURESFOR CHILD MENTAL HEALTH CARE:NEXT STEPS

As part of the Pediatric Quality Mea-sures Program, 3 of the Pediatric QualityMeasures Centers of Excellence re-ceived first-round assignments thatincluded the development and refine-ment of quality measures related tochild mental health. The topic areaswere ADHD, depression, and identifyingeligible populations for mental healthquality measurement. The lead centersfor these activities were, respectively,

the AHRQ-CMS CHIPRA Pediatric Mea-surement Center of Excellence (PMCoE)based at the Medical College ofWisconsin (Principal Investigator [PI]:Dr Sachdeva), the AHRQ-CMS CHIPRANational Collaborative for Innovation inQuality Measurement (NCINQ) based atthe National Committee on Quality As-surance (PI: Dr Scholle), and the AHRQ-CMS CHIPRA Center of Excellence onQuality of Care Measures for Childrenwith Complex Needs (COE4CCN) basedat Seattle Children’s Research Institute(PI: Dr Mangione-Smith). Second-roundassignments also included topics re-lated to child mental health care, andthe AHRQ-CMS CHIPRA Mount SinaiCollaboration for Advancing PediatricQuality Measures (PI: Dr Kleinman) willalso develop behavioral health meas-ures. The new areas for measure de-velopment are: (1) psychotropic (mentalhealth) medication reconciliation; (2)follow-up after psychiatric hospitaliza-tion; (3) alcohol and substance abusescreening, use, diagnosis, treatment, andfollow-up; (4) developmental screening

and follow-up diagnosis, treatment, andmanagement of follow-up diagnosis; (5)emergency department and hospitaluse and avoidable use for mentalhealth problems; (6) adherence to rec-ommended care processes for commonmental health problems in emergencydepartment and hospital settings; (7)antipsychotic medication management;and (8) quality for children served inchild welfare. The following discussionoffers brief updates of the centers’earlyactivities.

Attention-Deficit/HyperactivityDisorder

The PMCoE is working collaborativelywith the American Medical AssociationPhysician Consortium for PerformanceImprovement, the AmericanAcademyofPediatrics (AAP), the American Board ofPediatrics, and the research academiccenters Northwestern University andthe Medical College of Wisconsin on thedevelopment and refinement of qualitymeasures related to the care of ADHD.This disorderwas selected because it is

TABLE 2 Continued

Measure (Source, Number) EvidenceGradea,b,c

Steward Description

8. Documentation of DSM-IV diagnosticevaluation for depression (NQF-1364)

NR American Medical Association Percentage of patients aged 6 through 17 yearswith a diagnosis of MDD with documentedevidence that they met the DSM-IV criteria (atleast 5 elements with symptom duration of$2weeks, including: [1] depressed mood (can beirritable mood in children and adolescents) or[2] loss of interest or pleasure) during the visitin which the new diagnosis or recurrentepisode was identified

9. Diagnosis of ADHD in primary care for school-aged children and adolescents (NQF-106)

NR Institute for Clinical SystemsImprovement

Percentage of patients newly diagnosedwith ADHDwhose medical record contains documentationof DSM-IV or Diagnostic and Statistical Manualfor Primary Care criteria being addressed

NR, not rated as to grade of evidence.a Oxford Centre for Evidence-based Medicine Levels of Evidence (March 2009). Produced by Bob Phillips, Chris Ball, Dave Sackett, Doug Badenoch, Sharon Straus, Brian Haynes, Martin Dawessince November 1998. Updated by Jeremy Howick March 2009. Available at www.cebm.net. Accessed January 24, 2013.b Evidence grades reported in this column are quoted from the official measure materials, which can be found on the CHIPRA and NQF Web sites. More specifically, evidence grades for theCHIPRA measures are shown for each of the 3 measures in the summary table for all of the measures (http://www.ahrq.gov/chipra/corebackground/corebacktab.htm). For the measuresendorsed by NQF, the evidence grade or lack thereof can be found on the Measure Submission and Evaluation Worksheet 5.0 that is posted for each measure on the NQF Web site (http://www.qualityforum.org/Home.aspx).c From the CHIPRA Web site: the types and rigor of studies at various levels of evidence depend on the study purposes (eg, therapy/prevention, prognosis, diagnosis, differential diagnosis/symptom prevalence; economic and decision analyses). Most of the studies submitted or identified as documentation of underlying scientific soundness for the measures were therapy orprevention studies. For those studies, level 1 studies are systematic reviews of RCTs. Level 2 studies include systematic reviews of cohort studies or individual cohort studies (including low-quality RCTs and “outcomes” research). Level 3 studies are systematic reviews with homogeneity of case-control studies or an individual case-control study. Level 4 studies are case series andpoor-quality cohort and case-control studies. Level 5 evidence is defined as expert opinion without explicit critical appraisal or based on physiology, bench research. or “first principles.”

S44 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

prevalent, affecting an estimated 3% to9% of US children.52 It is 1 of the mostcommon reasons children are referredfor mental health services and repre-sents 15% to 45% of the mental healthconditions diagnosed in children andyouth.53,54,55 Considerable variationsand gaps in care regarding ADHD havebeen documented in the literature.11,12,17

Priority was therefore placed on es-tablishing metrics for effective ADHDdiagnosis, follow-up, and treatment,first, as a part of the development of aninitial set of 25 pediatric measures andthen as an assigned topic for pediatricquality measure development and test-ing through the PMCoE.

Several recent studies have providedimportant guidance regarding effectiveADHD diagnosis, follow-up, and treat-ment. To incorporate the current bestevidence about these topics, the AAPconducted a 2-year process to reviseand update the 2003 AAP ADHD guide-line. The most recent ADHD guidelinewas published in November 2011, mak-ing several changes to the previousguideline recommendations to directthe field toward care based on the bestexisting evidence through 6 primaryrecommendations.56 Based on thesenew AAP ADHD guideline recommen-dations, investigators for Northwest-ern University, along with investigatorsand staff from the American MedicalAssociation, the AAP, and the AmericanBoard of Pediatrics, established andengaged an expert workgroup com-prising experts across the broad spec-trum of stakeholders related to thediagnosis, follow-up, and treatment ofADHD. This workgroup included pedia-tricians, child and adolescent psycholo-gists, child and adolescent psychiatrists,neurologists, parents, teachers, schoolnurses, family physicians, and an oc-cupational therapist. Critical changesto the AAP’s ADHD guideline recom-mendations included that ADHD diag-nosis should be determined based on

Diagnostic and Statistical Manual ofMental Disorders, Fourth Edition (DSM-IV) criteria or through the use of a val-idated tool based on these criteria,lowering the potential age of ADHD di-agnosis to include children ages 4 and5 years, and making specific recom-mendations for behavior therapy andmedication treatment.

The draft measures address knownquality gaps and variations in ADHDcare in accordance with the recom-mendations in the new 2011 AAP ADHDguideline for effective diagnosis, follow-up, and treatment of pediatric patients,ages 4 to 18 years, after a diagnosis ofADHD has been made. After develop-ment and specification of pediatricquality measures for ADHD, these meas-ures will be tested for: (1) performanceof the measure (using) manual chartreview;(2) feasibilityandvalidityofusingthe electronic health record to calculatethe measure; and (3) the feasibility ofspecifying the measures for construc-tion by using administrative data sour-ces and the reliability of the resultingmeasure output.

Major Depression

The NCINQ is taking the lead on thedevelopment and refinement of qualitymeasures related to adolescent de-pression. Major depressive disorder(MDD) is a disabling condition that isassociated with long-term complica-tions and may lead to suicide.44 MDDaffects .7% of adolescents in theUnited States. In 2006, ∼2.3 million ado-lescents 12 to 17 years of age reportedexperiencing a major depressive epi-sode at some point in their lives.44 De-pression can have a major impact onchildren’s functioning, disrupting dailylife at home, school, or in the commu-nity, and resulting in serious long-termmorbidities such as generalized anxietydisorder and panic disorder.57–62 De-pression may also lead to engagementin risky behaviors such as substance

use (eg, alcohol, illicit drugs, tobacco),and it can also lead to suicide.58–61

Suicide, the third leading cause ofdeath among 15- to 24-year olds, isoften preceded by depression or long-term MDD.44,60 Adolescent-onset de-pression increases the risk of attemptedsuicide by fivefold44 and is stronglycorrelated with chronic and recurringdepression in adulthood.63 Further-more, depressive symptoms can be bothprolonged and episodic, recurring overweeks and months.57 The Centers forDisease Control and Prevention notedthat individuals who experience just 1episode of depression are at a 50%higher risk of experiencing additionalepisodes.64

Based on a review of all major guide-lines, evidence reviews, and advicefrom family partners, clinicians, andresearchers, theNationalCommitteeonQuality Assurancehasdevelopeda logicmodel for adolescent depression man-agement and follow-up. This modeladdresses several key aspects of man-agement, including: (1) screening andassessment; (2) treatment options andinitiationof treatment; and (3) symptommonitoring, treatment course, and re-mission. The logic model uses a “mea-surement-based care” approach toconceptualize the steps involved in op-timizing care.65 For depression manage-ment, measurement-based care startswith use of standardized tools to screenfor depression in primary care, followedby confirmatory assessment and moni-toring of symptom and functioningthroughout the episode of depressionto guide treatment decisions and to as-sess response and remission. The modelalso acknowledges that successful im-plementation depends on adequate read-iness of primary and specialty providers.NCINQ stakeholder panels providedfeedback both on the overall approachand to identify the most salient oppor-tunities where quality measures arelikely to improve quality and outcomes.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S45 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

Identifying Eligible Populations forMental Health Quality Measurement

The COE4CCN is working to developseveral measures intended to advancequality measurement in the area ofgeneral childmental health care. One ofthe center’s early efforts has focusedon ways of coding the presence ofmental health conditions based on di-agnostic codes available in adminis-trative data. Use of these codes toidentify children with mental healthproblems will go through a process ofvalidation by using abstracted medicalrecord data as the gold standard. If themethodology developed is found to bevalid, it will then be further tested andrefined by using existing, large datasets such as Medicaid claims fromentire states. These analyses are beingconducted by using data from 1 stateMedicaid agency as well as a largeurban tertiary care children’s hospital.

Through this approach, the COE4CCN isworking to build the capacity of usingexisting data infrastructure to identifychildren with mental health conditions,describe the services delivered, andexplore new approaches to link mea-sure adherence with clinical outcomes.The long-accepted observation thatmental health problems are under-recognized in pediatrics66 suggests thatthe prevalence of child mental healthproblems may be underestimated. De-livery of mental health care may alsobe underreported because procedurecodes for evidence-based mental healthcare are often missing in Medicaidclaims data.17,18 Nevertheless, this newdirection has the potential to bringa kind of “parity” with physically basedmedical diagnoses in the identificationof mental health problems. Secondly,an algorithm to identify children with“social complexity” by using Medicaidclaims and enrollment data is underdevelopment. For the purposes of thisproject, social complexity is defined asthe presence of $1social risk factor

hypothesized to be a strong correlateof mental health. Valid identificationof social complexity may enhance theidentification of mental health problemsthat might be underreported as diag-noses in Medicaid service encounterdata, stand in as a proxy, or serve as amarker for children at risk for mentalhealth problems who might benefitfrom early preventive interventions. Datasourceswill includeMedicaid claims andencounter data from 1 state and surveysfrom parents and health care providers.

EARLY LESSONS LEARNED

The inclusion of quality measures re-lated to child mental health care andrecent priority placed on developingnew ones are major advances thatare consistent with the recommendedtrajectory of integrating mental healthcare into the patient-centered medicalhome.67 The early work within the Pe-diatric Quality Measures Program isstimulating the refinement of existingchild mental health measures andgenerating new proposed measures.The NQF is also embarking on re-evaluating existing and proposed be-havioral health measures. At everyphase, these processes are being con-ducted in collaboration with multiplestakeholder groups, including parentand family representatives, providers,state agency representatives, and healthservices researchers. They all bringa breadth of perspectives on whatmakes adherence to a quality mea-sure “meaningful.”

The development of new child mentalhealth quality measures poses meth-odologic challenges. The constraints ofexistingdata infrastructure, at the stateand provider levels, must be addressedto enhance the capacity to capture datathat link measure adherence to im-proved care and meaningful outcomes.Generating these desired data demandstime; therefore, priority must also beplaced on reducing provider and parent

burden. Furthermore, new researchmodels that promote engagement ofcommunity clinicians may require ad-aptation to test the clinical validity ofchild mental health quality measures.68

A paradigm shift for quality measure-ment for children is needed to alignresearch with its accelerated pace andcapitalize on the rich network of col-laboration from CHIPRA, NQF, and otherrelated projects. Early dialogue andsustained communication channels forinformationexchange, fundingthatcutsacross these facets, and sharing thecommon goal of improving outcomesfor children can serve as a startingpoint. The adoption of electronic healthcare records may also serve as amechanism to further strengthen thesecollaborations through active engage-ment in their development and imple-mentation. Together, these activitiesshare the original vision of a quality-driven health care system for childrenthat can be attained through a continu-ous process of quality improvementconducted in full partnership.

ACKNOWLEDGMENTSThe authors thank Evan M. Williamson,MPH, MS, for his technical consultationon the evidence bases for the NQF be-havioral health measures. They alsothank and acknowledge the work ofall who participated in the ADHD Mea-surement Leadership Team for theircontribution to evidence-based pedi-atric ADHD diagnosis, follow-up, andtreatment quality measure develop-ment; Mark Antman and Molly Siegelfrom the American Medical Associa-tion, Physician Consortium for Perfor-mance Improvement; Jonathan Klein,Fan Tait, and Keri Theissen from theAAP and Nicole Muller and CarolineMazurek from the Institute for Health-care Studies, in the Feinberg School ofMedicine at Northwestern University;and Mark Wolraich and Karen Pierce,the Co-Chairs of the ADHD MeasuresExpert Workgroup.

S46 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

REFERENCES

1. PL 111-113 Children’s Health InsuranceProgram Reauthorization Act

2. PL 111-148 [HR 3590] Patient Protectionand Affordable Care Act

3. Cotter F. Behavioral health qualitymeasures and health care reform. Pa-per presented at: Network for theImprovement of Addiction Treatmentand State Associations of AddictionServices Conference; Boston, MA, July12, 2011

4. Zima BT, Mangione-Smith R. Gaps in qualitymeasures for child mental health care: anopportunity for a collaborative agenda.J Am Acad Child Adolesc Psychiatry. 2011;50(8):735–737

5. Institute of Medicine, Division of MentalHealth and Behavioral Medicine. Reportof a Study on Research on Childrenand Adolescents with Mental, Behav-ioral, and Developmental Disorders.Washington, DC: The National AcademiesPress; 1989

6. The National Advisory Mental HealthCouncil Workgroup on Child and Adoles-cent Mental Health Intervention Deploy-ment and Development. Blueprint forChange: Research on Child and Adoles-cent Mental Health. Bethesda, MD; Officeof Communications and Public Liaison,Information Resources and InquiriesBranch, National Institute of Mental Health;2001

7. New Freedom Commission on MentalHealth, Achieving the Promise: Trans-forming Mental Health Care in America.Final Report. DHHS Pub. No. SMA-03-3832.Rockville, MD: 2003

8. US Congress, Office of TechnologyAssessment. Children’s Mental Health:Problems and Services-A BackgroundPaper, OTA-BP-H-33. Washington, DC: USGovernment Printing Office, December1986

9. US Public Health Service. Report of theSurgeon General’s Conference on Child-ren’s Mental Health: A National ActionAgenda. Washington, DC; 2000

10. Bethell CD, Kogan MD, Strickland BB, SchorEL, Robertson J, Newacheck PW. A na-tional and state profile of leading healthproblems and health care quality for USchildren: key insurance disparities andacross-state variations. Acad Pediatr. 2011;11(suppl 3):S22–S33

11. Bussing R, Zima BT, Mason D, Hou W, GarvanCW, Forness S. Use and persistence ofpharmacotherapy for elementary schoolstudents with attention-deficit/hyperactivity

disorder. J Child Adolesc Psychopharma-col. 2005;15(1):78–87

12. Gardner WP, Kelleher KJ, Pajer KA, CampoJV. Follow-up care of children identifiedwith ADHD by primary care clinicians:a prospective cohort study. J Pediatr. 2004;145(6):767–771

13. Hoagwood K, Jensen PS, Feil M, Vitiello B,Bhatara VS. Medication management ofstimulants in pediatric practice settings:a national perspective. J Dev Behav Pediatr.2000;21(5):322–331

14. Kelleher KJ, Hohmann AA, Larson DB. Pre-scription of psychotropics to children inoffice-based practice. Am J Dis Child. 1989;143(7):855–859

15. Leslie LK, Weckerly J, Plemmons D,Landsverk J, Eastman S. Implementing theAmerican Academy of Pediatrics attention-deficit/hyperactivity disorder diagnosticguidelines in primary care settings. Pedi-atrics. 2004;114(1):129–140

16. Zarin DA, Suarez AP, Pincus HA, KupersaninE, Zito JM. Clinical and treatment charac-teristics of children with attention-deficit/hyperactivity disorder in psychiatric prac-tice. J Am Acad Child Adolesc Psychiatry.1998;37(12):1262–1270

17. Zima BT, Bussing R, Tang L, et al. Qualityof care for childhood attention deficit/hyperactivity disorder in a managedcare Medicaid program. J Am Acad ChildAdolesc Psychiatry. 2010;49(12):1225–1237

18. Zima BT, Hurlburt MS, Knapp P, et al. Qualityof publicly-funded outpatient specialtymental health care for common childhoodpsychiatric disorders in California. J AmAcad Child Adolesc Psychiatry. 2005;44(2):130–144

19. Children’s Health Insurance ProgramReauthorization Act of 2009. PL-, February 4,2009. Available at: www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm.Accessed March 4, 2012

20. CHIPRA Pediatric Healthcare Quality Mea-sures Program Centers of Excellence (U18).Available at: http://www07.grants.gov/search/search.do?oppId=53953&mode=VIEW. AccessedSeptember 5, 2012

21. Quality Demonstration Grants SummaryCHIPRA. Available at: www.insurekidsnow.gov/professionals/CHIPRA/grants_summary.html. Accessed March 9, 2012

22. Centers for Medicare & Medicaid Services.CHIPRA quality measures. Available at:http://hsd.aphsa.org/SMD_letters/pdf/SMD/21511_QualityReporting.pdf. Accessed Feb-ruary 15, 2012

23. PL 111-5 American Recovery and Rein-vestment Act of 2009

24. US Department of Health and HumanServices. 2012 Annual progress report toCongress: national strategy for qualityimprovement in health care. Available at:www.ahrq.gov/workingforquality/nqs/nqs2012annlrpt.pdf. Accessed July 15,2012

25. US Department of Health and HumanServices. Report to Congress: nationalstrategy for quality improvement in healthcare. Available at: www.healthcare.gov/law/resources/reports/nationalquality-strategy032011.pdf

26. National Quality Forum (NQF). Available at:www.qualityforum.org/. Accessed March 5,2012

27. Mangione-Smith R, Schiff J, DoughertyD. Identifying children’s health care qual-ity measures for Medicaid and CHIP: anevidence-informed, publicly transparent ex-pert process. Acad Pediatr. 2011;11(suppl3):S11–S21

28. Federal Register Notice 2472-NC-CMS. Re-quest for public comment on initial,recommended core set of children’shealthcare quality measures for voluntaryuse by Medicaid and CHIP programs.Available at: www.ahrq.gov/chipra/cor-ebackground/corebackgrnd.pdf. AccessedMarch 4, 2012

29. Brook R. The RAND/UCLA appropriatenessmethod. In: McCormick KA, Moore SR, SiegelRA, eds. Clinical Practice Guideline Develop-ment: Methodology Perspectives. Rockville,MD: US Department of Health and HumanServices; 1995

30. Mangione-Smith R, DeCristofaro AH, SetodjiCM, et al. The quality of ambulatory caredelivered to children in the United States.N Engl J Med. 2007;357(15):1515–1523

31. Dougherty D, Schiff J, Mangione-Smith R.The Children’s Health Insurance ProgramReauthorization Act quality measures ini-tiatives: moving forward to improve mea-surement, care, and child and adolescentoutcomes. Acad Pediatr. 2011;11(suppl 3):S1–S10

32. Centers for Medicare and Medicaid Serv-ices. Children’s Health Insurance ProgramReauthorization Act (CHIPRA) of 2009. Avail-able at: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111%20%C2%A0%C2%A0. Accessed March 4, 2012

33. Fairbrother G, Simpson LA. Measuring andreporting quality of health care for children:

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S47 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

CHIPRA and beyond. Acad Pediatr. 2011;11(suppl 3):S77–S84

34. Co JP, Sternberg SB, Homer CJ. Measuringpatient and family experiences of healthcare for children. Acad Pediatr. 2011;11(suppl 3):S59–S67

35. Sternberg SB, Co JP, Homer CJ. Review ofquality measures of the most integratedhealth care settings for children and theneed for improved measures: recommen-dations for initial core measurement setfor CHIPRA. Acad Pediatr. 2011;11(suppl 3):S49–S58.e3

36. deLeon SE, Hess CA. Medicaid and CHIPchildren’s healthcare quality measures:what states use and what they want. AcadPediatr. 2011;11(suppl 3):S68–S76

37. Greene-McIntyre M, Caldwell C. State Med-icaid and Children’s Health Insurance Pro-gram’s perspective on DHIPRA CoreMeasures. Acad Pediatr. 2011;11(suppl 3):S89–S90

38. Palfrey JS, Brei TJ. Children’s health careproviders and health care quality mea-surement. Acad Pediatr. 2011;11(suppl 3):S87–S88

39. Wells N, Partridge L. Families are key inimproving quality. Acad Pediatr. 2011;11(suppl 3):S85–S86

40. National Quality Forum. Consensus De-velopment Process. Available at: www.qualityforum.org/Measuring_Performance/Consensus_Development_Process.aspx.Accessed March 7, 2012

41. National Quality Forum. Behavioralhealth phase 1 and 2. Available at: www.qualityforum.org/Projects/Behavioral_Health_Phase_1_and_2.aspx. Accessed March7, 2012

42. Oxford Centre for Evidence-based Medicine.Levels of evidence (March 2009). Avail-able at: www.cebm.net/index.aspx?o=1025.Accessed February 24, 2012

43. Weissman MM, Wolk S, Goldstein RB, et al.Depressed adolescents grown up. JAMA.1999;281(18):1707–1713

44. Williams SB, O’Connor E, Eder M, Whitlock E.Screening for child and adolescent de-pression in primary care settings: Asystematic evidence review for the USPreventive Services Task Force. Pediat-rics. 2009;123(4). Available at: www.pediatrics.org/cgi/content/full/123/4/e716

45. Kuhlthau K, Jellinek M, White G, Vancleave J,Simons J, Murphy M. Increases in behav-ioral health screening in pediatric carefor Massachusetts Medicaid patients.Arch Pediatr Adolesc Med. 2011;165(7):660–664

46. Gall G, Pagano ME, Desmond MS, PerrinJM, Murphy JM. Utility of psychosocialscreening at a school-based healthcenter. J Sch Health. 2000;70(7):292–298

47. Hacker KA, Williams S, Myagmarjzv E,Cabral H, Murphy M. Persistence andchange in pediatric symptom checklistscores over 10-18 months. Acad Pediatric.2009;9(1):270–277

48. Murphy JM, Arnett HL, Bishop SJ, JellinekMS, Reede JY. Screening for psychosocialdysfunction in pediatric practice. A natu-ralistic study of the Pediatric SymptomChecklist. Clin Pediatr (Phila). 1992;31(11):660–667

49. Khoury MJ, Gwinn M, Yoon PW, Dowling N,Moore CA, Bradley L. The continuum oftranslation research in genomic medi-cine: how can we accelerate the appro-priate integration of human genomediscoveries into health care and diseaseprevention? Genet Med. 2007;9(10):665–674

50. Trochim W, Kane C, Graham MJ, Pincus HA.Evaluating translational research: a pro-cess marker model. Clin Transl Sci. 2011;4(3):153–162

51. Pincus HA, Spaeth-Rublee B, Watkins KE.Analysis & commentary: The case formeasuring quality in mental health andsubstance abuse care. Health Aff (Mill-wood). 2011;30(4):730–736

52. American Academy of Pediatrics. ADHD:Clinical practice guideline for the diagno-sis, evaluation, and treatment of Attention-Deficit/Hyperactivity Disorder in childrenand adolescents. Pediatrics. 2011;128(5):2011–2654

53. American Academy of Pediatrics. ADHD:Clinical practice guideline for the diagno-sis, evaluation, and treatment of Attention-Deficit/Hyperactivity Disorder in childrenand adolescents. Pediatrics. 2011;128(5):2011–2654

54. National Center for Health Statistics.Summary Health Statistics for U.S. Chil-dren: National Health Interview Survey2011 Vital Health Stat 10(254). 2011.Medicaid and CHIP Payment and Accesscommission (MACPAC). Report to theCongress on Medicaid and CHIP, March2012

55. Visser MS, Bitsko RH, Danielson ML, et al.Increasing Prevalence of Parent-ReportedAttention Deficit/Hyperactivity DisorderAmong Children; United States, 2003 and2007 November 12, 2010 / 59(44);1439–1443SN

56. American Academy of Pediatrics. ADHD:Clinical practice guideline for the diagnosis,

evaluation, and treatment of attention-deficit/hyperactivity disorder in children andadolescents. Pediatrics. 2010;128(5):1007–1022

57. Child Mind Institute. Major depressivedisorder. Available at: www.childmind.org/en/health/disorder-guide/major-de-pressive-disorder. Accessed September20, 2011

58. Foley HA, Carlton CO, Howell RJ. The re-lationship of attention deficit hyperac-tivity disorder and conduct disorders tojuvenile delinquency: legal implications.Bull Am Psychiatry Law. 1996;24(3):333–345

59. Friedman RM, Katz-Levey JW, ManderschiedRW, Sondheimer DL. Prevalence of seriousemotional disturbance in children andadolescents. In: Manderscheid RW, Sonnen-schein MA, eds. Mental Health. Rockville, MD:Center for Mental Health Services; 1996:71–78

60. National Research Council and Institute ofMedicine, Committee on Adolescent HealthCare Services and Models of Care forTreatment Prevention, and Healthy Devel-opment, Board on Children, Youth, andFamilies, Division of Behavioral and So-cial Sciences and Education. AdolescentHealth Services: Missing Opportunities.Washington, DC: The National AcademiesPress; 2009

61. Taylor E, Chadwick O, Heptinstall E,Danckaerts M. Hyperactivity and con-duct problems as risk factors for ado-lescent development. J Am Acad ChildAdolesc Psychiatry. 1996;35(9):1213–1226

62. US Preventive Services Task Force. Screen-ing and treatment for major depressivedisorder in children and adolescents: USPreventive Services Task Force recommen-dation statement. Pediatrics. 2009;123(6):1611

63. Garber J, Clarke GN, Weersing VR, et al.Prevention of depression in at-risk adoles-cents: a randomized controlled trial. JAMA.2009;301(21):2215–2224

64. Centers for Disease Control and Pre-vention. Depression. Available at: www.cdc.gov/mentalhealth/basics/mental-illness/de-pression.htm. Accessed February 24, 2012

65. Harding KJ, Rush AJ, Arbuckle M, TrivediMH, Pincus HA. Measurement-based care inpsychiatric practice: a policy frameworkfor implementation. J Clin Psychiatry. 2011;72(8):1136–1143

66. Costello EJ, Edelbrock C, Costello AJ, DulcanMK, Burns BJ, Brent D. Psychopathology inpediatric primary care: the new hidden

S48 ZIMA et al by guest on July 25, 2020www.aappublications.org/newsDownloaded from

morbidity. Pediatrics. 1988;82(3 pt 2):415–424

67. Agency for Healthcare Research and Qual-ity (AHRQ). CHIPRA core set candidatemeasures: follow up after hospitalizationfor mental illness. Available at: www.ahrq.

gov/chipra/corebackground/measures/CC-5.pdf. Accessed March 5, 2012

68. Ryan G, Berrebi C, Beckett M, et al. Reen-gineering the clinical research enterpriseto involve more community clinicians. Im-plement Sci. 2011;6(1):36

69. Agency for Healthcare Research and Qual-ity. CAHPS health plan survey 4.0: Medicaidversion. Available at: https://www.cahps.ahrq.gov/content/products/HP3/PROD_HP3_ExecSummary.asp?p=1021&s=211. AccessedJune 1, 2011

(Continued from first page)

Dr Zima was a Robert Wood Johnson Foundation (RWJF) Clinical Scholar, University of California, Los Angeles, 1991; Dr Mangione-Smith was a RWJF ClinicalScholar, University of California, Los Angeles, 1997, and RWJF Generalist Physician Faculty Scholar, 2000–2004.

Dr Zima drafted and submitted an abstract for consideration for manuscript submission, developed the conceptual framework, provided oversight to tabulations,coordinated coauthor contributions, drafted earlier versions of the manuscript, and made final edits; Dr Murphy provided consultation regarding the article’sconceptual framework, offered oversight on the literature review, and participated in writing early and final manuscript drafts; Dr Scholle provided consultationon the development of new depression screening measures and participated in writing early and final manuscript drafts; Dr Hoagwood provided consultation onthe conceptual framework and development of new depression screening measures, and participated in writing early and final manuscript drafts; Dr Sachdevaand Dr Woods provided consultation on the refinement of attention-deficit/hyperactivity disorder quality measures, and participated in writing early and finalmanuscript drafts; Dr Mangione-Smith provided consultation on the refinement of the algorithm to identify children with complex health care needs, andparticipated in writing early and final manuscript drafts; Ms Kamin conducted literature reviews, and participated in writing early and final manuscript drafts; andDr Jellinek participated in development of the conceptual framework, and the writing of early and final manuscript drafts.

www.pediatrics.org/cgi/doi/10.1542/peds.2012-1427e

doi:10.1542/peds.2012-1427e

Accepted for publication Dec 20, 2012

Address correspondence to Bonnie T. Zima, MD, MPH, UCLA Center for Health Services and Society, 10920 Wilshire Blvd, # 300, Los Angeles, CA 90024. E-mail:bzima@mednet.ucla.edu

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

Copyright © 2013 by the American Academy of Pediatrics

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

FUNDING: This study was supported by the National Institute of Mental Health (P30 MH082760), the Agency for Healthcare Research and Quality (1U18HS020506,U18 HS020503, 1U18HS020498). Funded by the National Institutes of Health.

SUPPLEMENT ARTICLE

PEDIATRICS Volume 131, Supplement 1, March 2013 S49 by guest on July 25, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2012-1427e2013;131;S38Pediatrics 

Kamin and Michael JellinekHoagwood, Ramesh C. Sachdeva, Rita Mangione-Smith, Donna Woods, Hayley S.

Bonnie T. Zima, J. Michael Murphy, Sarah Hudson Scholle, Kimberly EatonProgress, and Next Steps

National Quality Measures for Child Mental Health Care: Background,

ServicesUpdated Information &

http://pediatrics.aappublications.org/content/131/Supplement_1/S38including high resolution figures, can be found at:

References

BIBLhttp://pediatrics.aappublications.org/content/131/Supplement_1/S38#This article cites 39 articles, 4 of which you can access for free at:

Subspecialty Collections

subhttp://www.aappublications.org/cgi/collection/quality_improvement_Quality Improvemente_management_subhttp://www.aappublications.org/cgi/collection/administration:practicAdministration/Practice Managementfollowing collection(s): This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtmlin its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprintshttp://www.aappublications.org/site/misc/reprints.xhtmlInformation about ordering reprints can be found online:

by guest on July 25, 2020www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2012-1427e2013;131;S38Pediatrics 

Kamin and Michael JellinekHoagwood, Ramesh C. Sachdeva, Rita Mangione-Smith, Donna Woods, Hayley S.

Bonnie T. Zima, J. Michael Murphy, Sarah Hudson Scholle, Kimberly EatonProgress, and Next Steps

National Quality Measures for Child Mental Health Care: Background,

http://pediatrics.aappublications.org/content/131/Supplement_1/S38located on the World Wide Web at:

The online version of this article, along with updated information and services, is

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397. the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2013has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on July 25, 2020www.aappublications.org/newsDownloaded from