Pediatric Residency Training and Behavioral HealthModels and Outcomes from a Multi-site Study

• Jeffrey D. Shahidullah, PhD, Assistant Professor, Rutgers University, New Brunswick, NJ

•Paul W. Kettlewell, PhD, Director of Pediatric Psychology, Geisinger Health System, Danville, PA

•Mohammed H. Palejwala, MA, Doctoral Student, Michigan State University, East Lansing, MI

•Kathryn A. DeHart, MD, Pediatrician, Geisinger Health System, Danville, PA

Session # F6

CFHA 19th Annual ConferenceOctober 19-21, 2017 • Houston, Texas

Faculty DisclosureThe presenters of this session have NOT had any relevant financial

relationships during the past 12 months.

Conference Resources

Slides and handouts shared in advance by our Conference Presenters are available on the CFHA website at http://www.cfha.net/?page=Resources_2017

Slides and handouts are also available on the mobile app.

Learning Objectives

At the conclusion of this session, the participant will be able to:

• Identify current training needs related to behavioral health in pediatric residency programs

• Describe existing models of behavioral health training and the advantages/weakness of each

• Discuss how aspects of the enhanced training curricula discussed in this presentation may be tailored/adapted for implementation in other training programs across the country

1. American Academy of Pediatrics. (2009). Policy Statement—The Future of Pediatrics: Mental Health Competencies for Pediatric Primary Care, Pediatrics, 124, 410-421.

2. Horwitz, S. M., Storfer-Isser, A., Kerker, B. D., Szilagyi, M., Garner, A., O’Connor, K. G.,…Stein, R. E. (2015). Barriers to the identification and management of psychosocial problems: Changes from 2004 to 2013. Academic Pediatrics, 15, 613-620.

3. Stancin, T., & Perrin, E. C. (2014). Psychologists and pediatricians: Opportunities for collaboration in primary care. American Psychologist, 69, 332-343.

4. McMillan, J. A., Land, M., & Leslie, L. K. (2017). Pediatric residency education and the behavioral and mental health crisis: A call to action. Pediatrics, 139, 21-41.

5. Horwitz, S. M., Caspary, G., Storfer-Isser, A., Singh, M., Fremont, W., Golzari, M., & Stein, R. E. (2010). Is developmental and behavioral pediatrics training related to perceived responsibility for treating mental health problems? Academic Pediatrics, 10, 252-259.

Bibliography / Reference

Learning Assessment

A learning assessment is required for CE credit.

A question and answer period will be conducted at the end of this

presentation.

Key Points

• 3 behavioral health training curricula

• Assessing residents’ response to curricula

• 1st year outcomes/next steps

Background

• PCPs positioned to serve increasing number of youth w/ BH concerns in PC

• 50-70% of patients seen in PC have BH referral concerns (Belar, 2008; Gatchel & Oordt, 2003)

• ≈ 1/4 of pediatric visits involve discussion of BH concerns (Cooper et al., 2006)

Background• National shortage of specialty BH providers (Kim, 2003) →

often difficult for PCPs to make external referrals

• When specialty providers are available → lack of follow through by patients to these externally referred services (Cummings & O’Donohue, 2011)

• Resultedly, PCPs often tasked w/ managing BH care “in-house”

• PCPs report difficulty in evaluating & treating BH conditions (Steele et al., 2010)

Background• Oft cited barrier to providing effective care → lack of

training for PCPs in BH (Serby et al., 2002)

• Most PC medicine residency program directors report BH training is important & should be emphasized more (Chin et al., 2000)

• Most directors of accredited PC residency training programs in internal medicine & pediatrics report suboptimal training on topic (Leigh et al., 2006)

Background• Numerous calls for improved biopsychosocial training for

physicians: o Goldberg et al., (1980). The Lancet o IOM (2001). Crossing the Quality Chasmo Healthy People 2020 (2010)

• AAMC added large behavioral & social sciences section to MCAT in 2015

• Due to lack of formal training in BH that medical providers receive → AAP (2009) highlighted key aspirational BH competencies for pediatric PCPso AAP → attainment of competencies “requires innovations in residency

training”

Background

• Several BH training initiatives developed in past 10 years → varying levels of intensity & resultso Didactic focused (talks, seminars, readings,

vignettes)

o Varying levels of results, intensity, & feasibility

Overview• Describe an innovation in residency training by delivering a BH

curriculum to pediatric residents in setting in which BH services are integrated w/ pediatric PC

• Obtain data from 1st-, 2nd-, & 3rd-year residents → Attitudes, Knowledge, & Skills in BH service delivery

• Data from 3 pediatric residency programs in Northeastern United States

• Findings may support rationale for innovation in BH service-delivery that also aims to improve BH competencies for pediatric PC

• Implementation science framework → Most effective “dosage” (training intensity) from 3 relatively feasible options?

Dependent Variables

• Attitudes/Comfort

• Knowledge/Skills…in behavioral health

Independent Variables

• Innovation in residency training → formal behavioral health curricula

• 3 different “dosages” (intensity levels)1) Integrated Service-Delivery + Didactic focus

2) Didactic focus only

3) Control (training as usual)

16

Site 1

Service-Delivery + Didactic ExposureService-Delivery Components

Requiring an embedded BH provider

Didactic Components

Not requiring an embedded BH provider

Warm hand-offs• On-site BH provider (psychologists)

• Direct patient care in well and sick-child visit

Lectures on BH topics• 18 lectures throughout

• AAP/AACAP practice parameters

Behind-the-scenes consults• On-site behavioral health providers (psychologists)

• Indirect patient care in conjunction w/ a well-/sick-child

visit

Readings and quizzes• AAP/AACAP practice parameters

• Evidence-based evaluation of ADHD, anxiety,

depression, suicide/crisis intervention;

• Quizzes over readings with feedback

Observation/Performance Feedback• Feedback provided for ADHD, anxiety, depression, and/or

suicide risk assessments

• Direct observations by embedded psychologists

• Feedback provided

Vignettes• Vignettes describing behavioral health concerns

• Identify course of treatment based practice parameters

• Feedback provided

Joint Precepting• Psychologist precepts residents and psychology fellows in

shared space

• Psychologist also precepts the resident in conjunction with

medical preceptor

Performance Feedback

ADHD Components• Inquire about inattention, hyperactivity, impulsivity,

academic/behavior problems

•More than one setting

•Data regarding:

• Duration of symptoms

• Severity of symptoms

• Degree of impairment

•Physical exam

•Screen for sensory impairments (vision/hearing)

•Administer parent/teacher rating scales

•Determine diagnosis

• Symptom criteria met in IA and/or HI domains

• Age of onset

• Impairment in multiple settings

Components Completed?

1 Allowed for performance feedback to occur in a setting preferred by

the resident

Yes No

2 Clearly stated that the performance feedback conversation would

remain confidential and it would not be used for formal performance

evaluation within residency program

Yes No

3 Asked/allowed time for resident to self-assess their performance

before completing checklist and receiving feedback from observer

Yes No

4 Completed checklist of practice parameter steps completed by resident

Yes No

5 Provided positive feedback for checklist items completed and

observed strengths

Yes No

6 Provided feedback on interpersonal and communication skills

Yes No

7 Provided suggestions for improvement that were accompanied by

rationales for why changes are important and how changes will

improve outcomes

Yes No

8 Allowed residents time to reflect on/ask questions about suggestions,

if they so chose

Yes No

9 Summarized performance feedback session in the form of major

“take-aways” and/or upcoming action steps

Yes No

10 Performance feedback session lasted between 5 and 15 minutes

Yes No

Process Components

18

Site 2

Didactic Exposure-Only

Service-Delivery Components

Requiring an embedded BH provider

Didactic Components

Not requiring an embedded BH provider

Not provided

Lectures on BH topics• 18 lectures throughout the year

• AAP/AACAP practice parameters

Readings and quizzes• AAP/AACAP practice parameters

• Evidence-based evaluation of ADHD, anxiety, depression,

suicide/crisis intervention

• Quizzes over readings with feedback

Vignettes• Vignettes describing behavioral health concerns

• Identify course of treatment based practice parameters

• Feedback provided

19

Behavioral Health Seminar for Pediatric Residents

Date Topic

10/16 The Behavioral Health Referral

10/23 Pediatric Behavior Management: The Basics

11/6 Pediatric Behavior Management: The Skills

11/20 ADHD: Evaluation (AAP)

12/4 ADHD: Treatment (AAP)

12/18 Anxiety: Evaluation (AACAP)

1/8 Anxiety: Treatment (AACAP)

1/22 Motivational Interviewing in Primary Care (1)

2/5 Motivational Interviewing in Primary Care (2)

2/19 Depression: Evaluation (AACAP)

3/4 Depression: Treatment (AACAP)

3/18 Suicide/Crisis Response (AACAP) (1)

3/25 Suicide/Crisis Response (AACAP) (2)

4/1 Sleep

4/22 Feeding

5/6 Toileting

5/20 Child Abuse

6/3 Schools (SPED system & bullying)

6/17 Medically unexplained physical symptoms

Examples of Readings•AACAP & AAP Practice Parameters• American Academy of Child and Adolescent Psychiatry. (2007). Depression Practice Parameters.

• American Academy of Child and Adolescent Psychiatry. (2007). Anxiety Practice Parameters.

• American Academy of Child and Adolescent Psychiatry. (2001). Suicidal Behavior Practice Parameters.

• American Academy of Pediatrics. (2011). ADHD Practice Parameters.

•Pediatrics in Review articles

•Resources from AAP’s Mental Health Toolkit

21

Quizzes

Vignettes

What are your treatment recommendations?

V-1: Mom reports severe ADHD symptoms; teachers say pt is an “angel”; clearly does not meet DSM criteria for diagnosis

V-2: Mom reports severe and persistent ADHD symptoms; teachers report severe and persistent ADHD symptoms; clear impairment

V-3: Mom and teachers report borderline severity ADHD and borderline impairment

24

Site 3

Control Group; Treatment as Usual

Service-Delivery Components

Requiring an embedded BH provider

Didactic Components

Not requiring an embedded BH provider

Not provided Not provided

What to Measure?AAP (2009) - Policy Statement

Call for innovations in BH training of pediatricians –

Need to address attitudes, knowledge and skills in BH service delivery

How to Measure?What did we consider? – What had previously been used?

What to do when there is no “gold standard”?

Guiding Principles• Goal not primarily to develop a measure(s)

• Address attitudes, knowledge and skills with

• Frequently occurring BH problems that pediatricians deal with

• Those for which there are some published (well accepted) standards of care

• Which categories did we select?

Our ResponseQualitative and Quantitative Approach

Quantitative

1. Self-reported Knowledge – Their views about how skilled or competent they considered themselves to be

2. Measured Skills in Behavioral Health – key categories: ADHD, anxiety, depression, suicide risk assessment

1st Year Outcomes• Must use caution to not overstate results

◦ Pilot study

◦ Small sample size

◦ Difficult to detect significant differences

• Feasibility of implementation of curricula

• Utility of assessment tool for tracking response to curricula

Background: Participants/SitesVariable

Training as Usual(TAU)

(n = 12)

Didactic Only

(DO)

(n = 20)

Didactic + Integrated

Primary Care (DIPC)

(n = 24)

Mean age, y (SD) 30.17 (2.29) 30.15 (2.41) 31.35 (2.29) F=1.74, p=0.19

Males, n (%) 1 (8.33%) 2 (10%) 6 (25%) χ2=2.50, p=0.33

Hard science major, n (%) 10 (83.33%) 12 (60%) 21 (87.5%) χ2=5, p=0.11

MD degree (vs DO), n (%)

1 (8.33%) 12 (63.16%) 12 (50%)

χ2=9.27, p=0.01

DIPC > TAU

DO > TAU

Weeks in medical school on mental health rotation, w (SD) 4.83 (2.33) 5.75 (1.74) 5.08 (1.95) F=1, p=0.38

Completed mental health training (outside of medical school/

residency, n (%) 12 (100%) 19 (95%) 20 (83.33%) F=3.32, p=0.21

Clinical rotations in primary care in medical school, n (%) 6 (100%) 20 (100%) 16 (100%)

Did those primary care practice(s) have an embedded behavioral

health provider on site 5 (83.33%) 10 (50%) 2 (12.5%)

χ2=10.53, p=0.004

TAU > DIPC

DO > DIPC

What type were they? Psychologists 2 (40%) 7 (70%) 2 (100%) χ2=2.55, p=0.48

What type were they? Social Workers 4 (80%) 9 (90%) 1 (50%) χ2=1.86, p=0.66

How likely are you to go into primary care? (1-10) 3.67 (3.78) 6.90 (3.38) 5.06 (3.71) F=1.61, p=0.22

Would you be more likely to go into primary care pediatrics if it was

an integrated practice with embedded behavioral health providers? 1 (16.67%) 16 (80%) 9 (56.25%)χ2=8.2, p=0.01

DO > TAU

Current residency year, n (%)

PGY-1

PGY-2

PGY-3

5 (41.67%)

2 (16.67%)

5 (41.67%)

9 (45%)

4 (20%)

7 (35%)

13 (54.17%)

5 (20.83%)

6 (25%)

χ2=1.17, p=0.89

Completed DBP rotation, n (%) 2 (33.33%) 4 (36.36%) 4 (25%) χ2=0.43, p=0.88

Self-reported Knowledge in Behavioral Health

Evidence-based

Practice Parameter

Training as Usual

(TAU)

Didactic Only

(DO)

Didactic +

Integrated Primary

Care (DIPC)Significance

Pre

(n = 12)

Post

(n = 9)

Pre

(n = 20)

Post

(n = 17)

Pre

(n = 24)

Post

(n = 27)

ADHD

Evaluation

Treatment

7.27

6.36

8.2

7.2

6.42

5.21

6.91

5.91

6.09

5.52

7.47

6.36

Anxiety

Evaluation

Treatment

7.27

6.18

8.4

8

6.16

4.84

6.18

4.73

6.26

5.39

7

5.86

Depression

Evaluation

Treatment

7.73

6.27

9.4

8.4

7.16

5.21

6.45

4.73

7.17

5.91

7.33

6.21

Time*DO (p=0.02)

Knowledge scores range from 1 (least confident in knowledge of evidence-based practices) to 10 (most confident in

knowledge of evidence-based practice)

Measured Skills in Behavioral HealthEvidence-based Practice

Parameter

Training as Usual

(TAU)

Didactic Only

(DO)

Didactic +

Integrated

Primary Care

(DIPC)

Significance

Pre

(n = 12)

Post

(n = 9)

Pre

(n = 20)

Post

(n = 17)

Pre

(n = 24)

Post

(n = 27)

ADHD

Evaluation

Treatment

3.08

2.67

2.83

2.67

3.15

2.65

3.73

2.55

3.87

2.26

4.07

3.2

Anxiety

Evaluation

Treatment

2.55

2.42

2.67

2.33

2.55

2.05

2.82

2.18

2.83

2.09

3.8

2.47

Depression

Evaluation

Treatment

2.92

2.25

2.17

2.67

2.85

2.15

3.64

2.91

3.65

2.04

3.53

3.07

Time*DO

(p=0.04)

Possible scores ranged: ADHD evaluation, 0-16; ADHD treatment, 0-9; anxiety evaluation, 0-11; anxiety treatment 0-3; depression evaluation,

0-15; depression treatment, 0-8

Conclusions/Next StepsIBH is promising approach to training/learning enhancement of pediatric residents

Knowledge vs. Skills◦ Dunning Kruger Effect?

Continue to track outcomes over 3 years of residency

More research/advocacy needed for yet another benefit that IBH may provide (in addition to access, costs, clinical care improvement)

Session Evaluation

Use the CFHA mobile app to complete the

evaluation for this session.

Thank you!