K A T H E R I N E B . B E VA N S A M Y K R A T C H M A N

C H R I S T O P H E R B . F O R R E S T

C H I L D R E N ’ S H O S P I T A L O F P H I L A D E L P H I A U N I VE R S I T Y O F P E N N S Y L VA N I A

SUCCESSES AND CHALLENGES IN ENHANCING PATIENT ENGAGEMENT

DEVELOPMENT OF METHODS FOR IDENTIFYING CHILD

AND PARENT HEALTH OUTCOME PRIORITIES

ADVANCES IN PATIENT-REPORTED OUTCOME MEASUREMENT AND ADOPTION

• NIH Initiatives: Patient-Reported Outcome (PRO) Measurement Information System® (PROMIS), Neuro-QoL, NIH Toolbox • Provides qualitative and quantitative standards for

demonstrating the reliability, validity, sensitivity, and efficiency of PRO measurement tools.

• Food and Drug Administration guidance • Integration into EPIC • Increasing adoption in clinical research and practice

A MISSING PIECE…

• Usually, outcomes are selected without sufficient patient input • Precludes assessment of the full spectrum of patient concerns

• May lead to identification of “best practice” interventions that have little or no effect on the outcomes that matter most to children and parents

• Core sets = the “minimal standards” for the assessment and reporting of outcomes in clinical studies and encounters • Usually comprised of outcomes identified by researchers and

clinicians • Some attempts to obtain patient input through qualitative

means (e.g., focus groups)

CHILDREN WITH AUTISM SPECTRUM DISORDERS: COMMONLY EVALUATED PRO’S VS. PARENT-

IDENTIFIED OUTCOME PRIORITIES

Behavioral and emotional inhibition and control Sensory regulation Restricted interests, flexibility, and adaptation

Stereotyped behavior Energy level Externalizing behavior problems

Interpersonal communication Anxiety

PRO Parent-Identified

PURPOSE OF PCORI PILOT PROJECT

• To develop and evaluate novel methods for eliciting and prioritizing children’s health outcomes that are: • Comprehensible to both patients and clinicians • Measureable • Ready for integration into comparative effectiveness research

• Focused on six of the most common childhood conditions: ASD, ADHD, epilepsy, JIA, asthma, obesity

OUR TEAM

Patient Advisors (6)

Scientific Advisors (3)

Researchers Bevans +3

Patient-Scientists Kratchman +3

Harm

oniza

tion

Con

cept

V

alid

atio

n

Patie

nt

Und

erst

and

ing

Prio

ritiza

tion

CORES: Co-directed by a researcher and a patient-scientist

Patient Advocacy and Support Groups

CHALLENGES AND LESSONS LEARNED: DEVELOPING THE TEAM

• Inclusion of “representative” patients • Recognize and respect varying levels of patient

engagement • Related to patients’ readiness to engage as a patient-scientist

• Reasons for readiness = personal journey, experience, confidence…

• Researchers’ readiness to adopt the PCOR model • Power and team structure • Communication: Learning the language vs. using a

translator • Consider researchers’ and stakeholders’ unique and

complementary expertise in assigning roles and responsibilities

AIM 1: DEVELOP A TYPOLOGY OF HEALTH OUTCOMES

ICF-CY, PRO

concepts, NCS

Intervention Research Outcomes (Systematic

Literature Review)

Clinician-Identified Outcomes

Child- and Parent-Identified Outcomes

Focus on six common childhood conditions: ASD, ADHD, epilepsy, asthma, JIA, obesity

AIM 1: DEVELOP A TYPOLOGY OF HEALTH OUTCOMES

ICF-CY, PRO

concepts, NCS

Intervention Research Outcomes (Systematic

Literature Review)

Clinician-Identified Outcomes

Child- and Parent-Identified Outcomes

Focus on six common childhood conditions: ASD, ADHD, epilepsy, asthma, JIA, obesity

Researcher-Driven

Develop Controlled Vocabulary

Condition-Specific AND Measurement AND Self/Proxy-

Report Limited to Children

Execute Search PubMed

3189 articles

Identify Unique Article Remove Duplicates

2865 articles

Abstraction of Articles In-Scope

201 ASD articles (19%) 309 ADHD articles (39%)

292 epilepsy articles (29%)

Identification of Tools 40 ASD tools

36 ADHD tools 28 epilepsy tools

Obtain Tools 32 ASD tools

31 ADHD tools 24 epilepsy tools

Examine and Classify Items until Concept Saturation is Achieved

1592 ASD items 698 ADHD items

Epilepsy items (TBD)

IDENTIFYING COMMONLY ASSESSED OUTCOMES SYSTEMATIC LITERATURE REVIEW

TYPOLOGY MAPPING: VINELAND SOCIALIZATION

SCALE

Most commonly assessed ICF-CY categories:

b126: temperament and personality functions

d125: disposition and interpersonal functions

b125: emotional functions

d710: basic interpersonal interactions

d720: complex interpersonal interactions

d750: informal social relationships

d330: speaking

d880: engagement in play

d920: recreation and leisure

AIM 1: DEVELOP A TYPOLOGY OF HEALTH OUTCOMES

ICF-CY, PRO

concepts, NCS

Intervention Research Outcomes (Systematic

Literature Review)

Clinician-Identified Outcomes

Child- and Parent-Identified Outcomes

Focus on six common childhood conditions: ASD, ADHD, epilepsy, asthma, JIA, obesity

Patient-Scientist Driven

ASSESSING CHILD AND PARENT PERSPECTIVES

What problems would you like your child’s treatment to address?

How would you like your child’s treatment to improve his/her life?

A lot of research involving children with asthma focus on symptoms like difficulty breathing, but asthma impacts kids in other ways too. How does your child’s asthma impact his/her life? How would you like that to change? (Probe specific life domains: at school, with friends, how he/she communicates, etc.)

• Preference for positive focus • “Treatment”

facilitates focus on symptomatic outcomes

• Need to be specific about condition

• Consider impact on family, not just the child

Participants: Youth aged 12+ with six conditions and their parents; two in-person and two online data-collection modalities

AIM 1 PRODUCT

Outcome typology derived from patients (parents and youth), clinicians, and a review of the intervention research

CHALLENGES AND LESSONS LEARNED: AIM 1

• Empower patient-scientists to lead development of methodology

• Methods for accessing and engaging patient partners to elicit patient-centered outcome priorities

Data Quality Access Feasibility Acceptability Network

Barriers: Geography Sample Bias Cost

AIM 2: ENSURE THE COMPREHENSIBILITY AND CONTENT VALIDITY OF THE TYPOLOGY

e-Delphi with clinicians Round 1: Open-ended elicitation of outcome concept definitions for clinicians and patients

Rounds 2 and 3: Rate each outcome definition:

• Clinician understanding • Patient understanding

Suggest revisions

2x

Cognitive Interviews with youth and parents

Development of clinician and patient outcome definitions

Revisions

Test patient-centered outcome definitions using a cognitive interview approach

Iteratively refine and re-test until definitions are well-understood

Outcomes defined in BOTH clinician-oriented and lay language

AIM 2 PRODUCT Typology with outcomes defined in clinician-oriented and lay language. Uses: • To translate and standardize the dialects that health professionals

and patients use to describe health experiences • As a substrate for the outcome prioritization process

CHALLENGES AND LESSONS LEARNED: AIM 2

• Communication: Learning the language vs. using a translator • Use the expertise of patient-scientists • Capitalize on clinicians’ experience in communicating with

patients

• Commitment to developing a replicable and scalable approach to identifying patient-centered outcomes (rather than a product)

AIM 3: DEVELOP AND TEST METHODS FOR OBTAINING PATIENT-OUTCOME PRIORITIES

• Test two technology-enabled methods for prioritizing patient outcomes

• Method 1: Individually administered • Participants (youth with ASD and parents of children with ASD)

are asked to choose between two outcomes (with patient definitions).

• Outcome ratings are determined by analyzing results of the pair-wise comparisons using analytic hierarchy processes.

• Method 2: Administered in groups of similar participants • Participants rate the importance of each outcome using an

audience response system. • After each question, response frequencies are shown in real-

time to the group to facilitate discussion.

AIM 3 PRODUCT A scalable approach to identifying patient-prioritized outcomes

Uses: • To establish “patient-centered” core sets for use in clinical

research involving specific sub-groups (e.g., diagnostic, capacity, age)

• For further testing in clinical settings as a means for identifying individual patient’s outcome priorities

AIM 3: ANTICIPATED CHALLENGES AND FUTURE DIRECTIONS

• Selecting an outcome prioritization approach that is understandable, is feasible, and yields meaningful results

• Promote approaches for developing “patient-centered core sets” • Patient prioritization of outcomes included in PRO

development recommendations/standards • Sustain the patient-scientist network

• Involve patient-scientists in interpretation of results and dissemination efforts

• Strengthen and deepen engagement

Contact: Katherine Bevans, PhD Assistant Professor of Pediatrics CHOP/Penn Medicine bevans@email.chop.edu 267-426-2967

PCORI PILOT PROJECTS Aims

1. To develop a typology of health outcomes that is relevant to the experiences and preferences of children.

2. To ensure the comprehensibility of the health outcome typology for children and their parents.

3. To develop and test the feasibility and practicality of methods that use the typology to prioritize health outcomes from the perspectives of children and their families, using Autism Spectrum Disorder as a proof-of-principle condition.

Patient and Clinician Perceptions of Engagement in Research

1

Laura Forsythe, PhD, MPH AcademyHealth Annual Research Meeting June 25, 2013

Improved research recruitment and retention rates (Edwards et al. 2011)

Enhanced trust between researchers and participants (Decker et al. 2010; Edwards et al. 2011; Staniszewska et al . 2007)

Improved content and construct validity of measures (Cashman et al. 2008; Cotterell 2008)

Improved patient understanding of results (Chalmers 1995; McCauley et al. 2001; Doyle 2010)

Increased relevance of research results to patients (summarized in Nass et al. 2012)

Patient Engagement in Research

Objectives

Describe and compare patient and clinician attitudes about engagement in research

Identify perceived barriers to and facilitators of engagement in research

3

Study Methods

Methods: Survey Development

5

Identify Existing Survey Items

Develop New Survey Items • Perceived value of

engagement • Interest in engagement • Barriers and facilitators

for engagement

Partner with Patients and Clinicians for Feedback • Survey concepts • Item wording • Survey layout • Dissemination

Methods: Crowdsourced Survey

Recruitment from existing opt-in panels based on pre-supplied profile information Web based survey Rapid data collection Limited generalizability

6

Methods: Item Format Example

7

Respondents: Patients (N=900)

8

80% 20%

Disease group

Chronic disease patients

Rare disease patients

89% 11%

Primary Language

English Spanish

Respondents: Primary Care Clinicians (N=750)

9

53% 27%

12%

8%

Type of Provider

PhysiciansNurse PractitionersNursesPhysician Assistants

7%

23%

35%

25%

10%

Years in Practice

< 3 Years3 to 9 Years10 to 19 Years20 to 29 Years>30 Years

Results

10

Perceived Value of Research that Measures Things Patients Care About

11

87% 87%

0%

20%

40%

60%

80%

100%

Patients Clinicians

% V

ery

/ Mod

erat

ely

Impo

rtan

t

Survey Group

p>0.05

72% 77%

0%

20%

40%

60%

80%

100%

Patients Clinicians

% S

tron

gly

/ Som

ewha

t Agr

ee

Survey Group *p <0.05

Perceptions that Research Helps Patients Make Better Treatment Decisions

12

Perceived Value of Engaging Patients in Research

13

86% 72%

83%

0%

20%

40%

60%

80%

100%

CliniciansCliniciansPatients% S

tron

gly

/ Som

ewha

t Agr

ee

Survey Group *p<0.001

“Patients working directly with researchers can improve the value of medical research”

Perceived Value of Engaging Clinicians in Research

14

86% 72%

83%

0%

20%

40%

60%

80%

100%

CliniciansCliniciansPatients% S

tron

gly

/ Som

ewha

t Agr

ee

Survey Group *p<0.001

“Patients working directly with researchers can improve the value of medical research”

“Providers working directly with researchers can improve the value of medical research”

Interest in Engaging in Research

15

66% 55%

0%

20%

40%

60%

80%

100%

Patients Clinicians

% S

tron

gly

/ Som

ewha

t Agr

ee

Survey Group

*p<0.001

Barriers and Facilitators of Engagement

16

Barriers Facilitators

Patie

nts

• Lack of time (43%) • Concerns about privacy

(36%) • Work, school or caregiving

commitments (33%)

• Helping others with their medical condition (68%)

• Learning about their health (63%) • Helping the next generation (57%) • Getting paid (56%) • Making research more meaningful to

patients (49%)

Clin

icia

ns

• Lack of time (79%) • Lack of payment (47%) • Lack of research training

(35%)

• Helping patients receive better care (79%) • Getting paid (78%) • Contributing to scientific knowledge (61%) • Making research more meaningful for

patients (61%) • Improving professional satisfaction (52%) • Helping researchers decide what to study

(43%)

Strengths and Limitations

Strengths Exploration of understudied topic areas Inclusiveness of understudied populations: Spanish

speakers, rare-disease patients

Limitations Generalizability Self-reported data New survey items testing complex constructs

17

Conclusions

Most patients and clinicians agreed that engagement can improve the value of health research. Many patients and clinicians reported interest in engaging in research themselves. Strategies to facilitate both patient and clinician engagement: Establish link between engagement and patient care. Provide financial compensation. Minimize time burden.

18

Implications

Find ways to help those who are interested in engaging in research to be involved in meaningful ways. Develop evidence on the value of engagement in research, particularly for improving patient outcomes. Address barriers to engaging patients and primary care clinicians in research.

19

Next Steps

20

Foundational Elements

• Awareness of methods for PCOR

• Valuing patient perspective

• Interest in PCOR

• Ways for patients and researchers to partner

• Resources and infrastructure

• Policies/governance

Thank you!

Acknowledgements Patient, caregiver, and clinician partners Collaborators: Lori Frank, Kara Odom Walker, Diane

Hayes, Sue Levine, Ayodola Anise, Natalie Wegener, Gail Hunt, Anne Beal, Harlan Weisman, Freda Lewis Hall

Laura Forsythe, PhD, MPH

lforsythe@pcori.org www.pcori.org

21

Developing Conceptual Frameworks to Inform and Guide PCOR

25 June 2013

Understanding the choices

patients face

Aligning research questions and methods with

patient needs

Providing patients and providers with information

for better decisions

Patient Engagement Patient-Driven

Research Dissemination

Taking Patient-Centeredness Seriously

Research engagement

Who determines The research questions? The database to use? The variables to examine outcomes, comparators, and covariates? The analytic methods?

Who is on the research team? Patients or patient advocates? Other stakeholders?

What will the research team do with the information? How will results apply to health decisions?

Is there a plan for interaction between researchers and the community?

Frameworks for PCOR

Identify PCOR concepts and relationships between them Identify key components of PCOR and role of engagement Theoretical and evaluative frameworks: One conceptual model for PCOR A distinct measurement model

PCORI Conceptual Model of PCOR

Process for development informed by: Evaluative framework of research engagement

(AcademyHealth) Review of PCOR literature

Refined and vetted through: Discussion with AcademyHealth Input from PCORI’s Patient Engagement Advisory Panel

PCOR conceptual model

The following key changes were made based on discussion with the patient engagement advisory panel: TBD TBD

Final structure: PCOR conceptual model Use as basis for deriving testable measurement models.

Foundational Elements • Awareness of methods for PCOR

• Valuing of the patient perspective • Interest in PCOR

• Ways for patients and researchers to partner • Resources and infrastructure

• Policies/governance

Actions

• Initiate and maintain partnerships between researchers and stakeholders

• Facilitate cross-communication among research stakeholders

• Capture, use, and optimize patient perspective across phases of research

• Ensure meaningful influence on

research • Train for partnering • Share and use learnings

Outcomes

PCOR Principles Trust, Co-learning, Transparency, Respect

• Culture of patient-centeredness in research

• Meaningful & effective partnerships

• Optimal health outcomes

• Research relevant to patients/other stakeholders

• Use of research results in health decisions

• Quality health decisions

• Satisfaction with health care experiences

Conceptual Model for PCOR

Outcomes

Near-term

Long-term

Intermedia

AcademyHealth and PCORI Partnership

Contract awarded in 2012 to: Conduct literature scan on stakeholder engagement in

research (manuscript in progress) Develop framework for patient engagement in research Establish learning network for 50 PCORI pilot projects Monitor progress and experiences of PCORI pilot

projects in key areas

8 www.pcori.org

PCORI Pilot Projects Program

50 projects in 24 states and Washington, DC $31 million (over two years)

www.pcori.org 9

AcademyHealth Approach

Framework for patient and consumer engagement in research: Literature scan Initial framework draft (input from PCORI staff) Three iterations of input from Consumer Patient

Researcher Roundtable Revision and augmentation ongoing

Evaluative Framework for Patient Engagement in Research

Key Attributes Intended for use as evaluative tool Based on Donabedian model Agnostic with respect to entity “sponsoring” research

effort Inclusive definition of the term “patient” Acknowledges how much remains to be learned about

how to conduct PCOR

Evaluative Framework for Patient Engagement in Research

Key Components: Structure: refers to existence of assets, policies, and

other infrastructure deemed necessary for conduct of PCOR

Process: examines how patient engagement is incorporated or integrated into research process

Outcomes: near and longer-term end points of patient engagement in research (such as improvement in research design, health outcomes)

Engagement Infrastructure

Engagement infrastructure

Patient/researcher matching and

outreach

Patient-to-researcher mapping

and selection

Research stage(s) of engagement

Continuity and frequency

Structure Process

Increased quality and/or relevance of

research

More informed decision making and uptake of research

Improvements to dissemination and access to research

Policy deliberations/changes

Improvements in patient activation and

empowerment

Improvements in health outcomes and

health status

Outcomes

Attitudes, perceptions, and

activation

Modifications to research process

Interim Outcomes

Longer-term Outcomes

Next Steps

Anticipated: Still a work in progress, so welcome questions,

suggestions, and other feedback Developing manuscript for peer review

Hoped: Would like the framework to be used as a guidance tool

for those engaged in PCOR • Not a prescriptive “how to”

Thank You!

Consumer Patient Researcher Roundtable Contributors: Bryan Dowd, Jason Goldwater, Mark Gorman, Alice Leiter, Musa Mayer, Eva Powell, John Santa, Shoshanna Sofaer, and Mike Stoto AcademyHealth and PCORI Staff: Rochelle Bent, Laura Esmail, Laura Forsythe, Lori Frank, Emily Moore, Raj Sabharwal, Veronica Thomas, Rachel Witsaman, and Natalie Wegener

Improving the Impact of Patient-Engaged Research: Recommendations for Evaluation

Lori Frank, PhD and Sue Sheridan, MIM, MBA AcademyHealth Annual Research Meeting June 25, 2013

October 2012 Transforming Patient-Centered Research: Building Partnerships and Promising Models

Workshop purpose: Begin a dialogue on how to conduct patient-centered research.

Input received was used by PCORI staff to develop policy and programmatic recommendations.

2

Workshop Structure

The two-day event included 142 representatives from a range of communities. 75% of participants were patient representatives. Other participants: clinicians, providers, researchers

Small group brainstorming sessions

3

Breakout Session Topics

4

Identifying and Selecting Research Questions

How should PCORI identify and select specific research questions that are patient-centered for funding ?

Reviewing Research Proposals for Funding

How can PCORI effectively engage and use the real-world experience of patients to help evaluate research proposals we receive?

Matching Patients and Stakeholders with Researchers

How can PCORI connect patients and stakeholders with researchers for collaborative work that ensures studies reflect patient perspectives?

Disseminating Research to the Community

How do we ensure that patients and those who care for them can access and use PCORI’s research to make more-informed decisions?

Evaluating PCORI’s Patient and Stakeholder Engagement Programs

How can PCORI measure the effectiveness of its programs to involve patients and stakeholders throughout its work?

Evaluating Engagement Session: Questions for Consideration

Participants were asked to consider: How can PCORI best measure the effectiveness of

patient and stakeholder engagement in research? What novel methods can patients and patient advocates

propose for evaluation of research engagement that would capitalize on the growing networks of patients engaged in research?

5

Evaluating Engagement in Research

6

Nominations for Principles of Engagement

Transparency Empathy

Infrastructure change to achieve

parity between researchers and

patients

Incorporating learning in an ongoing way

Aiming for excellence in engagement

Evaluating the meaningfulness,

quality, and impact of the engagement

Recommendations for Evaluating Engagement in Research

• The objective for engaging patients and researchers as partners in research may be study-specific, but must be pre-defined to permit adequate evaluation of the partnership.

Define Success

• Topic generation will require different forms of engagement than will establishing research hypotheses or analyzing data and reporting findings.

Determine goals of engagement by research phase

• Use to ensure that all research partners can provide input on the quality and value of the research engagement, using established metrics based on definitions of success.

Establish feedback channels

• Including compensation for patient time and co-authorship

Address patient and researcher parity

• Assessments should happen before and after engagement.

Assess patient and researcher perceptions of the value and appropriateness of engagement

8

Items for Evaluating Engagement in Research

Stakeholders: How did engagement in the research change

the project? Has your PCORI-funded project been used by other research

groups? Was there a successful outcome for your study? Does it relate

to quality of patient engagement? Are people voting with their feet? Was the research idea collaborative? How quickly are results being used in further research and in

clinical care?

Additional metrics should include both subjective and objective measures of quality of engagement.

9

Process Recommendations for Evaluation

Funding applicants should be asked to build an evaluation of patient engagement into their proposals for the conclusion of their studies. Dissemination metrics: Go beyond peer-review. Evaluate quality and impact of engagement during and after the study.

Evaluate diversity of project participants.

Consider linking funding to evaluation results,

10

Recommendations for Strengthening Engagement in Research

Incentivize co-learning between researchers and patient partners. Organizational structures of health systems and research organizations should include patient research experts. “Seal of Approval” recognition of optimal research engagement methods Require it: Evaluate engagement in all PCORI funded projects.

11

Conclusions

Patients and other stakeholders provided concrete and feasible recommendations for evaluating quality of engagement in research. Metrics for success should be linked to quality of research and quality of engagement. Different metrics are needed by research phase. Organizational structures and funding requirements can be powerful levers for ensuring engagement quality.

12

Implications for Policy, Delivery, and Practice

Meaningfulness and impact of the research should be measured and link to engagement quality made overt. Research organizational structures can be modified to support success.

13

Thank You!

14

Patient, caregiver, and clinician partners

Contact: Lori Frank, PhD lfrank@pcori.org Sue Sheridan, MIM, MBA ssheridan@pcori.org

Engaging Stakeholders to Improve Depression

Management in a Tribal Health System

1

Renee Robinson (PI) Jennifer Shaw

Vanessa Hiratsuka Sara Norman

Julia Smith Denise Dillard

Southcentral Foundation Helene Starks

University of Washington

Presentation Objectives and PCORI Pilot Project Overview

• Familiarize audience with…

• Organization, population, and stakeholders • Depression management at our organization • Decision-support tools

• Stakeholder engagement • Approval versus engagement • Impact of engagement on process/project • Barriers encountered/proposed solutions

2

Southcentral Foundation (SCF) • Mission

– Working together with the Native Community to achieve wellness through health and related services

• Vision – A Native Community that

enjoys physical, mental, emotional, and spiritual wellness

3

Population

• SCF provides services to… – ~60,000 AN/AI people – Across 107,413 square miles – 231 federally recognized tribes – 60 villages

4

Our Stakeholders • AAIRB • Tribal Leaders (EC, BOD)

• Customer-owners

• Providers

• Steering Committee • Leaders

5

Customer-Owners

Providers Leaders

Depression Management at SCF • In 2001, SCF integrated

annual depression screening into all primary care clinics due to: – High prevalence of

symptoms – Low utilization of

behavioral health services

6

To make a good healthcare decision, you need to…

• Clarify the decision… – What is the decision you

need to make? – What is your reason for

making this decision? – Look at timeline for the

decision.

• Explore the decision… – What healthcare options

are available to you? – What are positive and

negative effects of options?

– What is the value of the positive and negative effects to you? (Explore your decision.)

7

Decision-Support Tools • Interactive tool to aid clinical decision-making

• Describe treatment options, risks, benefits, and efficacy • Help clarify preferences and values for treatment options • Communication between customer-owners and providers

• Facilitate improved depression management – Educational – Interactive – Selective

8

Pilot Project • Background and Rationale…

– Many people don’t pursue depression treatment • Negative effects of untreated illness (e.g., absenteeism)

– Many people prematurely discontinue treatment

• Wasted resources of healthcare system (e.g., unused Rx)

– Few people prefer holistic depression treatment • Under-utilized services

9

Project Aims and Overview

10

Aim 1

• Identify stakeholder priorities, preferences, and needs for depression management • Key informant interviews with customer-

owners, providers, and leaders

Aim 2 • Develop and evaluate decision-support

tool • Steering committee guidance and feedback • Pilot test in one primary-care clinic

Aim 3 • Evaluate health and economic outcomes

• Test refined tool in multiple clinics • Evaluate disease management, economic, and

health outcomes

Stakeholder Engagement: Aim 1

• Data Collection – 38 stakeholder interviews conducted Dec to Feb

• 19 Customer-Owners, 9 Leaders, 10 Providers

– Semi-structured interview guide • Depression screening and treatment resources • Decision-tool content, design, and deployment

• Analysis – Transcripts summarized – Themes identified – Synthesized information presented to Steering Committee

• Used to guide tool development

11

Deciding on Drug Therapy

12

I didn't even realize that I was depressed for a long time. I thought everyone felt this way; at least, everyone in my family seemed to. I probably would have just gone on like that if my doctor hadn't asked one day if I had ever thought about taking an antidepressant. I was relieved to find out that it isn't normal to feel like I do and that a lot of people are helped by medicines. I know it might take a while to find the right one, but I'm in no hurry; I've spent my whole life feeling sad.

I guess I'm just not comfortable with taking medicine for my depression. I feel like I ought to be able to manage this on my own without needing medicine. It seems too much like taking the easy route. But maybe I just don't feel bad enough yet.

I recently began going to counseling. I know that if I took an antidepressant, I might feel better sooner, but I don't like the sound of the side effects I could have. My therapist and I have set some goals for me to work on, and we agreed to revisit my decision in three months. I want to wait and see how the counseling goes before I take medicine.

Example Screens of First Draft of Decision-Support Tool

Approval versus Engagement

13

Get the facts • What is depression?

– A very common, highly treatable, medical illness that involves the body, mood, and thoughts. Physical, mental, emotional, and spiritual well-being. – It affects how people think about things, feel about

themselves, the way a person functions socially, at work and in relationships, as well as everyday activities like eating and sleeping.

– It is more than feeling blue, down in the dumps or sad about a particular issue or situation.(Link to Table with Symptoms of Depression, See next slide)

Link to Video about learn more about depression http://www.youtube.com/watch?v=IeZCmqePLzM&playnext=1&list=PL3E95B34CC364B7D0&feature=results_main

14

What healthcare options are available to you to manage your depression…

• Lifestyle Changes (Link to page expanding each)

– Healthy eating – Exercise – Relationship building – Stress management

• Behavioral Health Services - Therapy – Screening – Psychotherapy

• Medications • Traditional Healing • Other

Modifications to Decision-Support Tool

Impact of Engagement on Process Barrier and Proposed Solutions

16

Stakeholder Recommendations: Decision Tool Content • Provide context for the tool

– Who will see the information and how will it be used

• Give people information about depression – De-stigmatize/normalize (common life experience) – High level of depression in Alaska due to multiple factors

(seasonal, trauma, situational) – Different types (situational versus chronic/organic)

• Assess/offer resources – SCF and non-SCF resources, including Customer-Owner

personal resources (social support) – Include non-medication resources (dietician, traditional

healing, individual counseling, and groups) – Explain potential benefits and burdens (side effects, wait

times) – Follow-up/make multiple contacts

• Use story/culturally consonant communication – Include Native faces/voices; include testimonials of success

Context

Information

Resources

Story

17

• Adapt for multiple audiences – iPad generally endorsed for providing screener and general

information, especially with younger people – Elders and others may prefer personal or written option

• Strategize location and timing – Could deploy screen in waiting area or provide general info on

closed-circuit TV in waiting area and use iPad for screen in examination or talking room

• Maximize customer choice – Make tool voluntary, not mandatory – Provide staff assistance with tool (e.g., RN or BHC) – Allow time for customers to explore options, including time in

examination room and in follow-up appointments

• Align with other initiatives – Minimize duplication and multiple requests for information

Stakeholder Recommendations: Decision Tool Design and Deployment

Alignment

Audience

Strategy

Choice

18

Factors Related to Workflow

• Administration of tool – BHC – PCP

• Role of tool – Facilitate discussion – Provide information – Support Decision

• Time

19

Example Screens of Resultant Decision-Support Tool

Impact of Engagement on Process

20

The Signs and Symptoms that Bother Me Most Are …

Feel Hopeless

Loss of Interest in Daily Activities

Weight and/or Appetite Changes

Sleep Changes

Anger or Irritability

Decreased Energy

Self Loathing

Trouble Concentrating

Pain

Exit

What can I do about my depression?

Healthy Eating

Exercise

Stress Reduction

Support Groups

Clergy

Counseling

Traditional Healing

Drugs

Herbal Remedies

Exit

Engagement Barriers and Proposed Solutions

• Barriers – Cultural – Time – Communication – Information

• Solution – Cultural-adaptation – Technology/resources – Decision-support tool – Decision-support tool

and extended tool

23

Other Example Screen Shots of Old and New Tools

24

Every day people face complex health decisions…

However, most healthcare decisions have no clear best choice.

0%10%20%30%40%50%

BeneficialOutcomes

ProbablyBeneficial

Need toWeigh

Benefitsversus Risk

Probably NotBeneficial

InsufficientEvidence ofUsefulness

Healthcare Intervention Classifications

25

Balance, Health and Wellness

Exit

What causes depression?

Life Experiences Biology

Exit

Money problems/ unemployment

Grief and loss

Violence Drugs/ Alcohol Abuse

Genetics Neurotransmitters (brain chemicals)

Hormones Other Medical Conditions

AcademyHealth ARM

PCORI Engagement Panel

ADDRESSING MENTAL HEALTH NEEDS OF RURAL

AFRICAN AMERICANS: LESSONS FROM ENGAGEMENT

Acknowledgements

Funding:

• UAMS Translational Research Institute (UL1RR029884; KL2RR029883) • Patient-Centered Outcomes Research Institute (PCORI)

This Presentation Will:

1. Describe the process of engaging stakeholders in our pilot project 2. Identify lessons about engaging stakeholders 3. Briefly describe the current status of the project

The setting: Arkansas Delta

• Primarily rural with agriculture-based economy

• Characterized by:

• Poor economic conditions

• Higher prevalence of chronic health conditions

• Increased risk of early mortality

• Poor access to quality health services

• Highest percentage of cities with predominately African-American populations

Mental Health in the Delta

• Poorer self-reported mental health • Increased levels of frequent mental

distress (BFRSS)

• Poorer mental health outcomes • Increased burden of disability

• Increased mortality

• Poorer management of chronic physical health

• Underuse of mental health services • Limited access

• Stigma

UAMS Psychiatric Research Institute

• Division of Health Services Research • Improve the lives of individuals with

mental health or substance use disorders

• Focus on rural areas

• Focus on partnership with key stakeholders

What can we do to improve mental health service use in rural

Arkansas?

LESSONS FROM

ENGAGEMENT

Decide who you need to partner with: • What type of expertise do you need to answer your

question?

• Ground

• People affected by the condition

• Grassroots • People and organizations who are near the ground

• Caregivers, support system, and service organization

• Grass tops

• Policymakers

The Partners: Tri County Rural Health Network

• Improves the health of individuals in the Delta • Uses community health worker model

• Connects community members to health resources in rural counties

• Works with UAMS College of Public Health to create “research friendly communities”

• Connects with community members and policymakers

Mental Health in the Delta

I’m worried about our minds. About our ability to cope with life. Especially with our young people. I

used to didn’t hear about depression or suicide. That was never something you heard about with

our people. Now, I have had to sit with several families of young Blacks who’ve taken their own

lives. I have people who are depressed in my congregation. Something has to be done.

- A pastor from the Arkansas Delta

LESSONS FROM

ENGAGEMENT

Determine if you are ready to engage: 1. Do you have similar values/missions?

• Both focused on improving health in rural areas

2. Do you have the capacity to work together?

• Tri County experience with research

• DHSR experience with community engagement

3. Will this partnership be mutually beneficial?

What’s the Solution?

• Focus on improving access to and use of mental health services • Adapt community-connector program

• Build on both partners’ expertise

• Start with identifying the mental health needs of rural African Americans • Important to hear the voice of the

community

DHSR’s Idea Tri County’s Idea

Tri County was originally created to provide job training in the Delta. But the people said, ‘We ain’t got no jobs here, so we don’t need no job training. What we do need is better healthcare.’ Let’s hear

the community before deciding where to start.

Our Idea

• Conduct formative research to:

• Better understand the mental health needs of rural African Americans • Inform the development of culturally appropriate interventions to improve

mental health in rural African American communities

Writing the Application

• Focus Groups • Researcher identifies stakeholders

• Researcher creates interview guide

• More traditional research method

• Deliberative Democracy • Allows community to self-identify as

stakeholders

• Community creates the frame

DHSR Methods Tri County’s Methods

The energy is important. Forums create energy around an idea. What would happen if we

implemented an intervention in an area that has this energy already buzzing?

Writing the Application

• New Questions:

• What is the best way to gather stakeholder input?

• Does one method lead to more activation?

• Is activation important in intervention implementation?

Specific Aims

• Compare two ways of gathering stakeholder input: deliberative democracy and focus groups • Themes

• Empowerment

• Gather information to inform the development of an intervention to

improve mental health in rural African American communities

LESSONS FROM

ENGAGEMENT

Engagement before Funding 1. Helps decide what are the most

relevant research questions • Whole idea changed as a result of partner input

2. More responsive questions= Better applications

Starting the Research

• Community Advisory Board

• One provider

• Two mental health consumers

• One connector

• One college student

• One clergy

• One lay community member

• Advises on consent procedures, measurement tools, recruitment, and

data analysis

Mental Health versus Emotional Wellness

No one is going to talk to you about mental health. When people hear mental health they

think of crazy. Ain’t nobody going to talk to you about being crazy. I ain’t crazy. I don’t know how

to help you help crazy people

If you want my expertise, you have to ask me about things I know about.

CAB member

Mental Health versus Emotional Wellness

• Based on CAB feedback: • Application written using the medical frame

• Wellness framework is more culturally acceptable

• Interview guides and framing sessions used term “emotional wellness”

instead of “mental health”

LESSONS FROM

ENGAGEMENT

Respect your partner’s expertise 1. Partner is a part of the research team

• Not just approval

2. Listen and incorporate ideas when possible

• Using wellness frame versus illness frame

• Changing wording of measurement tools

• Not paying unless they finish

3. Respect builds trust • Trust builds better partnerships

LESSONS FROM

ENGAGEMENT

Prepare for some differences of opinions • Audio recording versus note taking

But remember: • Your core values; what are you working towards

• Respect each other’s point of view and expertise

Where are we now?

• Completed six focus groups (n=50) • Faith community

• College students and administrators

• Patients

• Providers

• Currently completing community forums (n=86)

• Four forums completed (two more planned)

• Lay community members

• Service organization leaders

• Political leaders

Where are we now?

• Preliminary analysis

• Stigma and low mental health literacy major barriers to care • Importance of “reaching people where they are”

• Community-based services versus clinic-based services

• Importance of community support in prevention, treatment, and recovery • Provide education and support

• Address contextual causes that affect emotional wellness

Conclusions

Engagement can lead to: 1. More culturally acceptable research questions

2. Applications that are more responsive to community needs

When engaging stakeholders in research:

1. Make sure you are ready to engage in research

2. Engage stakeholders before funding

3. Respect your partners’ expertise

4. Prepare for differences of opinions

Any real attempt to address the problems that are plaguing our community will take everyone coming to the table; the grassroots, the

grass-tops, and all the grass in between. Everyone talks about how difficult it is to bring everyone together, but if you focus on solving the

problem, the mountains become just bumps in the road.

Mary Olson, Tri County Rural Health

Academic Team

Greer Sullivan, MD MSPH (PI) Mary Olson, D. Min (Community PI)

Geoff Curran, PhD Ann Cheney, PhD

Keneshia Bryant, PhD FNP Christina Reaves, MPH

Naomi Cottoms Elise Allee Faye Smith

Community Advisory Board

Rev. George Barnes Linda Cole

Melva Trask Gloria Scott

Edlun Marshall Pamela Barnes Earnest

Virgil Ward