board of governors meeting, south sea tea washington
DESCRIPTION
Slide presentation from the September 19-20, 2011 Board of Governors Meeting in South Sea Tac, Washington.TRANSCRIPT
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
September 19, 2011
Patient-Centered Outcomes Research Institute
Communications, Outreach and Engagement Committee (COEC) Report
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Members
2
• Sharon Levine, MD (Chair)
• Debra Barksdale, PhD, RN
• Robert Jesse, MD, PhD
• Grayson Norquist, MD, MSPH
• Ellen Sigal, PhD
• Harlan Weisman, MD
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Charter
3
The Communication, Outreach and Engagement Committee shall advise and
assist the Board of Governors of PCORI, and provide recommendations to the
Board, regarding:
1. The Institute’s communications and branding work,
2. Strategies to engage all stakeholders in the work of PCORI, and
3. Methodologically sound approaches to disseminating and implementing the
research results and ensuring their utility to patients
and clinicians
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
COEC Report to the Board
4
I. Recent Opportunities for Public Input (Update)
II. Promoting PCORI RFPs (Update)
III. Redesigned PCORI Website (Update)
IV. Outreach Plan for Pilot Project Funding Announcement (Recommendation for Decision)
V. Stakeholder Outreach (Update)
VI. Speakers Bureau (Update)
VII.PCORI Communications Staff (Update)
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Recent Opportunities for Public Input
5
• Working definition of “patient-centered outcomes research”
– Nearly 600 responses were received (July 20-September 2, 2011)
– All responses received through the website or emailed PDFs will be posted on pcori.org
– PCORI issued an RFP to solicit proposals for the analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Analysis and Summary of Input on Working Definition of PCOR
6
Issue RFP (and reach out to potential applicants) August 24, 2011
Deadline for applications September 9, 2011
Complete review of applications September 16, 2011
Complete contract with chosen contractor September 19, 2011
Submit Phase I findings and draft questions for focus groups
September 30, 2011
PCORI completes revision of PCOR Definition October 26, 2011
Revise questions for focus groups November 9, 2011
Focus Groups (or other way of seeking patient input) November 18, 2011
Report due to PCORI December 16, 2011 (with preliminary report in time for December 8 meeting)
PCORI completes further revisions of PCOR Definition January 15, 2012
PCORI Board considers revised PCOR Definition January 18, 2012
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Working Definition of PCOR: A Foundation for PCORI’s Work
7
• At the time of PCORI’s creation, “patient-centered outcomes research” (PCOR) was not a commonly used term, nor had it been formally defined
• Defining PCOR helps clarify PCORI’s focus and scope of work
• Through iterative and transparent processes that elicit feedback from all stakeholders, including patients, we will continue to revise the definition, ensuring it:
– Emphasizes the “patient-centered” focus of PCORI’s mission
– Is consistent with the intent of the statute that established PCORI
– Is broad enough to support the range of research PCORI should fund
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Recent Opportunities for Public Input
8
• Initial topics for PCORI pilot projects.
– More than 150 responses were received (August 1-30, 2011)
– Responses provided to the PDC Working Group to review and update topics as appropriate.
• 89.26% Self Identified
• 10.74% Anonymous
• 59.76% Responded as Individual
• 40.27% On behalf of an Organization
Respondents
• 40.94% Academia
• 23.49% Provider
• 17.45% Patient
• 3.36% Industry
• 0.67% Government
• 11.41% Non-Identified
Organizational Respondents
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Promoting PCORI RFPs
9
• To date PCORI has issued 4 Requests for Proposals (RFP)
– Analysis of Input Received on Working Definition of “Patient-Centered Outcomes Research”
– Review and Synthesis of Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Literature Review)
– Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient's Perspective in Patient-Centered Outcomes Research (Interviews)
– Methods for Setting Priorities in Research (White Papers)
• For each RFP, PCORI performed outreach to: email list subscribers (620+), supplemental lists of clinical and translational research centers (60+), academic research institutions (30+), and provider and advocacy organizations (30+)
• PCORI is an independent, non-profit, non-governmental organization. All PCORI funding announcements will be posted on pcori.org. PCORI funding announcements will not be published in the Federal Register
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Redesigned PCORI Website
10
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Redesigned PCORI Website
11
• Includes visual changes based on logo and color palette approved at July Board Meeting
• Designed to be more user friendly
• Will support the expansion of PCORI content and opportunities for interaction as work progresses
• Expanded features include:
– Easier mechanisms for joining PCORI mailing list and providing general input
– An “Executive Director’s Corner” that provides regular updates on PCORI’s latest developments
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Outreach Plan for Pilot Project Funding Announcement
12
• Post announcement on pcori.org
• Direct email to stakeholders and potential applicants
• Conduct proactive media relations
• Promote the opportunity though grassroots outreach
• Integrate the announcement with speakers bureau presentations
• Host a Q&A teleconference for applicants
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Stakeholder Outreach
13
PCORI’s engagement with stakeholders around Board meetings
• Stakeholder discussion forum March
St. Louis, MO
• Stakeholder discussion forum May
New York, NY
• Two small group meetings with patients and caregivers
• Eight small group meetings with 43 stakeholder organizations
July
Washington, DC
• Invited presentations from Northwest Pacific stakeholders September
Seattle, WA
• Site visits to clinical care facilities November
New Orleans, LA
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Seattle Stakeholder Engagement Forum
14
• PCORI has invited individuals and organizations from the Pacific Northwest to make brief presentations to the Board about their work or perspective and how it can inform PCORI’s work
• Native American/Alaska Native perspectives
• Complementary and alternative medicine researchers and providers
• Outcomes and comparative effectiveness researchers
• Monday, September 19, 7:00-9:00 p.m. PT
• Open to the public
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Speakers Bureau
15
PCORI has presented at 24 meetings since March
Recent Speaking Engagements
• 7/25 – IOM CER Innovation Collaborative Meeting
• 7/25 – Center for Medical Technology Policy CER Institute
• 8/12 – Consumers United for Evidence Based Health Care Annual Meeting
• 8/25 – Florida Chiropractic Association National Convention & Expo
• 9/9 – University of Michigan School of Nursing
• 9/16 – Oregon Institute for Patient-Centered Comparative Effectiveness Annual Research Intensive
Upcoming Opportunities
• 9/27 -- AdvaMed 2011 MedTech Conference
• 9/28 – Partnership to Improve Patient Care Annual Membership Forum
• 10/11 – NIH National Center for Research Resources Clinical and Translational Science Awards Program
• 10/12 – Comparative Effectiveness Research Summit
• 10/12 – American Academy of Nursing, Council of Advancement of Nursing Science 2011 Special Topics Conference
• 10/19 – Health Industry Forum
• 10/20 – Pfizer Health Advocates Leadership Breakfast
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PCORI Communications Staff
16
• Seeking applicants for three positions:
– Director of Communications
– Director of Patient Engagement
– Director of Stakeholder Engagement
• Staff will develop and implement strategic communications plan
• Distinct engagement officers for patients and other stakeholders, recognizing their need for different engagement mechanisms
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
September 19-20, 2011 Seattle, WA
Executive Director Update Board of Governors
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Objectives for today
Highlight PCORI Activities since July 2011 BoG Meeting
Discuss Growing the PCORI Staff
Consider Critical Timelines/Milestones for Next 9 Months
2
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PCORI Activities – (Staff, Board, MC together)
Establishing Policies and Practice
HR Policies
Purchasing/RFP policies
COI policies
Research Policies
Re-designing Website
Outreach
Developing IT Infrastructure
Growing Staff
Finding Long-term Space
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
4
• $48 per sq. foot • Attractive, not • extravagant • 13,000 sq feet • Close to Metro • Green building • Move-in: February
PCORI’s Long-Term Home 1828 L Street, DC
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Urgent Issues Coming Out of July BoG Meeting:
Address the need for greater clarity in decision-making and take steps to find more time for substantive discussions among board, with MC
Patient Engagement! - Officer, Advisory Panel
Get started on the National Priorities and Research Agenda!
Support the PDC in preparing PCORI Pilot Projects RFP and review
Support the MC in getting the RFPs out for the Methodology Report
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Patient Engagement
6
Patient-Engagement Working Group formed; Job Description for Director of Patient Engagement drafted Directors of Stakeholder Engagement and Director of Communications Added All three positions are posted and a search is underway
Patient Engagement
Communi-cations
Stakeholder Engagement
External Engagement
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Program Development Committee
National Priorities and Research Agenda
• C Clancy and A Epstein, co-chairs
• Environmental scan on past CER analysis and developed an initial framework for the National Priorities which will serve as the foundation for developing specific priorities
• Recognized the close link between the National Priorities and Research Agenda (H Krumholz and L Hole-Curry, co-chairs)
• Developed a timeline and notional process for engaging stakeholders as part of the National Priorities development timeline
7
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Developing PCORI’s National Priorities and Research Agenda
8
Collect Stakeholder
Input
Develop Candidate Priorities
Public Comment
Finalize Priorities
Develop Candidate Priorities
Underway Sept - Nov Dec-Feb March
Collect Stakeholder
Input
Public Comment
Finalize Research Agenda
April Feb - Mar Sept - Jan
PRIORITIES:
RESEARCH AGENDA:
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Program Development Committee PCORI Pilot Projects
C Goertz and G Hunt co-chairing this process.
Public Input has been reviewed and incorporated into revised “areas of interest” in the PCORI Funding Announcement (PFA)
An application form and instructions have been developed and revised
Working closely with NIH staff on defining each step in the application and review process
8
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Supporting Methodology Committee
Interim researcher process established; at least one already working
RFPs (2) for Patient-centeredness Working Group posted
RFP (1) for Research Prioritization Working Group posted
Review Process in Development, including COI policies for review
9
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Definition of PCOR
Input on the working definition of “patient-centered outcomes research” gathered (n=600); to be posted shortly after anonymization
RFP posted for qualitative analysis and summarization of the input received and for recommendations on methodologies to receive further input from patient audiences posted on PCORI website
12 responses received and reviewed
First PCORI research contract issued on 9/16 to National Opinion Research Center
II MC I
9/16 9/30 11/15 12/31
Synthesis & Question Generation
Revision of Definition
Testing of Revised Definition
Preparation of MS for Publication Led by MC
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Three Engagement
Directors Applications
Arrive LOI’s
Arrive
Growing PCORI Staff:
09/18/11 –
09/20/11
We are here
11/1/11 2/1/12 05/1/12 04/1/12
11
1/1/12
COO
Begins
Mark
In!
Director
Finance
Begins
First
Scientists
Additional
Scientists
Additional
Admin
Ass’ts
Director of
Staff
12/1/11 3/1/12
Funding
Decision
Made
Funding
Issued
Program
Staff Hiring
Begins
Finance/Grant
Staff
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Growing PCORI Staff
13
Research Scientist Patient engagement
Research
Research Scientist Stakeholder Engagement
Research Scientist Communicaiton/Disseminat
ion Research
Research Scientist(s) Comparative Clinical
Effectiveness Research
Director Patient Engagement
Director* Stakeholder Eng.
Director Communications
Chief Financial Officer (CFO)
Director HR
Director IT
Director Grants Management
Research Scientist(s) Evidence Synthesis
External Engagement
Executive Director
Chief Operating Officer Chief Science Officer
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Critical Issues: September 2011 – June 2012
14
Gathering Stakeholder Input on National Priorities and Research Agenda
Thorough, balanced, incorporated
September – November:
November- December:
Finalizing First Draft of Priorities (Prior to Public Comment)
How are Priorities expressed, what’s in, is anything out?
January – February:
Finalizing First Draft of Research Agenda (Prior to Public Comment)
What (specifically) is in, out – what will the first RFAs fund?
Data Infrastructure Grants?
Translation/Implementation Research?
Systems research?
Pharmacogenetics?
Large clinical trials?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Finance & Administration Committee September 20, 2011, Seattle, WA
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Bylaws Revisions
• GAO currently reviewing
• Committee Name Change: Finance, Administration & Audit
• Adoption Proposed at November BOG Meeting
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Audit Update
• Released this week
• Bidders selected by annual revenues
• Proposals by end of October
• Recommendation to BOG
• Field work in November/December
• Completed audit by March
• Provided to GAO for report to Congress due April 1st
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Financial Statements – August 31, 2011
• Statement of Financial Position: Total Assets of $46.3M; $5M Cash on Hand
• Expenditures:
– Budget: $7.5M
– Actual: $3.7M
– Positive variances in Board compensation, management fees, professional services, travel
– Negative variance in conferences
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
Conflicts of Interest Policy
• Working group created
Members: Larry Becker, Chair
Debra Barksdale
Arnie Epstein
Sherine Gabriel
Bob Jesse
• Group is gathering key documents
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
1
September 19, 2011
PCORI Methodology Committee Progress Report
Sherine E. Gabriel, MD, MSc Sharon-Lise T. Normand, PhD
PCORI Methodology Committee
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
2
• Establishment
• Overview
• Timeline
• Structure
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology Committee Workgroups
3. Wrap Up
• Next Steps
Agenda
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
3
Methodology Committee Charge
Establishes priorities to address gaps in
research methods and their application
The Methodology Committee shall make
recommendations regarding methods for
patient-centered outcomes research
Provides guidance about the appropriate
use of methods
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
4
Methodology Committee Structure
Methodology Committee Co Chairs: Sherine Gabriel, MD, Mayo Clinic & Sharon-Lise Normand, PHD, Harvard Medical School
Patient-Centeredness
Research Prioritization
Methods to incorporate the patient perspective into all phases of PCOR
Methods to inform prioritization of new research studies
Methods for using data, design, and statistical analyses to conduct PCOR
Research Methods
Re
po
rt A
ssim
ilati
on
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
5
Methodology Committee Activity Timeline
Rese
arc
h
Me
tho
ds
Aug „11 Oct „11 Nov „11 Dec „11
Pa
tie
nt-
Cen
tere
dn
ess
Jan „12 Feb „12 Mar „12 Apr „12 May „12
Workshop Final
Report
Workshop
Identify
Gaps/
Issues
Translation
Table/Tool
Workshop Final
Papers
Due
Final
Draft Final
Report
Final
Report
Compile
First
Draft
Legend
Re
se
arc
h
Prio
ritiza
tio
n
Rep
ort
Assim
ilatio
n
Feb – Jul „11
Key Workgroup Activities MC Activities
Revise Report Outline & Edit Chapters
Achievements to Date
Final
Report
Final
Papers
Due
Electronic Data Systems Landscape Review
Identify & Implement Voting
Standards Process
Solicitations: White Papers/Identify Experts
Solicitations: Literature Review & Interviews
MC
Chart
er
MC
Work
Pla
n
Inte
rim
Researc
hers
Sta
ffed
PC
OR
Definitio
n S
olic
itation
Commission White Papers
Sep „11
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
6
Methodology Committee Discussion Points
• Importance of methods generated by Patient Centeredness Work Group for incorporating patient engagement
• Importance of methods generated by Research Prioritization Work Group vis-à-vis prioritization framework set by the board
• Generation of tools/standards for Methods Report is a multiyear process
• Importance of continuing to interact with Board to ensure coordination of efforts
• Board/Methodology Committee teleconference calls were a productive forum to receive input from the Board regarding their needs and priorities
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
7
• Establishment
• Overview
• Structure
• Timeline
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology Committee Workgroups
3. Wrap Up
• Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
8
Patient-Centeredness Workgroup
Ethan Basch, MD
Memorial Sloan-Kettering
Cancer Center
Workgroup Chair
Mary Tinetti, MD
Yale University School of
Medicine
Workgroup Co-Chair
Naomi Aronson, PhD
Blue Cross and Blue Shield
Workgroup Member
Brian Mittmann, MD
VA Center for Implementation
Practice & Research Support
Workgroup Member
David Flum, MD, MPH
University of Washington
Report Assimilation Group Liaison
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
9
-
Patient Centeredness Mission/Scope of Work
Development and prioritization of research
questions
Identify methodological standards for incorporating
the patient perspective into three key areas:
Design of study components, including
selection of interventions, comparators, and outcomes (including patient-reported
outcomes)
Processes of clinical decision-making/care
delivery
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
10
Patient Centeredness Progress to Date
Released RFP
for Literature
Review
Released RFP
for Interviews
“Review and Synthesis of
Evidence for Eliciting the
Patient‟s Perspective in
Patient-Centered
Outcomes Research”
“Expert Stakeholder
Interviews to Identify
Evidence for Eliciting the
Patient‟s Perspective in
Patient-Centered Outcomes
Research”
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
11
Proposals Due
Awardee Announced
Final Report Due
Oct 6, 2011
Oct 13, 2011
March 1, 2012
Issue Open RFI Plan Workshop
Develop Sections
for Methodology
Report
Patient Centeredness Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
12
Patient Centeredness Question
How does this work complement ongoing and planned activities of the
Board?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
13
Research Prioritization Workgroup
David Meltzer, MD, PHD
University of Chicago
Department of Medicine
Workgroup Chair
John Ioannidis, MD, DSc
Stanford Prevention
Research Center
Workgroup Member
Jean Slutsky, PA, MSPH
Agency for Healthcare
Research and Quality
Workgroup Member
Clyde Yancy, MD
Northwestern University
Feinberg School of Medicine
Workgroup Member
Alfred Berg, MD, MPH
University of Washington
Department of Family Medicine
Report Assimilation Group Liaison
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
14
Research Prioritization Mission
Mission
To provide guidance concerning the use of methods to inform the establishment of research prioritization approaches that best fulfill PCORI’s mission.
• To the PCORI Board to aid in development and future refinement of Research Priorities]
• To the broader multi-stakeholder communities to enhance understanding of PCORI’s approach to research prioritization and to encourage their engagement in PCORI’s research prioritization process
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
15
-
Commission White
Papers
To prepare a section for the May 2012 Methodology Committee Report that addresses methods to inform the establishment of research
prioritization
Synthesize White Papers during January 2012
Workshop
Research Prioritization Scope of Work
Topic Generation
Gap Analysis in
Systematic Reviews
Value of Information
Analysis Peer Review
Suggest strategies to identify possible future
priority areas
Suggest standards for how PCORI systematic reviews should be performed and used to generate research
topics
Provide PCORI Board and/or grant applicants with tools
to quantify expected benefits of research to
inform priorities
Provide data to inform how PCORI might design,
evaluate and continually improve its peer review
process
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
16
Research Prioritization Progress to Date
Identified
Methodological
Areas of Interest
in Research
Prioritization
Identified
Interim
Researcher to
Assist
Workgroup
Released RFA
for White
Papers on
September 9,
2011
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
17
White Paper Proposals Due
Review and select among White Paper proposals
Awardee Announced
Sept 30, 2011
Oct 1- 13, 2011
Oct. 14 2011
Explore the role of research prioritization methods vis-à-vis the broader
PCORI research prioritization process
Engage researchers as needed to fill in expected gaps from White Papers
Research Prioritization Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
18
Research Prioritization Question
How can we best work with you to understand your evolving plans for
research prioritization so the methods we are studying can be of
most use to you?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
19
Research Methods Workgroup
Steven Goodman, MD, MHS, PHD
The John Hopkins University
School of Medicine and Public Health
Workgroup Chair
Mike Lauer, MD
National Institute of Health
National Heart, Lung, and
Blood Institute
Workgroup Member
Robin Newhouse, PHD RN
University of Maryland
School of Nursing
Workgroup Member
Sebastian Schneeweiss,
MD, ScD
Harvard Medical School
Workgroup Member
Mark Helfand, MD MPH MS
Oregon Health & Science University
School of Medicine
Report Assimilation Group Liaison
Sharon-Lise Normand, PHD
Harvard Medical School
Methodology Committee Chair
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
20
-
Develop Translation
Table/Tool (from research question to design and
analysis)
Identify Standards for the use of data, design, and statistical analyses to
conduct patient centered outcomes research through:
Review current methodological “state of
the art” statements issued by expert bodies to guide
PCORI research
Research Methods Mission/Scope of Work
Review existing data systems designed to
permit exploration of causal questions from data gathered in the course of clinical practice. Identify PCORI role in this arena.
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
21
Research Methods Progress to Date
Established initial
specifications for
dimensions to be used
in translation
instrument, with
options for
implementation
Developing
prototype for
standards
documents that
combines expert
statements,
published examples
and PCORI MC input
Interviewing leading
researchers of
electronic data, from
which research and
methods
recommendations
will be developed
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
22
Research Methods Progress to Date (Detailed)
Framework for
Methods Standards /
Recommendations
1. Method 2. Key sources 3. Major recommendations 4. PCORI MC commentary 5. Published examples 6. Tools for researchers
Translation Table
Dimensions
Intrinsic Factors • Internal validity (aka bias) • External validity (aka
generalizability, transportability) • Precision • Heterogeneity in risk or benefit
(aka “personalized” evidence) • Ethical dimensions
Extrinsic Factors • Timeliness (Rapidly changing
technology, policy urgency) • Logistical burden (e.g. study
size, complexity, cost) • Constraints (Data availability,
randomization possible?)
Question: condition, population, treatment & comparator, outcomes, setting
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
23
Finalize translation table dimensions and
determine categories of development
Finalize “standards” document format and determine categories of development
Finish and summarize electronic data
systems environmental scan
Research Methods Next Steps
(e.g. diagnostic tests, missing data, systematic reviews, etc.)
(e.g. drug safety, therapeutic
efficacy, etc.)
Jan. 2012 Workshop
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
24
Research Methods Question
Can the Board provide feedback on the preliminary formats developed (i.e., translation table dimensions and methods standards format)?
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
25
Report Assimilation Workgroup
Mark Helfand, MD
Oregon Health & Science University
School of Medicine
Workgroup Chair
Alfred Berg, MD, MPH
University of Washington
Workgroup Member
David Flum, MD, MPH
University of Washington
Workgroup Member
Howard Balshem, MS
Oregon Health & Science University
Interim Researcher
Sherine Gabriel, MD
Mayo Clinic
Workgroup Member
Sharon-Lise Normand, PHD
Harvard Medical School
Workgroup Member
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
26
• Establishment
• Overview
• Structure
• Timeline
• Mission
• Scope of Work
• Progress to Date
• Next Steps
• Questions
1. Methodology Committee
2. Methodology Committee Workgroups
3. Wrap Up
• Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
27
Solicitation Responses and Questions Received to Date
Methodology Committee Next Steps
Letters of Intent Received**
Questions
Patient Centeredness WG -- Stakeholder Interviews
25 8*
Patient Centeredness WG - Literature Review
26 4*
Research Prioritization WG - White Papers
11 0
**as of 9/16 at 8:30 a.m. *This number of questions includes a document containing 13 individual sub-questions.
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
28
Administrative Review
Content Review
Programmatic Review
Executive Decision
1. Clear, transparent and reproducible review process based on best
practices (eg. NIH)
2. Aligned with PCORI Vision, Mission, Principles and Policies (eg CoI)
3. Timely (~Applicants notified of award two weeks after submission)
Will be Established prior to receipt of first response (September 30, 2011)
Solicitation Review Process and Principles
Methodology Committee Next Steps
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
29
Methodology Committee Next Steps
Rese
arc
h
Me
tho
ds
Aug „11 Sep „11 Oct „11 Nov „11 Dec „11
Pa
tie
nt-
Cen
tere
dn
ess
Jan „12 Feb „12 Mar „12 Apr „12 May „12
Workshop Final
Report
Workshop
Identify
Gaps/
Issues
Translation
Table/Tool
Workshop Final
Papers
Due
Final
Draft Final
Report
Final
Report
Compile
First
Draft
Legend
Re
se
arc
h
Prio
ritiza
tio
n
Rep
ort
Assim
ilatio
n
Feb – Jul „11
Key Workgroup Activities MC Activities
Revise Report Outline & Edit Chapters
Achievements to Date
Final
Report
Final
Papers
Due
Electronic Data Systems Landscape Review
Identify & Implement Voting
Standards Process
Solicitations: White Papers/Identify Experts
Solicitations: Literature Review & Interviews
MC
Chart
er
MC
Work
Pla
n
Inte
rim
Researc
hers
Sta
ffed
PC
OR
Definitio
n S
olic
itation
Commission White Papers
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE
30
Thank You!
1
Presentation to PCORI Board of Governors September 20, 2011
PATIENT-CENTERED OUTCOMES RESEARCH INSTITUTE:
Program Development Committee
2
Program Development Committee
Committee Members
• Carolyn Clancy
• Francis Collins
• Arnold Epstein
• Christine Goertz
• Leah Hole-Curry
• Gail Hunt
• Harlan Krumholz
• Richard Kuntz
Methodology Committee Representatives
• Sherine Gabriel
• Sharon-Lise Normand
3
Report Outline
PCORI Pilot Project Grant Program
National Priorities and Research Agenda
Landscape Review Program
Program Development Committee
4
Presentation to PCORI Board of Governors September 20, 2011
Christine Goertz, DC, PhD
PCORI Pilot Projects
5
Report Outline:
• Purpose of PCORI Pilot Projects
• PCORI Funding Announcement - Key Highlights
• Areas of Interest
• Overview of the Four-Stage Review Process
• Application Review Criteria
• Grant Application & Instructions
• Webinar & Process to Handle Inquiries
• Timeline, Key Dates, and Activities
6
Purpose of PCORI Pilot Projects
The purpose of the PCORI Pilot Projects Grant Program is threefold: 1. National Priorities. Pilot projects will provide information to
PCORI that informs future iterations of national research priorities for patient centered outcomes research.
2. Research Agenda. The program will support the collection of preliminary data that can be used to advance the field of patient-centered outcomes research, providing the platform for an evolving PCORI research agenda.
3. Methodologies. The program will support identification of research methodologies that advance patient-centered outcomes research
7
PCORI Funding Announcement: Key Highlights
Applicants must address at least one of the eight areas of interest.
Stakeholder involvement is required unless the application can sufficiently explain why it is not feasible.
Expect to commit approximately $13 million in FY 2012 to support approximately 40 projects.
Applicants can propose project periods of up to two years. Second year funding is subject to noncompetitive review by PCORI staff.
Direct costs are limited to $250,000 per year with an additional 40% of salary and fringe allowable as indirect costs.
Letters of intent are due on November 1 with applications due on December 1.
NIH will conduct the merit review process with all other aspects of grant awards and grants management handled by PCORI.
8
Areas of Interest: Development
Areas of Interest
Board Members
Methodology
Committee
Stakeholder / Public Input
Projects focused on developing, testing, and/or evaluating methods or approaches that: Inform the evolving
National Priorities Bring together different
stakeholders Translate research into
practice Identify gaps in relation to
issues for disadvantaged populations
Identify predictors of patient outcomes
9
Areas of Interest
Developing, testing, refining, and/or evaluating new or existing methods (qualitative and quantitative) and approaches that can inform the process of establishing and updating national priorities for the conduct of patient-centered outcomes research (PCOR). This may include research prioritization approaches (such as Value of Information (VOI), burden of illness, peer review/expert opinion/Delphi approaches) or methods for incorporating the perspectives of patients or other stakeholders into the development of national priorities.
Developing, testing, and/or refining existing methods for bringing together patients, caregivers, clinicians including non-traditional partners, and other stakeholders in all stages of a multi-stakeholder research process, from the generation and prioritization of research questions to the conduct and analysis of a study to dissemination of study results – including methods for training participants in participatory research and the potential use of new technologies to facilitate engagement.
Developing, refining, testing, and/or evaluating patient-centered approaches, including decision-support tools, for translating evidence-based care into health care practice in ways that account for individual patient preferences for various outcomes. This may include developing or comparing conceptual models of translation or dissemination of CER research findings from the patient perspective.
Developing, refining, testing, and/or evaluating methods to identify gaps in CE knowledge such as tools for the ongoing collection and assessment of gaps as perceived by patients and providers. Of special interest are gaps that are particularly relevant to vulnerable populations, including but not limited to, low-income populations; minorities; children; elderly; women; people with disabilities, chronic, rare and/or multiple medical conditions.
10
Areas of Interest
Identifying, testing, and/or evaluating patient-centered outcomes instruments. This may include predictive tools (eg: instruments that measure or predict outcomes of interest to patients) or identifying standards for measurement properties of patient-reported outcomes for use in comparative effectiveness research, across a variety of interventions and patient populations.
Identifying, testing, and evaluating methods that can be used to assess the patient perspective when researching behaviors, lifestyles, and choices within the patient’s control that may influence their outcomes.
Identifying, testing, refining and/or evaluating methods for studying the patient care team interaction in situations where multiple options for wellness, prevention, diagnosis or treatment exist. Of special interest are strategies that respect patient autonomy and promote informed decision-making, incorporating the best health care knowledge into the application of care.
Advancing analytical methods for CER. Examples include but are not limited to the incorporation of mixed methods research designs (qualitative/quantitative), identifying existing methodology to statistically accommodate irregularly spaced multivariate longitudinal data, the use of instrumental variables; and potential solutions for assessing treatment heterogeneity in observational and randomized CER studies.
11
Overview of the 4-Step Process
Preliminary
Check
Merit Review
Deliberation
Business
Determination that the application meets the minimum technical requirements enumerated in the PFA and fits within programmatic priorities.
PCORI Responsive/eligible applications will be forwarded for review.
Remaining applications are assigned to 3 reviewers with appropriate expertise who will critique independently and then meet to discuss and designate a final score.
NIH A rank-ordered list of applications and summary sheets
The Pilot Projects Review Committee considers the merit review scores and programmatic balance across several categories to develop a recommended award slate for consideration by the BOG.
PCORI Funding decisions are made.
A final review for suitability to manage an award, adherence to human subjects requirements, receipt of IRB approval, budget review, etc.
PCORI Negotiations, where needed
Description Responsible Result
12
Preliminary Check
Technical Requirements
• Receipt of Letter of Intent
• Eligible applicant
• Stakeholder requirements
• Page limit and formatting requirements
• Deadline met and application properly submitted
Programmatic Requirements
• Does the application fit within PCORI program requirements listed in the PFA?
Transfer to NIH
• Applications meeting both technical and programmatic requirements will be forwarded to NIH for the Merit Review.
Conducted by PCORI staff
13
Merit Review
Prior to the Review Meeting
• Each application is assigned to 3 reviewers
• Reviewers include scientists and other stakeholders
• Stakeholders without previous review experience will be trained
• Reviewers assign an initial priority score of 1 to 9 based on PCORI-provided review criteria
• Provides basis for discussion at in person meeting
In-Person Review
• In-person meeting of reviewers to discuss the most promising applications
• Review and discuss applications based on impact, stakeholder involvement, innovation, and significance of the science
• Provide a final priority score of 1 to 9
• Scientific Review Officer compiles a summary statement with reviewer critiques for each application
Summary & Rankings
• The review process results in a ranked, scored summary of applications for PCORI consideration
Conducted by NIH
14
PCORI Deliberations
PCORI Staff
• Analyzes the applications using priority score and classification categories.
• Creates funding scenario options based on analytics.
• Supports the PCORI Review Committee and Board of Governors by providing revised options, as needed.
PCORI Review Committee
• Meets to review materials
• Considers the balance of priority-scored applications across classification categories, requesting additional analysis and options from staff, as needed
• Prepares a recommended slate of selected projects for funding consideration
Board of Governors
• Meets to consider the recommended slate
• Reviews the slate based on priorities and balance to ensure appropriate distribution
• Requests additional options, if needed
• Approves a final slate of selected projects for funding
Classification categories may include: Applicability of the 8 areas of interest, geographic distribution, research methodology, and identification of target populations.
15
Recommended Funding Slate Analysis
Following the Review Committee meeting, analysts will prepare the recommended slate along with accompanying analysis for consideration by the Board of Governors.
16
Merit Review Criteria
Significance. Does the project address an important problem or a critical barrier to patient-centered outcomes research? Does the project address one of the key questions outlined in the PCOR definition described above? Is the project focused on one of the areas of interest identified in this PFA? Will the results produce new knowledge that can advance PCOR methods or infrastructure? Does the investigator demonstrate thorough knowledge of previous and ongoing work related to their proposed topic?
Patient/Stakeholder Engagement. Will the research make a unique contribution to learning about engagement of patients and non-traditional stakeholders in PCOR research efforts? Does the research team demonstrate authentic, feasible, sustainable, novel partnerships with patients, families and care givers, providers, and other appropriate non-traditional stakeholders? Is there evidence that non-traditional stakeholders were involved in the preparation of the research proposal?
Investigator(s). Is the research team well suited to the project? Is it multi-disciplinary? Is there appropriate scientific expertise? Does at least one member of the study team have experience in patient and other stakeholder engagement? Is there a high level of confidence that the Principle Investigator and rest of the study team will be able to achieve the study aims as described? Does the study team have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?
17
Merit Review Criteria (continued)
Innovation. Does the project either address a new method or approach or apply a proven method or approach in a novel way to the field of PCOR? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?
Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? Will the strategy establish feasibility and will particularly risky aspects be managed? Is the proposed budget and timeframe appropriate for the research plan? Are there appropriate plans for dissemination among key PCOR stakeholders in education, practice and policy?
Environment. Will the environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the research environment, community involvement, patient populations, or non-traditional stakeholder collaborative arrangements?
18
Grant Application & Instructions
• Application is based on the NIH PHS 398 form with changes to ease responses and to meet the needs of PCORI.
• Applicants will be bound by the Human Subjects policies of NIH and will include appropriate plans within the Research Strategy section.
• Applicants will provide information related to classification categories to enable the Review Committee and Board to consider this information when developing a balanced slate for funding.
Self-Reported Classification Categories:
• Applicability of the “4
questions” to the research • 8 areas of interest • Identification of specific
populations including:
― Underserved or disadvantaged populations
― Specific ethic or cultural populations
― Rural or urban populations
― Disabled populations
19
Webinar & Inquiry Process
• Webinar for Applicants: October 12, 2011 at 2PM EST.
• Policy for responding to inquiries:
– A set of FAQs will be developed and issued along with the PFA with frequent updates posted on the PCORI website. Questions and answers will be framed in terms of general applicability.
– Inquiries should be directed to the PCORI website at pcori.org. PCORI staff and appropriate consultants will receive, triage, route, and ensure answers are provided and shared generally through the website.
20
Timeline, Key Dates, and Activities
2011 09 /11
PFA Posted
11/01/11 Letters of
Intent Due
12/1/11 Applications
Due
11-12/11 Identification of Merit Reviewers
2012 12/11
Completeness/ Compliance/
Eligibility Check
2/12 Merit Review
3/12 Slate of Projects
to Board & Selection Made
05/12/12 Award
Notification & Funding
PCORI Applicant NIH/PCORI Applicant
PCORI NIH PCOR PCORI / Applicant
21
Questions?
www.pcori.org
22
Request Approval
23
Arnold Epstein, MD Presentation to PCORI Board of Governors September 20, 2011
PCORI National Priorities Development
24
National Priorities are the critical beginning and will be dynamic
Incre
asing Sp
ecificity
National Priorities
Research Agenda
Individual PFAs
25
Develop
Candidate Framework
Engage
Stakeholders
Public
Comment
Finalize National Priorities
Currently Underway
September-November
December-February
March
Proposed Timeline for National Priorities
26
Environmental scan of existing priorities and
criteria
Candidate priorities and
criteria identified
Framework to inter-relate
Priorities and Criteria
Reviewed initial stakeholder input advising us to not “reinvent the wheel.”
Reviewed prior CER frameworks (e.g., IOM, FCCCER, National Priorities Partnership, NQF)
Identified broad priorities and criteria that were used often in prior frameworks and fit PCOR.
Framework to be used for refining priorities, and determining Research Agenda and PFAs.
Initial Stakeholder
feedback
Development of a National Priorities Framework
27
Previous priorities have varied in granularity
Granularity
IOM 2009 (e. g., Compare the effectiveness of upper endoscopy utilization and frequency for patients with gastroesophageal reflux disease on morbidity, quality of life, and diagnosis of esophageal adenocarcinoma.)
National Priorities Partnership (e. g., Engage patients and families in managing their health and making decisions about their care.)
National Prevention Council (e. g., Elimination of health disparities)
FCCCER (e.g., Expanding high-impact patient registries)
National Quality Forum (e. g., Infrastructure – Information Technology)
Least Most
National Quality Strategy (e. g., Promoting effective communication and coordination of care)
28
Ten candidate priorities consistent with prior efforts
Source P
reve
nti
on
Acu
te C
are
Ch
ron
ic
Dis
eas
e C
are
Pal
liati
ve C
are
Car
e
Co
ord
inat
ion
Pat
ien
t En
gage
me
nt
Safe
ty
Ap
pro
pri
ate
U
se
HIT
to
Imp
rove
P
atie
nt
Exp
eri
en
ce
Imp
act
of
New
Te
chn
olo
gy
IOM 2009: Priorities for CER
√ √ √ √ √ √ √ √ √
Federal Coordinating Committee for CER
√ √ √ √ √
AHRQ National Quality Strategy
√ √ √ √
AHRQ Effective Health Care Program
√ √ √ √ √ √
National Quality Forum √ √ √ √ √ √ √ √
National Prevention Council
√ √
National Priorities Partnership
√ √ √ √ √ √
29
Proposed criteria include those in statute and additional criteria
Used in Prior Efforts
Criterion Required in
statute
√ Impact on Health √
√ Improvability via Research √
√ Inclusiveness of Subpopulations √
√ Impact on Health System Performance √
√ Gaps in Knowledge √
√ Variation or Disparities in Delivery or Outcomes √
√ Potential to Inform Decision-Making at Point of Care √
√ Responsiveness to Expressed Needs √
Advances CER Methods
Fits the Definition
30
•Impact on Health of Individuals and Populations
•Probability of Improvability via Research
•Inclusiveness of Different Sub-populations
•Current Gaps in Knowledge / Variation in Care
•Impact on Health System Performance
•Current Health Disparities
•Potential to Influence Decision-Making at Point of Care
•Responsiveness to Expressed Needs
•Advances CER Methods
•Fits the Definition of PCOR
Framework to Inspire Drafting of PCOR-Specific National Priorities
Health Information
to Improve Patient
Experience
Prevention &
Screening
Acute Care
Impact of New
Technology
PCORI National Priorities (Illustrative Examples)
Prevention & Screening
Develop information that will guide
patients and providers to improve
outcomes in chronic conditions.
Develop information to help patients
coordinate their care across multiple
providers and care settings.
Develop evidence to help patients
improve the safety of their health
care. Appropriate Use
PCORI Criteria
Proposed PCORI Criteria
Existing Priority Areas
In S
tatu
te
Chronic Disease
Care
Palliative Care &
Pain Management
Care Coordination
Patient
Engagement
Safety
31
Stakeholder Engagement
Guiding Principles
Balanced Representation
Transparency
Facilitate Participation
Enable an open engagement process that makes clear how
participants can get involved and how their input will be
incorporated
Obtain feedback from a diverse and representative range of
patients, health care stakeholders and the general public
Provide easy, accessible forums for participants to provide
feedback equitably
32
Stakeholder Engagement Overview
Stakeholder Engagement
Plan
Identify
Engage
Track
Assess
Report
33
PATIENTS PROFESSIONAL ORGANIZATIONS
Individuals with Disease Families
Caregivers Patient Advocacy Groups
General Public
Providers & Practitioners Employers
Payers Research Societies
Academia Health Information Exchanges
Federal/State/Local Government
Congress Life Sciences Industry
Who are PCORI’s stakeholders?
Key Stakeholder Groups
34
Proposed Stakeholder Engagement Timeline
Create
Engagement
Plan
and Identify
Stakeholders
Decide on
Content,
Methods
of Dialogue
Conduct Dialogue with Stakeholders
Review, incorporate and Report
on Feedback
September October November December Week
1 Week
2 Week
3 Week
4 Week
5 Week
6 Week
7 Week
8 Week
9 Week
10 Week
11 Week
12 Week
13 Week
14 Week
15 Week
16 * *
* Board Meetings *
35
Questions?
www.pcori.org
36
Richard Kuntz, MD
Presentation to PCORI Board of Governors
September 20, 2011
Landscape Review Program
37
Goals of Landscape Reviews
• Identify existing knowledge and gaps in existing knowledge related to PCOR and CER
• Avoid funding patient-centered outcomes research that duplicates ongoing or existing research
• Provide background information to spur innovation in research and methodology, and contribute to the infrastructure for conducting patient-centered outcomes research
• Create shared understanding about the state of the art of dissemination and patient engagement strategies and contribute to best practices
• Provide input into National Priorities and Research Agenda
38
Chronology of Landscape Review Development
Initial Phase: Environmental Scan of
Comparative Effectiveness Research
Discovery Phase: Learning about
Lewin Group Inventory of CER
Refining the Scope of Landscape Reviews &
Methodology Committee
Requirements
Coordination of Landscape
Reviews across PCORI: Landscape Review Program
Landscape Review Program:
• Process to initiate and manage landscape reviews
• Templates to develop contract specifications and budget requirements
• Resource library to share findings
39
Landscape Review Process Overview
Confirm questions to be answered
Document known information & gaps
Assess relevance / usefulness of information
Engage with NIH, AHRQ, others to leverage work of interest to PCORI
Identify additional information
needed
Prepare RFI as needed to acquire
information
PCORI RESOURCE LIBRARY
40
Landscape Reviews/Summaries Completed or Underway
Report Status
“Existing and Ongoing NIH Work Related to Dissemination of Research: Overview of NIH Briefing Book”
Completed 06/04/2011
“Existing and Ongoing Work Related to Patient Involvement in Peer Review: Overview of NIH Briefing Book”
Completed 06/04/2011
“Overview of Lewin CER Inventory Project” Completed 07/07/2011
“Organizations that Conduct CER” Completed 08/09/2011
Review and Synthesis of Evidence for Eliciting the Patient’s Perspective in Patient-Centered Outcome Research (Literature Review)
RFP issued 8/31/2011
Expert Stakeholder Interviews to Identify Evidence for Eliciting the Patient’s Perspective in Patient-Centered Outcome Research (Interviews)
RFP issued 8/31/2011
Landscape review of existing methodological standards Under development by MC
Landscape review of patient engagement Under consideration by COEC
41
Questions?
www.pcori.org