patient viewpoint: a website for patient-reported outcomes assessment claire snyder, phd albert wu,...

Patient Viewpoint:Patient Viewpoint:A Website for A Website for

Patient-Reported Outcomes Patient-Reported Outcomes AssessmentAssessment

Claire Snyder, PhDClaire Snyder, PhDAlbert Wu, MD, MPHAlbert Wu, MD, MPH

Johns Hopkins Johns Hopkins Schools of Medicine and Public HealthSchools of Medicine and Public Health

Presentation ObjectivesPresentation Objectives

To present the results from a literature review To present the results from a literature review and qualitative research study supporting the and qualitative research study supporting the development of a prototype website for patient-development of a prototype website for patient-reported outcomes assessment, including:reported outcomes assessment, including:– aspects of patient reported health that are useful to aspects of patient reported health that are useful to

assessassess– website features that patients and providers desire website features that patients and providers desire – barriers that need to be addressed barriers that need to be addressed

Prototype demonstrationPrototype demonstration

Background (1)Background (1)

2004 NIH Roadmap RFA: Meetings and 2004 NIH Roadmap RFA: Meetings and Networks for Methodological Development Networks for Methodological Development in Interdisciplinary Researchin Interdisciplinary Research– ““applications for meetings or networks to applications for meetings or networks to

develop methodologies that will facilitate develop methodologies that will facilitate interdisciplinary health research among interdisciplinary health research among behavioral or social scientists and behavioral or social scientists and investigators in the biomedical, investigators in the biomedical, mathematical/computational, physical mathematical/computational, physical sciences, and/or engineering”sciences, and/or engineering”

Background (2)Background (2)

Creation of the Creation of the Quality of Life in Clinical Quality of Life in Clinical PracticePractice Interest GroupInterest Group within the within the International Society for Quality of Life International Society for Quality of Life Research Research Goals of using quality of life in clinical Goals of using quality of life in clinical practice: practice: – To improve clinician-patient communicationTo improve clinician-patient communication– To facilitate individual patient managementTo facilitate individual patient management– To assist with medical decision-makingTo assist with medical decision-making

Research ProposalResearch Proposal

Creation of a network of Creation of a network of internationally known researchers internationally known researchers

and practitioners to develop a and practitioners to develop a centrally accessible website for centrally accessible website for

patient-reported outcomes patient-reported outcomes assessment assessment that links to the that links to the

electronic medical recordelectronic medical record

Goals of the GrantGoals of the GrantDetermine what aspects of patient reported health Determine what aspects of patient reported health would be most useful for evaluating the effectiveness would be most useful for evaluating the effectiveness and quality of outpatient oncology practiceand quality of outpatient oncology practice

Explore desirable features for a web-based system Explore desirable features for a web-based system to administer standardized questionnaires to patientsto administer standardized questionnaires to patients

Explore potential technical, cultural, ethical, and legal Explore potential technical, cultural, ethical, and legal barriers to the use of such a systembarriers to the use of such a system

Develop a prototype website for use by outpatient Develop a prototype website for use by outpatient oncology providers to collect patient-reported data in oncology providers to collect patient-reported data in the clinical settingthe clinical setting

Research Network MembersResearch Network MembersAlbert W. Wu, MD, MPH (Johns Hopkins University)Albert W. Wu, MD, MPH (Johns Hopkins University)

Neil Aaronson, PhD (Vrije Universiteit, the Netherlands)Neil Aaronson, PhD (Vrije Universiteit, the Netherlands)

Michael Brundage, MD, MSc (Queens University, Canada)Michael Brundage, MD, MSc (Queens University, Canada)

Carolyn Gotay, PhD (University of Hawaii)Carolyn Gotay, PhD (University of Hawaii)

James Hodge, JD (Johns Hopkins University)James Hodge, JD (Johns Hopkins University)

Denise Hynes, RN, MPH, PhD (Veterans Administration)Denise Hynes, RN, MPH, PhD (Veterans Administration)

Claire Snyder, MHS, PhD (Johns Hopkins University)Claire Snyder, MHS, PhD (Johns Hopkins University)

John Wasson, MD (Dartmouth University)John Wasson, MD (Dartmouth University)

Susan Yount, PhD (Evanston Northwestern Susan Yount, PhD (Evanston Northwestern Healthcare/Northwestern Univ.)Healthcare/Northwestern Univ.)

Brad Zebrack, PhD, MSW, MPH (University of Southern Brad Zebrack, PhD, MSW, MPH (University of Southern California)California)

Ably Assisted By:Ably Assisted By:

J.B. Jones, MBA, PhD(c)J.B. Jones, MBA, PhD(c)

S. Orion Courtin, MHSS. Orion Courtin, MHS

Roxanne JensenRoxanne Jensen

Web Development TeamWeb Development Team

MethodsMethods

Literature ReviewLiterature Review

First Research Network Meeting (Oct ’05)First Research Network Meeting (Oct ’05)

Develop Website SpecificationsDevelop Website Specifications

Second Research Network Meeting (Apr ’06)Second Research Network Meeting (Apr ’06)

Website DesignWebsite Design

Qualitative ResearchQualitative Research

Third Research Network Meeting (Feb ’07)Third Research Network Meeting (Feb ’07)

Website DevelopmentWebsite Development

Usability TestingUsability Testing

Literature ReviewLiterature Review

Rationale for web-based PRO assessmentRationale for web-based PRO assessment

Existing electronic and web-based Existing electronic and web-based platforms for collecting PROsplatforms for collecting PROs

Desirable features from clinician’s and Desirable features from clinician’s and patient’s viewpointpatient’s viewpoint

Potential ethical and legal issuesPotential ethical and legal issues

Jones et al. Quality of Life Research 2007;16:1407-1417

Advantages of Advantages of Web-Based PRO AssessmentWeb-Based PRO Assessment

Patient can complete the assessment anytime Patient can complete the assessment anytime from anyplace with Internet accessfrom anyplace with Internet access

Reminders and decision support can be built inReminders and decision support can be built in

Scoring is automaticScoring is automatic

Comparison scores immediately availableComparison scores immediately available

Data is easily accessible (privacy and security Data is easily accessible (privacy and security issues)issues)

Summary reports can be generated at the Summary reports can be generated at the patient level or practice levelpatient level or practice level


Desirable FeaturesDesirable Features

Efficiency and Ease of UseEfficiency and Ease of Use– Will the website make efficiency better or worse?Will the website make efficiency better or worse?– Will the website fit into clinical workflow?Will the website fit into clinical workflow?– Will it add to my workload?Will it add to my workload?– Will patients have an easy time using it?Will patients have an easy time using it?

Results Presentation and InterpretationResults Presentation and Interpretation– What do the results mean?What do the results mean?– What should the clinician/patient do about them?What should the clinician/patient do about them?


Ethical and Legal IssuesEthical and Legal Issues

Privacy Rule:Privacy Rule:– Protected health information can only be disclosed Protected health information can only be disclosed

with written authorization unless certain criteria for with written authorization unless certain criteria for exemption are metexemption are met

– Exemptions include public health research, minimal Exemptions include public health research, minimal risk research that cannot be practicably done without risk research that cannot be practicably done without a waivera waiver

Security Rule:Security Rule:– Defines the technical, administrative, and physical Defines the technical, administrative, and physical

safeguards that must be put in place to protect the safeguards that must be put in place to protect the confidentiality, integrity, and availability of protected confidentiality, integrity, and availability of protected health informationhealth information


Patient ViewpointPatient ViewpointA web-based system with a database back end for A web-based system with a database back end for management and storage of patient responses management and storage of patient responses – Developed using Microsoft ASP.Net and SQL databaseDeveloped using Microsoft ASP.Net and SQL database

Linked to the electronic medical recordLinked to the electronic medical recordThree main functions provided by the system:Three main functions provided by the system:– Clinicians select questionnaires to administer to patients and Clinicians select questionnaires to administer to patients and

how often how often – Patients enter responses, submit comments, and view the Patients enter responses, submit comments, and view the

results of their responseresults of their response– Clinicians have a text and graphical view of patients’ responses Clinicians have a text and graphical view of patients’ responses

and scores over time and scores over time

SecuritySecurity– Database and programming code located on separate serversDatabase and programming code located on separate servers– Servers backed up regularly and backups stored at a remote Servers backed up regularly and backups stored at a remote

locationlocation– Secure Sockets Layer secure the site as patients enter their Secure Sockets Layer secure the site as patients enter their

responses and physicians view the resultsresponses and physicians view the results– Access to the system is controlled by requiring username and Access to the system is controlled by requiring username and

passwordpassword

Qualitative ResearchQualitative Research

Interviews Interviews – 20 Cancer Patients20 Cancer Patients– 3 Oncologists3 Oncologists– 4 Research Nurses4 Research Nurses

Semi-structured in-depth interviews– Followed structure of the prototype website– Probed for opinions, possible improvements,

and innovations

Findings-Patients (1)Findings-Patients (1)

Keep surveys short and simple“I always have questions for him about my well-being, but if it becomes too long or too complicated, I’ll give up.”

Response: Worked with physicians to choose a concise package of measures to lower respondent burden


Present multiple questions per survey page, but avoid long scrolling““I would [prefer multiple questions], as long as there’s not too much.”

Response: Will present questions with same response type together on a page


Some patients are interested in their survey results, others just want to comply with the request of their care provider“Somebody is going to find it interesting, but I’m not, you know personally, once I’ve done the survey and made my additional feedback, I’m basically done.”

Response: Provide simple feedback for patients who are interested


Patients interested more in changes over time than survey values“It will be more meaningful just to see the individual change. I want to be able to look back six months and say, where was I six months ago.”

Response: Will generate simple graphs showing change over time


Patients suggest making direction of improvement consistent across types of measures“I’d want them to run in the same direction, with high being bad and low being good. That’s like these tests they give us, and you need to be consistent.”

Response: Explore options for making results easy to interpret while keeping in mind issues with violating conventions


Patients want direct communication, want their comments to inform the next visit, worried it will not be read“You need to go to your doctor with questions in writing, and it seems that this would be a vehicle to get those questions there.”

“I am concerned where you say that you won’t read it until the next appointment,…, what am I chopped liver or something?”

Response: Will keep the free text response and integrate it into the clinician results


Patients perceive the site’s purpose as valuable, concerned it will become burdensome“It would be useful from their point of view, so I’d be willing to take five minutes to fill out the thing.”

Response: Will clearly state the purpose of the site, keep surveys short and continue to improve usability

Findings-Clinicians (1)Findings-Clinicians (1)

Clinicians need guidance for interpreting the meaning of values“Of course I have no idea if this is a good score or a bad score.”

Response: Will present results on commonly understood dimensions for their patients on standard scales


Clinicians are most interested in meaningful individual change over time“The individual patient values would be more useful, I want to know about this individual patient, some patients will always score low, this doesn’t represent change over time, you can’t always make a miserable person better, is there something that I can fix?”Response: Focusing results presentation on individual changes over time, exploring ways to determine what constitutes a significant change for each measure


Need ways to emphasize the key findings to focus on during the visit“We’re all taught to fix the one thing that the patient needs today, is there one parameter that is tipping the scale, or are they generally having trouble.”

Response: Added a question after each survey asking what a patient would most like their care provider to address at their next visit


The website could be useful in clinical practice, but it needs to be easy to integrate into current routines with little administrative burden“I would want someone to print the information out to be reviewed with the chart, or linked directly in the EPR. I need something to hit me over the head to indicate that there’s something I need to look at.”

Response: Adding a batch print feature to print results for patients based on schedulingResponse: Working with Electronic Patient Record personnel to integrate with their system


Initially we envisioned clinicians choosing from a menu of surveys for each patient, but clinicians were concerned about administrative burden and lack of familiarity with different surveys

Response: We conducted research to narrow the choice of measures, then met with physicians to choose the ones most appropriate to their practice

Summary of Qualitative Summary of Qualitative Study FindingsStudy Findings

Respondents optimistic about potential value but concerned about practicality and integration into current practice– Use short surveys and a straightforward interface for

patients– Integrate results into the current clinical routine

through the Electronic Patient Record or print for the paper chart

In results presentation, both patients and clinicians were most interested in individual changes over time

Demonstration of Patient WebsiteDemonstration of Patient Website

Patient visits site and logs inPatient visits site and logs in


Patient receives welcome screenPatient receives welcome screen


Patient is presented with disclaimerPatient is presented with disclaimer


Patient is presented with survey(s) to Patient is presented with survey(s) to completecomplete


Patient survey – sample questionPatient survey – sample question


Patient gets a last chance to answer Patient gets a last chance to answer skipped questionsskipped questions


Patient completes survey and can ask Patient completes survey and can ask their Doctor a question or view resultstheir Doctor a question or view results


Patient views resultsPatient views results

Demonstration of Clinician WebsiteDemonstration of Clinician Website

Clinician logs in and can search for patientClinician logs in and can search for patient


Clinician can search for patient by surveyClinician can search for patient by survey


Clinician can view patients resultsClinician can view patients results


Clinician can search for another clinicianClinician can search for another clinician


Clinician can add or edit a patientClinician can add or edit a patient


Clinician can schedule a patient a surveyClinician can schedule a patient a survey


Clinician can add/edit another care team Clinician can add/edit another care team clinicianclinician

Next StepsNext Steps

Usability testing is ongoingUsability testing is ongoing

R21 submitted to conduct initial pilot testR21 submitted to conduct initial pilot test– UseUse– UsefulnessUsefulness– AcceptabilityAcceptability

Questions?Questions?

patient viewpoint: a website for patient-reported outcomes assessment claire snyder, phd albert wu,...

Documents

research network meeting

quality of life research

prototype website

patient viewpoint

aspects of patient

public health slide

clinical setting slide

prototype demonstration