does a smartphone app help patients with cancer …...that interventions delivered via mobile...

1

Does a Smartphone App Help Patients with Cancer Take Oral Chemotherapy as Planned?

Joseph A. Greer, PhD; Jamie Jacobs, PhD; and Molly Ream, BA

Massachusetts General Hospital Cancer Center and Harvard Medical School, Boston,

Massachusettss

Original Project Title: Mobile Application for Improving Symptoms and Adherence to Oral

Chemotherapy in Patients with Cancer

PCORI ID: IHS‐1306‐0316

HSRProj ID: 20143571

ClinicalTrials.gov ID: NCT02157519

_______________________________

To cite this document, please use: Greer J, Jacobs J, Ream M.(2019). Does a Smartphone App Help

Patients with Cancer Take Oral Chemotherapy as Planned? Washington, DC: Patient‐Centered

Outcomes Research Institute (PCORI).

https://doi.org/10.25302/4.2019.IHS.130603616

2

Table of Contents

ABSTRACT .......................................................................................................................................... 3 BACKGROUND .................................................................................................................................... 5 PARTICIPATION OF PATIENTS AND OTHER STAKEHOLDERS IN THE DESIGN AND CONDUCT OF RESEARCH AND DISSEMINATION OF FINDINGS .................................................................................. 9

Phase 1 ............................................................................................................................................................. 12

Phase 2 ............................................................................................................................................................. 22

METHODS ......................................................................................................................................... 23

Phase 1 ............................................................................................................................................................. 23

Forming the study cohort ................................................................................................................................. 23

Phase 2 ............................................................................................................................................................. 26

Study outcomes: primary outcome measures .................................................................................................. 28

Study outcomes: secondary outcome measures .............................................................................................. 29

Potential moderators: measures for exploratory analyses ............................................................................... 30

Aim 1: To implement a patient‐centered mobile app to assess symptoms, side effects, and adherence to oral

chemotherapy that is feasible for use with oncology patients ......................................................................... 35

Aim 2: To evaluate the efficacy of the mobile application

in improving adherence and patient‐reported clinical outcomes .................................................................... 35


in improving quality of oncology care. ............................................................................................................. 35

Exploratory Aim: To determine whether particular patient demographic and clinical characteristics

moderate the effect of the study intervention .. .............................................................................................. 35

RESULT ............................................................................................................................................. 38

Phase 1 .............................................................................................................................................................. 38

Phase 2 ............................................................................................................................................................. 40

Aim 1: To implement a patient‐centered mobile app to assess symptoms, side effects, and adherence to oral

chemotherapy that is feasible for use with oncology patients. ........................................................................ 44


in improving adherence and patient‐reported clinical outcomes ................................................................... 45


in improving quality of oncology care. ............................................................................................................. 45

Exploratory Aim: To determine whether particular patient

demographic and clinical characteristics moderate the effect of the study intervention. ............................... 48

DISCUSSION ......................................................................................................................... 68 The study results in context .............................................................................................................................. 69

Implementation of study results ....................................................................................................................... 71

Generalizability ................................................................................................................................................. 73

Subpopulation Considerations .......................................................................................................................... 73

Study limitations ............................................................................................................................................... 74 Future research ................................................................................................................................................. 74

CONCLUSION ........................................................................................................................ 75

REFERENCES ......................................................................................................................... 77

3

PUBLICATION LIST ................................................................................................................ 82

APPENDIX ............................................................................................................................ 83

B. ABSTRACT

Background: Patients prescribed oral chemotherapy receive less support for adherence and

monitoring of symptoms from oncology clinicians than do patients prescribed traditional

infusion chemotherapy, resulting in poor adherence, lower‐quality care, and worse disease

outcomes. No theory‐based, efficacious interventions exist to promote adherence and

symptom monitoring for patients prescribed oral chemotherapy.

Objectives: The primary aims of this study were to (1) develop a patient‐centered, smartphone

mobile application (app) to facilitate adherence to oral chemotherapy and symptom

management for patients with cancer; and (2) test the effect of the app on improving

adherence to oral chemotherapy, symptoms, quality of life (QOL), and quality of care in a

randomized controlled trial (RCT).

Methods: A multidisciplinary research team worked with key stakeholders to develop the

mobile app, soliciting feedback on app content, usability, and patient‐centeredness from 4

groups: patients/families (n = 8); oncology clinicians (n = 8); cancer practice administrators (n =

8); and representatives from the health system, community, and society (n = 8), as well as

patients (n = 10) and oncology clinicians (n = 8) from the Massachusetts General Hospital.

Then, from February 18, 2015, to October 31, 2016, 181 patients with diverse malignancies

prescribed oral chemotherapy enrolled in an RCT to receive the mobile app intervention or

standard oncology care. The primary outcomes were adherence and self‐reported symptoms

and QOL. Adherence was measured by the Medication Event Monitoring System Cap

(MEMSCap) and by self‐report. The secondary outcomes were patient perceptions of quality of

care and utilization (ie, hospitalizations and emergency department visits). Patients completed

the self‐report questionnaires at baseline prior to randomization and at 12 weeks.

4

Results: Feedback from stakeholders and patient participants greatly informed intervention

development and showed that the app was perceived as useful and acceptable. The final app

incorporated features including a treatment plan, reminder system, symptom reporting

modules, and patient resources. Patient‐reported data were transmitted to the oncology team

via HIPAA‐compliant email on a weekly basis. The mobile app intervention group and control

group did not differ over time with respect to the primary outcomes of adherence, self‐

reported symptoms, and overall QOL, or in the secondary outcomes of quality of care and

utilization. In examining specific domains of QOL, patients in the mobile app group had a

smaller reduction in social well‐being over time (Mdiff = 1.67; SE = 0.74; F1161 = 5.13; p = .025;

95% CI, –3.12 to –0.21). Subgroup analyses showed that patients with poor self‐reported

adherence and high anxiety at baseline who were randomized to the app had improved

MEMSCaps adherence rates compared with the standard care group. Finally, older patients

randomized to the app reported improved QOL compared with those receiving standard care.

Conclusions: Feedback from stakeholders and patient partners was instrumental in optimizing

relevancy, feasibility, and acceptability of the study methods and app intervention. Across all

patients, the mobile app was not efficacious in improving adherence or symptoms. However,

patients at greater risk for nonadherence may benefit.

Limitations: Use of daily MEMSCap as the primary study outcome may have raised participant

awareness of adherence across both study groups, perhaps diminishing intervention effects.

Additionally, generalizability of study findings is limited due to the restricted diversity of this

well‐educated sample at an academic institution.

5

C. BACKGROUND

Cancer care delivery has shifted in the past decade, with a substantial increase in the

prescription of oral cancer therapies as an alternative to traditional intravenous

chemotherapy. In 2010, approximately 16% of patients receiving cancer treatment were

prescribed oral agents, and this figure is expected to surpass 25% in coming years, given

advances in the study of tumor genetics and the number of oral chemotherapy agents in

current development.1 Patients overwhelmingly prefer oral administration to intravenous due

to the enhanced convenience of home administration, the mitigation of problems related to

intravenous access such as pain or discomfort, and an increased sense of control of the

chemotherapy environment.2 In fact, patients prescribed oral chemotherapies report less

interference in their daily activities, corresponding to better quality of life (QOL).2,3

Patients and oncology clinicians have encountered unique challenges as cancer care

becomes increasingly delivered in the outpatient and home setting.4,5 While patients

prescribed traditional intravenous chemotherapy receive direct supervision in infusion centers,

where they are monitored and treated for symptoms and side effects, individuals prescribed

oral chemotherapy take their medications at home with limited oversight, monitoring, and

support from their oncology clinicians.6,7 The toxicities of oral chemotherapy are equivalent to

those of intravenous chemotherapy, including nausea, vomiting, fatigue, and diarrhea,8 yet the

lack of regular contact with the oncology team is a barrier to proper use of this regimens.5,9 For

example, symptoms such as difficulty swallowing, nausea, and vomiting may interfere with

taking oral agents if not treated appropriately. Patients and their families often must assess

and manage symptoms on their own and, in turn, may not adhere to the treatment regimen as

intended.

Patient adherence is often defined as taking a medication as prescribed regarding daily

amount, dosage, and frequency; it is vital to the efficacy of an oral chemotherapy regimen.10

Importantly, poor adherence to oral anticancer treatment is associated with poor survival

rates and with disease progression.10‐14 Despite the importance of adherence for optimal

cancer outcomes, several systematic literature reviews have shown that adherence rates to

6

oral chemotherapy in patients with cancer vary widely, with adherence ranging from as high as

100% to as low as 16%.15‐18 These estimates vary based on patient sample, medication type,

follow‐up period, assessment measure, and calculation of adherence. Various patient,

provider, treatment‐related, and health care system factors are associated with treatment

adherence,19 including patient health beliefs regarding treatment efficacy, cognitive

impairments, inadequate social support, psychological distress, poor communication with

providers, adverse effects of treatment, and difficulty accessing care or costs of

medications.15,16,19‐23 More specifically, studies have shown that patients who are male, older,

living alone,24 nonwhite,25 of low socioeconomic status,26 treated in community versus

academically based centers,26 or depressed are more likely to be nonadherent.27 In addition,

presence of severe side effects,28 greater complexity of cancer treatment (eg, variable dosing

schedules), and greater length of time on treatment are associated with poor adherence to

oral chemotherapy.24,29,30 Other potential barriers to oral chemotherapy adherence may

include patient forgetfulness, misunderstanding of dosing instructions, and attitudes toward

the effectiveness of the chemotherapy.31,32 In addition, patients with elevated distress are

more likely to struggle with adherence.33,34 In our own longitudinal investigation of patients

receiving chemotherapy for advanced non–small cell lung cancer, approximately 30% had

heightened baseline anxiety symptoms, which significantly predicted the occurrence of

chemotherapy dose delays and reductions.35 Given that 10% to 25% of patients receiving

cancer treatment become clinically depressed,36,37 many patients on oral chemotherapy will

experience psychological distress that could interfere with adherence.

There is a critical need to overcome the challenges associated with the fragmentation

of care related to oral chemotherapy administration, with specific attention to medication

adherence and symptom management.38 Recently updated standards from the American

Society of Clinical Oncology (ASCO) and Oncology Nursing Society now include comprehensive

guidelines for prescribing, documenting, and monitoring patient treatment with

chemotherapy, including oral agents.1 These standards include recommendations for

discussing and documenting a chemotherapy treatment plan based on the type of medication,

dosage, anticipated duration of treatment, and goals of therapy. Furthermore, the ASCO

7

Quality Oncology Practice Initiative has been examining quality metrics for oral chemotherapy

administration pertaining to documentation of treatment plan, patient consent and education,

and ongoing monitoring of oral agents.39 Despite these guidelines, very few interventions to

improve adherence and monitoring for patients prescribed oral chemotherapies have been

tested. In a recently published systematic review, we identified only 12 adherence

interventions for patients with cancer, with some resulting in mixed findings40‐42 and most

lacking methodological rigor, with nonrandomized designs and small sample sizes.18 Thus,

theory‐based interventions that are accessible to patients in order to promote adherence and

symptom management are critically needed.

Mobile health (mHealth) technology provides an opportunity for support and

monitoring in a minimally burdensome, maximally accessible approach.43 Evidence suggests

that interventions delivered via mobile technologies can improve health behaviors in patients

with cancer.44 In addition, mobile smartphones allow for ecological momentary assessments

by facilitating repeated evaluation of participants’ symptoms and adherence behaviors in real

time, which may enhance the provision of care for patients prescribed oral chemotherapies.

Smartphone mobile applications (apps) may be an ideal platform to administer a supportive

intervention that promotes adherence and symptom management for patients prescribed oral

chemotherapy. Thus, with support from the Patient‐centered Outcomes Research Institute, we

conducted a 2‐phase study to develop a patient‐centered mobile app to assess symptoms, side

effects, and adherence to oral chemotherapy that is feasible and efficacious for use with

oncology patients.

In phase 1, we developed an acceptable and feasible patient‐centered mobile app

informed by qualitative feedback from key stakeholders, patients, and oncology clinicians. In

phase 2, we conducted a randomized controlled trial (RCT) to demonstrate feasibility and

evaluate the efficacy of the mobile app in improving adherence as well as patient‐reported

clinical outcomes. Aim 1 of phase 2 was to test feasibility based on rates of completion of

symptom reports in the mobile app. We hypothesized that at least 75% of participants

assigned to the mobile app intervention would complete symptom surveys for at least 9 of the

12 study weeks. With respect to evaluating the efficacy of the mobile app in improving primary

8

outcomes (aim 2), we hypothesized that patients prescribed oral chemotherapy for cancer

who were randomly assigned to the mobile app intervention would report better medication

adherence, fewer symptoms and side effects, and improved quality of life compared with the

control group (ie, patients receiving standard care). The third aim of phase 2 was to evaluate

the efficacy of the mobile application in improving quality of oncology care. We hypothesized

that patients who were randomly assigned to use the mobile application would report greater

satisfaction with medical care and have fewer emergency department visits and

hospitalizations compared with the control group. Finally, we explored treatment

heterogeneity by examining whether particular patient demographic and clinical

characteristics (eg, cancer type, age, gender, baseline self‐reported adherence) moderated the

effect of the study intervention, thereby identifying any key subgroups of participants who

may have responded differently to the mobile application.

9

D. PARTICIPATION OF PATIENTS AND OTHER STAKEHOLDERS IN THE DESIGN AND CONDUCT OF RESEARCH AND DISSEMINATION OF FINDINGS

In accordance with PCORI Methodology Standard PC‐1, we engaged individuals

representing the population of interest (ie, patients with cancer, their family members,

clinicians, administrators, and policymakers) in formulating research questions; defining

characteristics of the intervention, study design, and outcomes; monitoring study progress;

and developing plans for dissemination and implementation. To include a representative,

diverse, and comprehensive group of stakeholders,45 we identified 4 core stakeholder groups

(Figure 1) by drawing from a population‐based model for patient‐centered care from the

Medical College of Wisconsin.46 We selected stakeholders from across the United States (13

states), thus reaching outside our local academic medical community. We identified patients

and family members from the Massachusetts General Hospital (MGH) Cancer Center Patient

and Family Advisory Council. To be eligible, the stakeholder must have been able to represent

the interests and perspectives of at least 1 of the 4 groups. Members of the investigative team

(Drs. Greer, Temel, Pirl, Safren, Lennes, Jethwani, and Buzaglo) organized a list of stakeholders

from these 4 cancer community groups. We contacted stakeholders to explain their

involvement and study procedures. Thirty‐two stakeholders assisted with the study,

representing the following 4 key stakeholder groups: (1) oncology patients and family

members (n = 8); (2) oncology clinicians (n = 8); (3) cancer practice setting administrators (n =

8); and (4) representatives of the health system, community, and society (n = 8). Stakeholders

were involved in the study as research collaborators/consultants and were remunerated up to

$1 000 for their time and effort. These stakeholders were involved in both phase 1 and phase 2

of the study. In addition to the stakeholder groups, we enrolled 10 MGH patients prescribed

oral chemotherapy and 8 MGH oncology clinicians as participants during phase 1 of the study

to review the mobile app wireframes (ie, screen blueprints) and provide feedback. These

patients and clinicians were considered study participants, and they each signed IRB‐approved,

HIPAA‐compliant consent forms prior to participation. Relevant characteristics of these

participants are presented in Table 1. The specific involvement of the stakeholders, as well as

the patient and clinician participants, in each study phase is detailed below.

10

Figure 1. Stakeholder Groups and Engagement

FO

CU

S G

RO

UP

S

Patients/ family

Oncology clinicians

Cancer practice settings

Mobile app development

FO

CU

S G

RO

UP

S B

IAN

NU

AL U

PD

AT

ES

Complete analysesprepare dissemination

Trial and data collection

Preliminary analyses

Patients/ family Oncology

clinicians Cancer practice settings

Health system,

community, and society

Health system,

community, and society

11

Table 1. Phase 1: Characteristics of MGH Patients and Oncology Clinicians MGH Patient Characteristic (n = 10) M (SD) or N (%)

Age 58.40 (8.02)

Gender Women 6 (60%) Men 4 (40%) Race/ethnicity White 8 (80%) Asian 1 (10%) Hispanic or Latino/a 1 (10%) Education College 6 (60%) High school graduate/GED 2 (20%) Unknown 2 (20%) Marital status Married 7 (70%) Single 3 (30%) Cancer type Non–small cell lung cancer 5 (50%) Breast cancer 2 (20%) Prostate cancer 1 (10%) Chronic myeloid leukemia 1 (10%) Multiple myeloma 1 (10%)

MGH Oncology Clinician Characteristic (n = 8) N (%)

Gender Women 6 (75%) Men 2 (25%) Clinician type Physician 4 (50%) Nurse practitioner 4 (50%) Area of expertise Genitourinary oncology 3 (37.5%) Breast oncology 2 (25%) Thoracic oncology 2 (25%) Melanoma 1 (12.5%)

12

Phase 1

First, we conducted a pretrial planning interview with the first 4 stakeholder groups to

solicit feedback about the proposed study topic, design, and intervention to ensure relevancy,

acceptability, and the potential for dissemination. The patient/family interview took place in

person at the MGH Cancer Center, and the 3 other group interviews occurred as

teleconference calls. Specifically, we addressed the following topics: (1) perceived importance

of monitoring oral chemotherapy remotely; (2) barriers to communication between patients

and the oncology team regarding management of side effects and medication adherence; (3)

the potential role of the mobile app to address barriers to quality of cancer care; (4) the

potential feasibility, acceptability, and usability of an mHealth intervention; and (5) system

barriers and facilitators to implementation. We identified consistent themes about the

planned intervention and study design from these interviews. Feedback from this stage was

integral in informing the development of the mHealth intervention. For example, stakeholders

recommended a symptom monitoring feature with interpretable graphics, emphasized the

importance of distinguishing urgent versus nonurgent symptoms within the symptom

reporting module, provided guidance on optimizing patient–physician communication while

minimizing burden, and suggested methods for promoting participant engagement with

incentivizing app features. We incorporated each of these recommendations into the final

mobile app version. The interview guides for these focus groups are presented in Appendix A

and a summary of feedback is presented in Table 2.

13

Table 2. Phase 1 and Phase 2 Feedback From 4 Stakeholder Focus Groups

Phase 1 (Pretrial Focus Groups) Mobile App Development and Design

Phase 2 (Posttrial Focus Groups) Ideas for Analyses, Future Iterations, and Dissemination

Patients/family members

Weekly symptom checker:

Consider using a graph as a display for symptom progress over time. Patients often have a good day followed by a series of bad days, so it can be hard to remember the good days—this will help patients visualize their progress and motivate them.

Identify the immediate symptoms and resolutions, and the long‐term symptoms and resolutions.

Make patients aware of the symptom reporting process and what will be reported to the physician:

Give patients an option to rate a symptom as “bad” but manageable without needing a call from the physician.

Make sure that patients are not afraid to report symptoms.

Have patient bring her or his phone to clinic with app to show any changes/progress to the physician.

Educational resources:

Make sure that websites are user‐friendly, patient‐specific, and reputable.

Some websites and online support groups include patients that are very sick, which can provoke a lot of anxiety for many patients who have fewer symptoms.

Patient–physician communication: • Ensure that physicians using the application will act on

symptom reports and feedback regarding patient’s adherence and symptom management.

• Patients would appreciate getting notification that the physician has viewed their message (eg, read receipt).

Other possible variables we should analyze in this or future iterations of this study:

Provider–provider communication (specifically coordination between the oncologist and the primary care physician)

Mental health services and palliative/supportive medication (ie, antinausea drugs)

Distance from patient’s residence to where patient is receiving care

Distance from patient’s residence to residence of caregivers/support system

Proxy use by caregivers (“did/how often did your loved one help you use the app?”)

Tech literacy or comfort with technology

“Are you confident in your own efficacy of treatment?” “How confident are you with your own ability to adhere to your treatment plan?”

Perhaps look at subgroup of hematologic malignancies only.

Possible features for future iterations of the app:

Provide a line graph view over entire months/length of app use displaying symptom trends.

Option to give app access to loved ones/caregivers as well, so that they can see how patient symptoms/adherence has been, or help the patient use the app

Track all meds the patients is taking, huge investment to make something broadly applicable.

Weekly symptom reports to go directly into medical records rather than be sent via email

14

• Provide patients with an option to report high‐severity symptoms, but mark as manageable, and not in need of a call from their doctor.

Usability, feasibility, acceptability:

If patients are very sick, then they might not report symptoms accurately—some might report more modestly or not at all.

Find a way for the app to screen different symptoms (eg, loss of appetite, depression) and triage the information to a relevant clinician (nutritionist, social worker, psychiatrist, etc).

• Patients who are easily distracted or bored might not use the application frequently if it is not interactive enough.

• It is essential that doctors positively reinforce patients in clinic by bringing up their app‐related symptom/adherence reports.

• Conduct a brief feasibility study with patients before the RCT and meet with them a few weeks in to see if they are using the app correctly or having any difficulties.

Organizations to communicate findings with:

Healthcare for All (umbrella organization for Patient and Family Advisory Councils): host webinars every 2 to 3 weeks; also may do newsletter

Facing Cancer Together

Facebook

ASCO

Drug Information Association

Partners‐affiliated hospitals

Oncology clinicians

Personalized oral chemotherapy plan:

Consider using Adhere Tech bottles (which

have a technology that counts pills).

Make sure the plan is editable.

Have a feature that allows patients to opt out/edit/utilize/personalize medication

reminders.

Tailor personalized plan: ability to reduce frequency of reminders if patient is stable,

Can examine any age differences? Were younger people more likely to use the app?

Were there any reports or questionnaires on the clinician side?

Were patients with more symptoms at baseline more likely to benefit from the app?

Was the amount of time patients have been on oral therapy related to how well they are managing their symptoms, and how much an app may benefit them? The first 1 to 2 months tend to be the most challenging and include the most side effects.

15

AND ability to increase frequency of reminders if patient is more symptomatic.


Drive educational content based on common side effects of the patient’s specific drug/disease.

Collaborations with a pharmaceutical company could help with dissemination of information and sustainability.

Would be helpful to have a feature that organizes medical information (appointments/scan schedule).

Patient–physician communication: • Be mindful of when messages/reports are delivered to physicians so that we are engaging them and not annoying them. • Make sure that patients do not view the app as the only way to communicate with their medical team.


Print on a smartphone is small and might be hard to read—check with Connected Health to see if having the app on an iPad is feasible.

Consider customizing app for specific regimens.

Create subgroups/cohorts that might benefit more than others (very beneficial for data collection and seeing effects based on disease/medication/age group).

Think of a way to make app appealing to patients (move away from an app that serves as an at‐home reminder that patient has cancer): Utilize wellness as a driving force,

How was the complexity of the medication regimen? Could examine how many other medications patients were taking.

Did you look at whether patient had a caretaker or someone else who is looking out for him or her?

Information should automatically upload into EPIC so that clinicians could easily see adherence and symptom reports right before meeting with a patient in clinic.

Next steps: Wouldn’t want to get notified every time a pill is missed, but would want to be notified for things that are clinically relevant (eg, if patient missed taking medication for a whole week).

Allow for customization for each clinician: What is the threshold each physician wants for different patients or groups of patients?

There is a worry of extra burden for staff if they are getting too much information that can wait. Maybe extra step question for patient is do you want this to send immediately or would you like to log and bring it up at next appointment.

16

although wellness could be too broad so try to hone in on specifics.

• Consider the liability of symptom reporting, especially for symptoms like fever or neutropenia.

Health care representatives


This section is important and distinguishes

itself from other general resources because it

is customized and tailored to the specific

patient.

Identify/compare patients to others in their same cohort (disease)—by motivation, support system.


It is important to strike a balance so it is user friendly, yet not burdensome.

Provide anchors and operational definitions for symptom severity and frequency (eg, constipation: definition, how often, how severe, when to contact doctor).

One goal should be to empower patients to report symptoms through app and hopefully speak up more during their clinic visits.


Provide patients with education on lab results and how to access them (on the app, possibly) and interpret results.

Patient–physician communication: • Foster communication between patient and health care team (not just physician ‐ ie NP). • Provide patients with resources on how to talk to their physician during clinic visits regarding symptoms.

Is it possible that people who could engage with the app were already better off in terms of adherence?

If the patient uses the app, but the clinician does not read the symptom report email, that could affect the experience and satisfaction. Is there a way to look at this?

How might disease progression have affected results (very heterogeneous group with heme and solid malignancies and staging)?

Is it possible to examine any data on financial distress or cost of medications in relation to adherence?

Next steps: Get impressions of oncologists who were involved in the study (was the email format for symptom reports useful; what kind of patients would they want to have access to this app?).

Potential avenues for dissemination: Cancer Support Community Newsletter; Oncology Nursing Society; Medscape; Apple, Google, and social networking technological spheres; joint effort between number of oncology organizations to host a webinar

17

• Understand patient satisfaction on an ongoing basis and satisfaction with communication with care team. • Capture patient–physician communication as an outcome.


Sustainability: Consider if patients in RCT will use app after the trial is over.

If they will discontinue app, make sure that transition is done in a way that will not create a void.

Design program so that it is rewarding for patients to use.

It might be helpful to collect baseline data on patient beliefs and expectations regarding oral chemotherapy meds, as well as coping styles.

Education and training for the mobile app: ‐Research assistants, study staff ‐Training video as part of app

• Identification of patients/recruitment: ‐Query electronic health records THEN approach physician. • Engage clinicians throughout app development and RCT so that they will be interested and compliant with patient–physician communication—it is crucial that physicians bring up reported symptoms at clinic visit or app could be a total flop. •Feedback from patients ‐Access to medication and having enough meds throughout the study

Practice administrators


Make sure to stratify by line of therapy if variation exists (ie, oral chemo as first line/first line of oral chemo, but not first line of treatment/second line of oral, or above).

Is there any way to capture cost? Prevent hospital admissions?

Think of the app as a motivational tool. Some will like and benefit from more connectivity with their

18

Ability to track patient hits on oral chemo plan

Info for patients on how to proceed when they miss a medication dose

Feature that allows patients to log in when they have renewed their prescription


If patient decides to report symptoms more than once a week, how will that be reported to the team?

Give patients an option with new/worsening symptoms to have someone call them when they report a symptom; don’t rely only on hard stops.

Method of tracking phone calls regarding symptoms and compare between arms

Once a week is not burdensome for physician, but every day would be.

Ability to triage new/worsening symptoms for communication purposes


Ability to track patient hits on educational resources

Patient–physician communication:

Consider asking patients if they feel like their symptom reports are being heard and addressed by team.

Have a point person for patient emails aside from MD (ie, NPs).

Ability to see who is viewing the patient emails

Look at past info on how physicians act on new/worsening symptoms


providers, but others may not. Would like to think of other engagement features.

It would be interesting to see whether patients who used the app for certain reasons (ie, symptom reporting versus medication reminders) were more likely to benefit.

Could examine specific symptoms on the MDASI to see if app had any benefit?

It is import to capture utilization as an outcome to generate interest with payers.

Potential for pharmacy involvement as a next step; the app might be more helpful for the pharmacy team than the oncology clinicians

Dissemination of findings to pharmacy and nursing groups: Hematology/Oncology Pharmacy Association and American Society of Health Systems Pharmacists; Oncology Nursing Society; Sigma Theta Tau

19

Comfort level with mobile app may be different for elderly population—good thing to assess in the beginning.

Provide patients with contact info for app troubleshooting.

• Emphasize that patients can still call their doctor if they are having new/worsening symptoms so that we do not hinder their willingness to speak up.

20

Next, we met individually with the 10 MGH patient and 8 oncology clinician

participants to review the app content using wireframes created by the research and design

teams (see Figure 2). Using semistructured interview guides (see Appendix B), we solicited

feedback in 3 domains: (1) components of the mobile app, (2) feasibility and usability of the

app, and (3) weekly in‐app symptom assessments. We directed integrated feedback from this

stage regarding the aesthetics, frequency of push notifications, and incorporation of the

patient’s treatment plan into the mobile app design (see Appendix C).

Finally, after developing the beta version of the app, we invited members of the initial

patient and family stakeholder group (n = 8) to participate in user acceptance testing. Research

and development staff observed stakeholders during their initial interactions with the mobile

app and asked them to complete specific tasks (eg, “How do you think you would go about

adding your oral chemotherapy medication into this app?”). Stakeholders were asked to share

general and specific feedback about task intuitiveness. We further refined the app based on

their responses. In summary, feedback from key stakeholder groups as well as patient and

clinician participants in phase 1 had a significant impact regarding maximizing the patient

experience, optimizing patient–clinician communication within the app, and refining study

procedures for phase 2.

21

Figure 2. Wireframes (Screen Blueprints) for CORA Mobile App

22

Phase 2

The 4 key stakeholder groups from phase 1 were also involved as research

collaborators/consultants for phase 2 of the study, during which we tested the efficacy of the

mobile app intervention in a randomized controlled trial. We maintained consistent

communication with stakeholders throughout the RCT in the form of surveys, quarterly

newsletters, a midstudy luncheon, and a final presentation and focus group. At the initiation of

the RCT, we emailed stakeholders a survey to collect feedback regarding participant

recruitment and retention, as well as clinician engagement. We then distributed a newsletter

summarizing the recommendations we received and describing how we had incorporated

stakeholder feedback into our study procedures (see Appendix D). We also sent biannual

newsletters that described updates about overall study progress, including participant accrual,

upcoming stakeholder engagement opportunities, recent press highlights, study‐related

presentations or publications, and any other relevant information. We held a midstudy

luncheon at MGH (and via teleconference call) for all stakeholders, during which we discussed

study progress and facilitated initial conversations about dissemination. Last, we conducted

final focus groups with each of the initial 4 key stakeholder groups at the end of the study to

present preliminary results and discuss plans for dissemination and implementation. The

patient/family stakeholder group participated in person at a luncheon while the other 3 groups

from across the country participated via teleconference. Feedback from this final engagement

was instrumental in informing the next steps for this project. Table 2 displays a summary of

feedback from these final stakeholder focus groups. For example, stakeholders recommended

examining the role for social support in monitoring adherence and symptoms, suggested the

option to integrate information directly into the electronic health record (EHR), provided ideas

for implementation with involvement of pharmacy groups, and encouraged dissemination via

society newsletters, organizational webinars, posts, and listservs.

23

E. METHODS

Phase 1

Study design. We used an mHealth intervention development framework47 to guide

the creation of our smartphone‐based, patient‐centered intervention with maximum usability,

acceptability, and feasibility. In phase 1, we (ie, the investigative team of oncologists,

psychiatrists, and psychologists) developed the mobile app intervention through an iterative

process with the Partners Center for Connected Health, patients, clinicians, and the 4 key

stakeholder groups (n = 32) described previously. Stakeholder groups also provided feedback

regarding the design and implementation of the RCT in phase 2 of the study. See the

Stakeholder Engagement section for an in‐depth description of the iterative, multistep process

we undertook to ensure usability and feasibility of the mobile app intervention. Briefly, we first

led focus groups with key stakeholders to solicit feedback regarding the study design, clinically

relevant content, and functionality of the mobile app. We then worked with our technology

partners to create screen blueprints (known in the software development industry as

wireframes) of the proposed mobile app. Next, we presented the wireframes to MGH patients

(n = 10) and clinicians (n = 8) to solicit feedback on the content, design, and patient

centeredness of the intervention. After incorporating this feedback and refining the mobile

app content, we invited patients and families from the original stakeholder group to

participate in user acceptance testing with the beta version of the app to assess task

intuitiveness and to share general feedback. We further modified the app based on feedback

from each stage of this process in order to ensure optimal usability and feasibility for testing in

the RCT. The Dana Farber/Harvard Cancer Center Institutional Review Board approved the

study.

Forming the study cohort

Stakeholder groups. Drawing from a model of population‐based patient‐centered care,

the investigative team identified stakeholders from various cancer community groups across a

diverse range of expertise. Thirty‐two stakeholders comprised 4 key stakeholder groups:

patients and families (n = 8); oncology clinicians (n = 8); cancer practice administrators (n = 8);

24

and representatives of the health system, community, and society (n = 8). Stakeholders

included pharmacists, health care leaders, lawyers, and patient advocates. These individuals

were consultants on the study and not participants; therefore, no demographic or other

personal information was collected from these individuals.

MGH patients. Patients were eligible to participate if they had a cancer diagnosis, had a

current or past prescription for oral chemotherapy, and were the primary owner and user of a

smart mobile phone with an iOS or Android operating system. Eligibility criteria also included

age ≥ 18, ability to respond to survey questions in English, and a performance status ≤ 2 on the

Eastern Cooperative Oncology Group (ECOG) measure.48 We implemented the age criterion to

maximize the likelihood that patients were administering their own medications. We chose an

ECOG performance status ≤ 2 to ensure that patients had sufficient functioning to participate

in the study. We required patients to be receiving their care at MGH or a community affiliate

(ie, Mass General/North Shore Cancer Center, Mass General West, or Mass General Cancer

Center at Emerson‐Bethke). We excluded patients with comorbid acute or psychiatric

symptoms or neurological dysfunction that would interfere with consent and participation.

Additionally, we excluded patients who were enrolled in oral chemotherapy clinical trials

because the strict adherence monitoring of drug trials, could influence the proposed study

outcomes. After screening the EHR to determine preliminary eligibility and obtaining

permission from the patient’s treatment team, a trained Research Assistant (RA) either

contacted the patients by phone or approached them in private clinic settings within MGH

Cancer Center to explain the study and invite the patient to complete the eligibility screen.

MGH oncology clinicians. Oncology clinicians included board‐certified oncologists or

nurse practitioners who maintained at least 25% clinical practice at one of the study sites.

Study staff directly approached and recruited 8 oncology clinicians to participate in qualitative

interviews either in person or over the telephone.

Study setting. The patient and family member stakeholder focus groups, as well as the

individual MGH patient and clinician interviews, took place in person on site at the MGH

Cancer Center in order to optimize involvement and feedback. Interviews with the remaining 3

25

stakeholder groups took place via teleconference call to accommodate individuals who resided

throughout the United States.

Intervention. We did not administer an intervention during phase 1.

Follow‐up. Stakeholders continued to provide feedback into phase 2 regarding the

mobile app content. During this period, we solicited feedback primarily in the form of email

surveys (eg, see Appendix D), biannual newsletters, a midstudy luncheon, and final focus

groups (see Table 2).

Study outcomes. The primary study outcome for phase 1 was to develop a patient‐

centered mobile app to assess symptoms, side effects, and adherence to oral chemotherapy

that is feasible for use with oncology patients. The criteria for success was to ensure that the

mobile app met standards for usability, acceptability of delivery, and patient‐centeredness per

expert evaluation and qualitative feedback from interviews with oncology patients, clinicians,

and key stakeholders.

Data collection and sources. A trained clinical psychologist and psychology

postdoctoral fellows administered all individual and group interviews. The semistructured

interviews with the stakeholder groups, MGH patients, and MGH oncology clinicians were

audio‐recorded and stored on the secure, encrypted MGH server. See Appendices A and B for

the interview guides used during phase 1 of the study.

Analytic and statistical approaches. Study staff reviewed all interviews so that the

feedback obtained could inform modifications and refinements to the mobile app. Specifically,

trained research assistants transcribed the feedback from the interviews to generate a

complete list of comments, impressions, and recommendations from the MGH patient and

clinician participants in phase 1 as well as from the 4 key stakeholder groups. After the

completion of each focus group, we shared a summary of the focus group results with the

stakeholders via email and again elicited any final feedback. The investigative team, including

the study staff who conducted the interviews, then reviewed these reports for

comprehensiveness and accuracy. Finally, in close collaboration with the technology experts at

the Partners Center for Connected Health, the investigative team decided by consensus how to

modify the app and optimize user engagement in consideration of all feedback generated from

26

the interviews. The investigative team also considered the impact of stakeholders’

recommendations on the scientific integrity and feasibility of the study.

Conduct of the study. During phase 1 of the study protocol, we submitted 1

amendment to the IRB. Specifically, on July 10, 2014, we submitted an amendment proposing

to change the format of stakeholder focus groups to individual or group interviews in order to

best accommodate the scheduling needs of our collaborators. This amendment was approved

by the IRB on July 15, 2014.

Phase 2

Study design. We enrolled patients with diverse malignancies who were prescribed

oral chemotherapy to participate in a nonblinded, randomized, parallel assignment efficacy

trial of the mobile app intervention compared with standard oncology care (ClinicalTrials.gov

Identifier: NCT02157519). Patients were receiving care at the MGH Cancer Center or a

community affiliate. Independent of the research team, the study statistician developed a

computer‐generated randomization scheme stratified by cancer type (hematologic malignancy

versus solid tumor) to ensure that relatively equal proportions of diagnoses were represented

in each study group. The Dana‐Farber/Harvard Cancer Center Office of Data Quality then

randomly assigned participants 1:1 to either the mobile app intervention group or the

standard oncology care control group. The first 5 patients who were randomized to the mobile

app intervention participated in beta testing. After they completed the study, research staff

conducted a semistructured 20‐minute interview to gather their feedback on the app’s

feasibility, usability, and aesthetics, allowing users to suggest revisions. The research team also

interviewed 5 oncology clinicians whose patients were randomized to the mobile app

regarding their conversations about the app with patients and helpfulness of the symptom

reporting feature.

Forming the study cohort. Eligibility criteria for patients in the RCT during phase 2 of

the study were nearly identical to those of MGH patients participating in phase 1 development

(see Phase 1: Forming the Study Cohort: MGH Patients). However, for the RCT, patients were

also required to have a current and active prescription for oral chemotherapy in order to

enroll.

27

After screening the EHR to determine potential eligibility of patients, a study staff

member obtained permission from the patient’s oncology team to approach the patients and

explain the study. On receiving permission from the oncology clinicians, an RA contacted the

patients by telephone or approached them in a private clinic room to assess interest and

complete a brief screen to confirm eligibility. Our recruitment protocol addressed

Methodology Standard PC‐2 by systematically identifying all patients who were prescribed oral

chemotherapy via the EHR. Unlike other methods of recruitment that we considered, such as

patient self‐referral or clinician referral, systematic searching of the EHR eliminated any

selection bias in screening and enrolling of participants. Furthermore, once they were enrolled,

we used the same standard operating procedures with all participants to ensure that there

were no biases in retention. To address the representativeness of participants, we recruited

patients at 3 community affiliate sites in addition to the main academic hospital site. This

approach facilitated the enrollment of participants who choose not to receive care at a tertiary

medical center for financial, geographical, or other reasons.

Study setting. We recruited patients from MGH Cancer Center or one of the

community affiliates listed previously. Study visits took place in conjunction with scheduled

outpatient oncology appointments.

Interventions

Mobile app intervention. Patients randomly assigned to receive the mobile app

intervention met with the RA to download the mobile app (Chemotherapy Assistant, or CORA)

to their personal smartphone. RAs oriented the patients on how to use the app, enter their

treatment plan, and complete weekly symptom and adherence reports. They instructed

participants to use the mobile app for approximately 12 weeks. The mobile application

intervention consisted of several elements that we had developed and refined based on

stakeholder and participant feedback in phase 1. The essential app components included the

medication treatment plan and reminder features, a symptom and adherence reporting

module that was transmitted weekly to the respective oncology clinician, and educational

resources (see Appendix E). Push notifications reminded patients to take their medications and

complete weekly symptom and adherence reports. Push notifications are pop‐up messages

28

that appear on the mobile device to remind the user to engage with the app. No extra staffing

was required, as patients who reported serious symptoms (eg, fever) were instructed within

the app to call their oncology clinician or go to the nearest emergency department. Patients

were informed during the consent process and app orientation process that their reporting

would not be monitored in real time, so that there was no expectation of an immediate

response. The research team encouraged oncology clinicians to follow up on the weekly

symptom reports based on their clinical judgment, though no data were collected from

clinicians about how such reports may have affected their clinical care. Patients were asked to

store their oral chemotherapy medication in a Medication Event Monitoring System cap and

bottle.

Standard oncology care. Patients randomly assigned to standard care did not receive

the mobile app but rather received care as usual from their oncology clinicians. These

participants were also asked to store their oral chemotherapy medication in the Medication

Event Monitoring System Cap and bottle.

Follow‐up. Patients completed the baseline self‐report surveys prior to randomization.

Subsequently, study staff contacted patients by telephone or during a routine clinic visit to

have them complete an identical survey 12 (+/–3) weeks after the baseline assessment.

Patients had the option to complete surveys on paper or via REDCap,49 an electronic HIPAA‐

compliant survey tool. On completion of the postassessment survey, RAs instructed

participants on how to delete the mobile app from their smartphone.

We followed up with participants at 2 weeks postbaseline and at 6 weeks postbaseline

to ensure that they completed study procedures per the protocol. Additionally, if a patient

who was randomized to the mobile app group did not complete a weekly symptom report

during the first active week of the study, a study staff member called to make sure the mobile

app was working properly. Attrition was not significant in our study.

Study outcomes: primary outcome measures

Adherence to oral chemotherapy medication. We employed a multimethod

assessment of adherence given that all sources of measurement (eg, self‐report, pill counts,

pharmacy refill data, and electronic monitoring) have different strengths and limitations with

29

potential for bias. The assessment therefore included remote electronic monitoring devices

and self‐report instruments as follows:

1. Medication Event Monitoring System (MEMS)® Cap. The MEMSCap records the date

and time that the pill bottle is opened and medication is taken. These data were stored

on the MEMSCap and collected by the study team postassessment. MEMSCaps are

widely used in adherence monitoring and have been used in patients with cancer.50

2. Morisky Medication Adherence Questionnaire (MMAS‐4). The MMAS‐4 is a brief, self‐

report, validated measure to assess medication‐taking behavior over the past week.

Patients are asked to respond to each of 4 items with a “yes” or “no.” The 4‐item scale

has good sensitivity in identifying nonadherent individuals.51

3. Pill Diary. We provided patients with a weekly log to keep track of medication doses

that they took without using the MEMSCap. Usage of the pill diary was optional, but all

patients received one as a backup for documenting adherence.

Symptoms and side effects. To assess symptoms, patients completed the M.D.

Anderson Symptom Inventory (MDASI), a 19‐item instrument that assesses the most common

physical and psychological symptoms related to cancer. The MDASI assesses the severity of

symptoms at their worst in the past 24 hours on a 0‐to‐10 scale, with 0 being “not present”

and 10 being “as bad as you can imagine.” Two subscales are computed to measure

interference and severity of symptoms. The measure has been validated in patients with

diverse malignancies, and test–retest and internal consistency reliability is confirmed.52 The

MDASI demonstrated strong reliability in this sample (severity α = .93; interference α = .94).

Quality of life. We administered, the Functional Assessment of Cancer Treatment–

General (FACT‐G), a 27‐item questionnaire that assesses physical, social/family, emotional, and

functional well‐being during the previous week, to assess QOL. The validated measure utilizes

a 5‐point scale from 0 (not at all) to 4 (very much). It has sound psychometric properties, is

used widely in patients with cancer,53 and showed good reliability in this sample (α = .70).

Study outcomes: secondary outcome measures

Treatment satisfaction. The Functional Assessment of Chronic Illness Treatment–

Treatment Satisfaction–Patient Satisfaction (FACIT‐TS‐PS) is a 29‐item questionnaire that

30

assesses patient satisfaction with doctor and staff communication, competence, and

confidence, as well as trust in providers and overall satisfaction. Higher scores indicate greater

satisfaction. The FACIT‐TS‐PS has high validity and reliability,54 and the instrument

demonstrated strong reliability in this sample (α = .91). To reduce questionnaire burden on

patients, we administered 5 subscales of the FACIT‐TS‐PS, which assess satisfaction with (1)

clinician explanations, (2) interpersonal treatment, (3) comprehensiveness of care, (4) nurse

communication, and (5) confidence and trust in the doctor and treatment staff.

Urgent visits. We administered the Resource Utilization Questionnaire, an adapted 3‐

item questionnaire, to inquire about the number of emergency department visits and hospital

admissions in the past 3 months.

Potential moderators: measures for exploratory analyses

Sociodemographics. Participants reported their gender, race, ethnicity, religion, marital

status, smoking history, income, and level of education on a baseline demographic

questionnaire. Research staff collected data from the electronic health record on patients’ age,

cancer diagnosis, ECOG performance status, therapy dosing schedule (continuous dosing

versus interval dosing), type of oral therapy (targeted therapy versus oral chemotherapy),

number of concomitant medications, and duration of oral therapy treatment.

Mood. The Hospital Anxiety and Depression Scale (HADS) was designed for medical

patients and demonstrates adequate psychometric properties for use among individuals with

cancer.55 Composed of 14 items, the instrument contains 2 subscales that measure anxiety and

depression symptoms in the past week, with scores ranging from 0 (no distress) to 21

(maximum distress). A threshold of > 7 indicates clinically significant anxiety or depression, and

a score of > 11 indicates definitive anxiety or depression.55 A trained study psychologist

followed up with all patients who scored > 11 on the depression subscale. The HADS

demonstrated strong reliability in this sample (α = .94).

Social support. The Multidimensional Scale of Perceived Social Support (MSPSS) is a 12‐

item questionnaire that assesses perceived social support on 1‐to‐7 scale, with 1 being “very

strongly disagree” and 10 being “very strongly agree.” Three subscales, each comprising 4

items, are computed to assess perceived social support from family, friends, and significant

31

others. The MSPSS has adequate test–retest and internal reliability, and high factorial

validity.56 The MSPSS demonstrated strong reliability in this sample (α = .96).

Health literacy. The Rapid Estimate of Adult Literacy in Medicine is a 2‐ to 3‐minute

assessment of medically relevant vocabulary (66 total words) that has been shown to correlate

well with other measures of various literacy skills.57

App usability. The study team adapted the App Usability Questionnaire from the

System Usability Scale (SUS),58 a validated, easily administered scale. We adapted the 10‐item

SUS to a simplified, 6‐item Likert scale ranging in response from “strongly disagree” to

“strongly agree.” Final scores can range from 0 to 30. Higher scores indicate higher perception

of usability. Scores above 21 (ie, 70% of total score) can be considered to have good usability.59

Data collection and sources. The study RA called all participants at the time of post‐

assessment and reminded them to complete the self‐report questionnaire and return the

electronic pill bottle, either at their next clinic visit or by mail. RAs checked questionnaires in

real time for incomplete items and solicited clarification from participants. Of the 181 patients

randomized in this trial, 12 did not complete post‐assessment questionnaires. Of these

patients, 7 withdrew from the study: 4 patients opted to discontinue because of study burden,

2 were unable to continue because their phone became incompatible with the mobile app,

and 1 became too ill to continue in the study. An additional 4 patients died prior to completing

post‐assessment questionnaires, and 1 patient was lost to follow‐up.

We were unable to retrieve MEMS data from 11 patients. Of these patients, 9 did not

return their MEMS pill bottle to our study team, we were unable to download data from 1

participant’s bottle due to a technical issue, and 1 participant’s MEMScap was lost in the mail.

Analytic and statistical approaches. We used SPSS ([computer program] Version 22.0.

Chicago, IL: SPSS) to conduct statistical analyses, first with all available baseline and follow‐up

data and then using Multiple Imputation to account for missing data. We described

demographic and clinical characteristics with measures of central tendency or percentages

(see Table 3).

32

Table 3. Phase 2: Sociodemographic, Clinical, and Psychosocial Characteristics in the Full Sample and by Study Group

Characteristic Mean (Standard Deviation) or N (%)

Full Sample (n = 181)

Standard Care (n = 90)

Mobile App (n = 91)

Age (range = 21‐88) 53.30 (12.91) 53.76 (12.08) 52.85 (13.74) Gender Women 97 (53.6%) 51 (56.7%) 46 (50.5%) Men 84 (46.4%) 39 (43.3%) 45 (49.5%) Race White 159 (87.8%) 75 (83.3%) 84 (92.3%) Asian 10 (5.5%) 4 (4.4%) 6 (6.6%) Black or African American 5 (2.8%) 5 (5.6%) 0 (0.0%) Hispanic or Latino/a 4 (2.2%) 4 (4.4%) 0 (0.0%) Multiracial 2 (1.1%) 2 (2.2%) 0 (0.0%) Other 1 (0.6%) 0 (0.0%) 1 (1.1%) Ethnicity Hispanic or Latino/a 5 (2.8%) 4 (4.4%) 1 (1.1%) Education Advanced degree 81 (44.8%) 35 (38.9%) 46 (50.5%) Some college/technical school 44 (24.3%) 27 (30.0%) 17 (18.7%) College graduate 42 (23.2%) 21 (23.3%) 21 (23.1%) High school graduate/GED 14 (7.7%) 7 (7.8%) 7 (7.7%) Relationship status Married/ living with someone as if married 136 (75.1%) 68 (75.6%) 68 (74.7%) Single, never married 17 (9.4%) 7 (7.8%) 10 (11.0%) Divorced/separated 13 (7.2%) 8 (8.9%) 5 (5.5%) Noncohabitating relationship 9 (5.0%) 6 (6.7%) 3 (3.3%) Loss of long‐term partner/widowed

Declined to respond 5 (2.8%) 1 (0.6%)

1 (1.1%) 0 (0.0%)

4 (4.4%) 1 (1.1%)

Employment status Full‐time or part‐time work or school 110 (60.8%) 56 (62.2%) 54 (59.3%)

33




Mobile App (n = 91)

Retired/unemployed/disability 69 (38.1%) 33 (36.7%) 36 (39.6%) Other or missing 2 (1.1%) 1 (1.1%) 1 (1.1%) Has children 140 (77.3%) 72 (80.0%) 68 (74.7%) Religion Catholic 79 (43.6%) 45 (50.0%) 34 (37.4%) None 37 (20.4%) 20 (22.2%) 17 (18.7%) Protestant 24 (13.3%) 10 (11.1%) 14 (15.4%) Other 26 (14.4%) 12 (13.3%) 14 (15.4%) Jewish 9 (5.0%) 1 (1.1%) 8 (8.8%) Muslim 1 (0.6%) 0 (0.0%) 1 (1.1%) Declined to respond 5 (2.8%) 2 (2.2%) 3 (3.3%) Income < $25 000 16 (8.8%) 9 (10.0%) 7 (7.7%) $25 000‐$50 000 19 (10.5%) 12 (13.3%) 7 (7.7%) $51 000‐$100 000 40 (22.1%) 17 (18.9%) 23 (25.3%) $101 000‐$150 000 49 (27.1%) 24 (26.7%) 25 (27.5%) > $150 000 51 (28.2%) 25 (27.8%) 26 (28.6%) Declined to respond 6 (3.3%) 3 (3.3%) 3 (3.3%) Cancer type Hematologic 60 (33.1%) 30 (33.3%) 30 (33.0%) Non–small cell lung cancer 33 (18.2%) 16 (17.8%) 17 (18.7%) Breast cancer 26 (14.4%) 15 (16.7%) 11 (12.1%) High‐grade glioma 21 (11.6%) 12 (13.3%) 9 (9.9%) Sarcoma 12 (6.6%) 4 (4.4%) 8 (8.8%) Gastrointestinal 8 (4.4%) 2 (2.2%) 6 (6.6%) Genitourinary 7 (3.9%) 3 (3.3%) 4 (4.4%) Melanoma 7 (3.9%) 3 (3.3%) 4 (4.4%) Low‐grade glioma 5 (2.8%) 4 (4.4%) 1 (1.1%)

34




Mobile App (n = 91)

Nongastrointestinal stromal tumor sarcoma 2 (1.1%) 1 (1.1%) 1 (1.1%) Stage of disease (solid staged tumors only; n = 85) 0 1 (1.2%) 1 (2.6%) 0 (0.0%) I 2 (2.3%) 2 (5.1%) 0 (0.0%) II 5 (5.9%) 1 (2.6%) 4 (8.7%) III 4 (4.7%) 2 (5.1%) 2 (4.3%) IV

73 (85.9%)

33 (84.6%)

40 (87.0%)

Metastatic disease (solid tumors only; n = 121) 77 (63.6%) 34 (56.7%) 43 (70.5%) Type of oral therapy Targeted therapy 121 (66.9%) 56 (62.2%) 65 (71.4%) Chemotherapy 60 (33.1%) 34 (37.8%) 26 (28.6%) Duration of oral therapy in months (range = 0‐136) 12.70 (20.87) 13.36 (22.12) 12.04 (19.67) ECOG performance status 0 89 (49.2%) 49 (54.4%) 40 (44.0%) 1 87 (48.1%) 40 (44.4%) 47 (51.6%) 2 5 (2.8%) 1 (1.1%) 4 (4.4%) Number of prescribed medications (range = 0‐19) 5.82 (3.99) 5.98 (4.02) 5.67 (3.97)

Abbreviation: ECOG, Eastern Cooperative Oncology Group.

35

Aim 1: To implement a patient‐centered mobile app to assess symptoms, side effects,

and adherence to oral chemotherapy that is feasible for use with oncology patients. To

assess feasibility of participants using the mobile app, we examined completion rates of

symptom reports during the 12‐week study period. The app was considered feasible if 75% of

participants assigned to the intervention completed 75% of possible symptom reports or more

than 9 total reports. We examined participants’ perception of app usability by interpreting the

means and standard deviations on the app usability questionnaire. Scores above 70% were

considered acceptable, those between 80% and 90% were considered good, and those above

90% were considered superior.

Aim 2: To evaluate the efficacy of the mobile application in improving adherence and

patient‐reported clinical outcomes. For tests of aim 2, we examined between‐group

differences in changes in the primary outcomes from baseline to the 12‐week follow‐up

assessment using linear regression models. We created difference scores (post minus baseline)

and conducted each model by regressing the change in each outcome (dependent variable) on

study group assignment (independent variable) and interpreting the unstandardized

coefficients, represented by the capital letter B. We considered estimates statistically

significant based on a 2‐sided α of 0.05 and 95% confidence intervals. We included the change

in perceived social support on the MSPSS as a covariate in all models for 2 reasons. First, we

selected this for use as a covariate a priori, due to the documented relationship between social

support and adherence. Second, we observed that there was a marginally significant

difference in perceived social support over time on the MSPSS, such that patients assigned to

the mobile app intervention reported larger decrements in perceived social support compared

with those assigned to the standard care group (MeanDiff = 0.39; SEDiff = 0.20; t162 = 1.90; p =

.060).

Aim 3: To evaluate the efficacy of the mobile application in improving quality of

oncology care. For tests of aim 3, we examined between‐group differences in changes in the

secondary outcomes, conducted in an identical fashion to tests of aim 2.

Exploratory Aim: To determine whether particular patient demographic and clinical

characteristics moderate the effect of the study intervention. We also conducted tests of

36

treatment response heterogeneity with the goal of determining whether the treatment effect

of the mobile app varied among levels of baseline and other factors. We prespecified

subgroups of interest in the study design based on our previous research.18 To identify

moderators of the treatment effect, we first examined demographic and clinical characteristics

known to be related to poorer adherence. These factors included being less educated or less

health literate, not being married or partnered, having lower perceived social support, having

higher anxiety or depression, and reporting memory problems. We also examined

demographic and clinical factors that have been inconsistently related to adherence (ie,

gender, age, number of concomitant medications, duration of oral therapy treatment), or

those that were theoretically believed to influence the treatment effect or overall adherence

(ie, type of cancer [hematologic malignancy versus solid tumor], therapy dosing schedule

[continuous dosing versus interval dosing], type of oral therapy [targeted therapy versus oral

chemotherapy], and functional performance status [ECOG]).

To conduct subgroup analyses, we first created interaction terms (study condition by

subgroup characteristic) and regressed each outcome on the interaction term, the subgroup

characteristic, and study group assignment, controlling for change in perceived social support

(per the MSPSS). Given that tests for interactions usually have limited power, and that the lack

of a significant interaction does not definitively eliminate the possibility of treatment

heterogeneity, we further probed interaction terms with α < 0.10 to examine the effects of

study group assignment on the outcome across levels of the moderator.60 For categorical

moderators, we examined the effect of study group assignment on the outcome for each

subgroup. For continuous moderators, we used an empirical cutoff when applicable, or applied

the Johnson‐Neyman technique in the PROCESS macro for SPSS,61 which uses iterative

approximation to calculate regions of significance and identify the optimal cutoff.

Power analyses. Using the effect size estimates from our prior pilot investigation, we

had 80% power to detect a statistically significant improvement in adherence rates from 0.70

to 0.90 with a sample size of 150 patients (75 patients per group). While we originally aimed to

enroll 180 participants in the study based on this power analysis, we increased the accrual goal

to 220 participants to account for attrition. The larger sample size also helped increase power

37

to explore potential moderators (ie, identify subgroups of patients who may respond

differently to the mobile app intervention).

Missing data analyses. Data were missing at postassessment for 12 participants.

Reasons for missing data were as follows: withdrawal (n = 7); death (n = 4); lost to follow‐up (n

= 1). Due to a clerical error with administering the MDASI, data on this measure were missing

for 31 participants at the baseline assessment. Otherwise, 1 participant did not complete the

baseline survey and therefore had missing data on most baseline measures. We first

conducted statistical analyses using all available baseline and follow‐up data and then, to

address missing data concerns, we repeated the analyses with imputed data using the Multiple

Imputation method in SPSS.62

Conduct of the study. Over the course of this study we amended the protocol

(Appendix F) to restructure the assessment timeline; add a resource utilization questionnaire;

add the M.D. Anderson Symptom Inventory questionnaire; add an optional pill diary; add

community cancer clinic affiliates in North Shore, Emerson, and MGH West as study sites; add

an app usability questionnaire; and increase accrual from 180 to 220 participants to ensure

that at least 180 patients were randomized.

38

F. RESULTS

Phase 1

Participant characteristics. We previously described stakeholder involvement, and

characteristics of MGH patients and oncology clinicians are presented in Table 1.

Final mobile app intervention. To meet criteria for phase 1, aim 1, we developed a

patient‐centered mobile app to assess symptoms, side effects, and adherence to oral

chemotherapy that is feasible for use with oncology patients. We successfully created the app

content with input from key stakeholders, the research team, and technology experts.

Summary findings of feedback from the individual interviews with MGH patients and clinicians

as well as the 4 stakeholder focus groups during phase 1 are presented in Table 2 and

Appendix C. In addition, example feedback in email communication with stakeholders and the

research team’s response is presented in Appendix D.

The mobile app, CORA, was written primarily in JavaScript language and developed on

the Titanium 3.5 and 5.0 platform to ensure cross‐platform functionality on both Apple iOS

and Android devices. CORA was supported by a PHP/MySQL database and hosted on a LAMP

server that met HIPAA Security Rule requirements. The entire study team participated in code

reviews and quality assurance testing with each code release. We included MGH oncology

patients in usability testing and beta testing to ensure app usability for implementation in the

RCT. Qualitative feedback from interviews with oncology patients, family members, clinicians,

and key stakeholders indicated that the app was feasible and acceptable for use in this patient

population. Table 4 illustrates examples of how we incorporated feedback from stakeholders

into the final mobile app. CORA underwent 7 version updates to address integration with

third‐party smartphone operating systems (n = 4) and to fix software bugs or make minor

improvements (n = 3). The final version of CORA is organized in functional modules (Appendix

E), including a medication treatment plan with a timeline and reminder system, reporting

features for adherence and symptoms along with graphics, educational resources and recipes,

integrated wearable fitness tracking with Fitbit, and a section for notes and questions.

39

Table 4. Mobile App Modifications Based on Stakeholder Feedback

Stakeholder Group Stakeholder Feedback Module: Feature Implementation

Patients and families

“Connect patients with the same disease type for social support.”

Education Library Module: Resources and Social Networking

CORA includes a list of reputable, disease‐specific resources for patients looking to connect with others.

Health care representatives

“Provide patients with anchors and definitions of symptoms so they can

appropriately determine the severity and urgency of their

symptoms.”

Symptom Reporting: Frequency and duration

When a patient reports a symptom, CORA asks several questions about the frequency and duration before providing tailored feedback.

Oncology clinicians

“The weekly symptom reports that are sent to clinicians should be concise and easy to understand.”

Symptom Reporting: Trends

Weekly Symptom Reports provide a list of symptoms reported by the patient, as well as a color and numeric value (1‐10) denoting severity.

Practice administrators

“Provide resources and contact information for patients to use when they miss a dose of their

medication.”

Symptom Reporting–“Touch to call clinical team”

Patients receive study team contact information at baseline. Embedded in the symptom reporting feature is a “touch to call” button for their specific clinic.

40

Phase 2

Participant characteristics. Of the 696 potentially eligible patients screened via the

EHR, 196 were not approached for the study because the oncologist denied our request to

approach (n = 64) or did not respond to our request to approach (n = 134) the patient. We

therefore approached 500 patients in clinic, 178 (35.6%) of whom did not own a smartphone,

and 110 (22.0%) of whom declined to participate. Reasons patients cited for refusal included

not interested in the intervention (n = 43), not interested in participating in any research (n =

25), not comfortable using their smartphone (n = 25), belief that the study would be too

burdensome/disrupt current treatment (n = 16), or concerns about the security of their data (n

= 1). The remaining 212 enrolled in the study and were scheduled to complete baseline

assessments at their next outpatient oncology visit. The baseline visit occurred on average 36

days (SD = 49 days) after enrollment. During this time, 28 participants dropped out of the

study, 3 were lost to follow‐up (see CONSORT flow diagram, Figure 3), and 181 completed

baseline assessments and were then randomized to either the mobile app (n = 91) or standard

care (n = 90). Of this total of 181 patients, we had recruited 173 from MGH and 8 from

community affiliate sites. A total of 169 patients completed the postassessment survey at the

12‐week follow‐up. Reasons for incomplete assessments included patient withdrawal (n = 7),

death (n = 4), and loss to follow‐up. MEMSCap data were available on 170 patients; 9 patients

did not return their pill bottle, 1 cap was lost in the mail, and 1 did not have available data on

the cap. No study‐related adverse events occurred over the course of the study. As noted, 1

participant’s MEMSCap data were lost in the mail (Table 5); however, confidentiality was not

breached because no identifiable participant information was in the envelope or on the bottle.

Table 6 presents Patient Intervention Comparison Outcome (PICOT) descriptors, and Appendix

G references results tables submitted to ClinicalTrials.gov.

41

Figure 3. CONSORT Flow Diagram

42

Table 5. Adverse Events Overview

Events Mobile App (n = 91) Standard Care (n = 90) Affected/At Risk (%) No. Events Affected/At Risk (%) No.

Events

All‐cause mortality 3 (3.3%) 3 1 (1.11%) 1 Serious adverse event 0 (0%) 0 0 (0%) 0Other adverse event Product issuea,b 1 (1.1%) 1 0 (0%) 0

a Adherence data were lost due to the patient’s deidentified MEMSCappill bottle being lost in the mail. There was no breach of confidentiality. b Collection approach: nonsystematic assessment.

43

Table 6. PICOT Descriptors

PICOT Descriptor Study‐specific Description

Patient group Patients with diverse malignancies prescribed oral chemotherapy

Intervention 12‐week mobile app intervention to monitor adherence and symptoms

Control or comparator Standard oncology care

Outcomes (main/important outcomes)

Primary outcomes: changes in patient‐reported adherence (MMAS‐4),

electronically monitored adherence (MEMSCap), symptom severity and

interference (MDASI), quality of life (FACT‐G)

Secondary outcomes: treatment satisfaction (FACIT‐TS‐PS), urgent emergency

department visits and hospitalizations (RUQ), app usability (percentage of

symptom reports completed)

Timing (duration of follow‐sup) 12 weeks (+/– 3 weeks)

Setting Massachusetts General Hospital Cancer Center and community affiliates:

outpatient oncology clinics

Study design Semistructured interviews and 2‐arm randomized, parallel assignment, efficacy trial

Abbreviations: FACIT‐TS‐PS, Functional Assessment of Chronic Illness Therapy–Treatment Satisfaction–Patient Satisfaction; FACT‐G, Functional Assessment of Cancer Therapy‐General; MDASI, M.D. Anderson Symptom Inventory; MEMSCap, Medication Event Monitoring System Caps; MMAS‐4, Morisky Medication Adherence Scale, 4 items; RUQ, Resource Utilization Questionnaire.

44

Within the sample, patients were 53.30 years of age on average (SD = 12.91), half were

women (53.6%), the majority were Caucasian (87.8%), and 80.1% were partnered (see Table 3

for demographic characteristics of the sample). Participants were well‐educated, with 23.2%

having graduated from college, and an additional 44.8% having an advanced degree.

Approximately one‐third of patients had a hematologic malignancy (33.1%), followed by non–

small cell lung cancer (18.2%), breast cancer (14.4%), and high‐grade gliomas (11.6%). Most

patients (66.9%) were prescribed targeted therapies (ie, agents that specifically target cancer

cells with known oncogenic mutations) while the remainder were prescribed other oral

chemotherapies (33.1%). Appendix H lists the types of oral therapies patients were prescribed.

Patients had been taking oral therapies for an average of 12.70 months (SD = 20.87). At the

baseline assessment, 21.5% of patients (n = 39) reported problems with adherence to oral

therapies. Of the 170 participants with MEMSCaps data available, 52.9%, 22.4%, and 12.9% of

patients were less than 90%, 70%, and 50% adherent over the course of the study,

respectively.

Aim 1: To implement a patient‐centered mobile app to assess symptoms, side effects,

and adherence to oral chemotherapy that is feasible for use with oncology patients. In phase

2, tests of aim 1 showed that the feasibility aim was not met, with only 34% of patients

assigned to the mobile app completing the adherence and symptom reports on more than 75%

of the total possible study weeks. On average, patients assigned to the mobile app condition

completed 15.92 (SD = 14.15) reports over the course of the study (median = 14.00; IQR = 5.00

to 21.00). Patients completed the adherence and symptom reports on a mean of 6.43 weeks

(SD = 3.86) out of the 12 study weeks (57.1% of possible weeks). The average app usability

rating was good (M = 71.22; SD = 17.36), with 23.1% of patients reporting acceptable usability

(scores 70‐79), 21.2% reporting good usability (scores 80‐89), and 15.4% reporting superior

usability (scores 90‐100). On average, patients used the app for 59 minutes and 32 seconds (SD

= 1 hour, 8 minutes, and 15 seconds) over the course of the 12‐week study period and

accessed the app on 21.75 discrete days (SD = 21.24 days) out of 84 possible days. The

medication treatment plan timeline was the most frequently visited page of the app, followed

by the educational library, the symptom graph review, the ad hoc symptom reporting module,

45

and the free notes section. The most frequently reported symptoms in the app were fatigue

and disturbed sleep.

Aim 2: To evaluate the efficacy of the mobile application in improving adherence and

patient‐reported clinical outcomes. Tests of aim 2 evaluated the efficacy of the mobile app in

improving adherence as measured by MEMSCap, patient‐reported adherence, change in

symptom severity and interference, and change in QOL (Table 7). These analyses showed that

the mobile app intervention group and usual care control group did not differ with respect to

the primary outcomes of MEMSCap adherence rates, self‐reported adherence, symptoms, or

overall QOL. Specifically, at the postassessment, 23.3% of patients in the standard care group

and 13.8% of patients in the mobile app intervention reported poor adherence; however, this

difference was not statistically significant. Study groups also did not differ with respect to

objective MEMSCap adherence rates, change in symptom severity or interference, or overall

QOL. We observed a significant effect of group assignment on change in social and family well‐

being on the FACT‐G; patients in the mobile app intervention had a smaller reduction in social

and family well‐being from baseline to postassessment (Mchange = –0.55; SE = 0.53) compared

with the standard care group (Mchange = –2.22, SE = 0.50; Mdiff = 1.67, SE = 0.74, F1161 = 5.13, p =

.025, 95% CI [–3.12 to –0.21]).

Aim 3: To evaluate the efficacy of the mobile application in improving quality of

oncology care. Tests of aim 3 evaluated the efficacy of the mobile application in improving

secondary outcomes of quality of oncology care (Table 8). Study groups did not differ

significantly with respect to satisfaction with clinicians and treatment or the number of

emergency department visits or hospitalizations. We observed a marginally significant

difference in the change in Satisfaction with Interpersonal Treatment subscale on the FACIT‐

TS‐PS; patients in the mobile app intervention had a slight improvement in their satisfaction on

average (Mchange = 0.07; SE = 0.13) compared with those in the standard care group, who had a

slight reduction in satisfaction (Mchange = –0.29, SE = 0.13; Mdiff = –0.35, SE = 0.18, F1159 = 3.67, p

= .057, 95% CI [–0.72‐0.01].

46

Table 7. Differences in Primary Outcomes by Study Groupa

Primary Study Outcome N Mean (SE) or N (%)

Standard Care Mobile App

Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 162 20/86

(23.3%) 11/80 (13.8%) – 0.56 (0.46) .186 [0.23‐1.33]

Objective adherence rate (MEMSCap) 158

79.16 (2.78) 81.50 (2.93) –2.34 (4.06) –

.566 [–10.35‐5.68]

Δ Symptom severity (MDASI) 138 0.08 (0.15) –0.30 (0.15) 0.11 (0.21) – .603 [–0.31‐0.53]

Δ Symptom interference (MDASI) 137 –0.11 (0.23) –0.12 (0.23) 0.01 (0.33) – .982 [–0.64‐0.65]

Δ Quality of life (FACT‐G) 162 –1.93 (1.13) 0.49 (1.19) –2.42 (1.65) – .144 [–5.66‐0.83]

Δ Physical well‐being 163 0.11 (0.45) 0.81 (0.48) –0.70 (0.66) – .294 [–2.00‐0.61]

Δ Social/family well‐being 164

–2.22 (0.50) –0.55 (0.53) –1.67(0.74) –

.025b [–3.12 to –

0.21]

Δ Emotional well‐being 163 0.53 (0.40) 0.04 (0.41) 0.49 (0.58) – .392 [–0.64‐1.63]

Δ Functional well‐being 164 –0.40 (0.41) 0.35 (0.43) –0.75 (0.60) – .216 [–1.94‐0.44]

Abbreviations: Δ, change from baseline to postassessment; CI, confidence interval; FACT‐G, Functional Assessment of Cancer Therapy‐General; MDASI, M.D. Anderson Symptom Inventory; MEMSCap, Medication Event Monitoring System Caps; MMAS‐4, 4‐item Morisky Medication Adherence Scale; SE, standard error. a Self‐report adherence (MMAS‐4) analysis is controlling for baseline self‐reported adherence on MMAS‐4. All analyses are controlling for change in perceived social support (Multidimensional Scale of Perceived Social Support). b p < .05.

47

Table 8. Differences in Secondary Outcomes by Study Groupa

Secondary Study Outcome N Mean (SE) or N (%)

Standard Care Mobile App Mean

Difference (SE) P Value 95% CI

Satisfaction with treatment (FACIT‐TS‐PS)

Δ Clinician explanations 163 –0.34 (0.19) 0.04 (0.20) –0.38 (0.28) .170 [–0.93‐0.17]

Δ Interpersonal treatment 162 –0.29 (0.13) 0.07 (0.13) –0.35(0.18) .057 [–0.72‐0.01]

Δ Comprehensive care 162 –0.89 (0.49) 0.08 (0.52) –0.97 (0.72) .178 [–2.39‐0.45]

Δ Nursing communication 159 –0.26 (0.19) –0.45 (0.20) 0.20 (0.28) .481 [–0.35‐0.75]

Δ Trust and confidence in clinicians 163 –0.24 (0.12) –0.24 (0.13) 0.01 (0.17) .970 [–0.34‐ 0.35]

Emergency department visits (RUQ) 162 0.14 (0.04) 0.16 (0.04) –0.03 (0.06) .682 [–0.15‐0.10]

Hospitalizations (RUQ) 160 0.15 (0.07) 0.20 (0.07) –0.05 (0.10) .640 [–0.24‐0.15]

Abbreviations: Δ, change from baseline to postassessment; CI, confidence interval; FACIT‐TS‐PS, Functional Assessment of Chronic Illness Therapy–Treatment Satisfaction–Patient Satisfaction; RUQ, Resource Utilization Questionnaire. a All analyses are controlling for change in perceived social support (Multidimensional Scale of Perceived Social Support).

48

Exploratory Aim: To determine whether particular patient demographic and clinical

characteristics moderate the effect of the study intervention. The exploratory aim addressed

treatment heterogeneity by testing efficacy of the mobile app intervention within key patient

subgroups. We first examined the presence of an interaction between study group assignment

and the proposed baseline moderator factors in predicting primary outcomes. In tests of

moderation, we did not find evidence for moderation by the following factors: gender,

education, health literacy, relationship status, depression, perceived social support, type of

cancer (hematologic malignancy versus solid tumor), type of oral therapy (chemotherapy

versus targeted therapy), duration of oral therapy treatment, number of concomitant

medications, therapy dosing schedule (continuous dosing versus interval dosing), and

functional performance status (ECOG).

We did find evidence of moderation by the following factors: baseline self‐reported

adherence (MMAS‐4), baseline anxiety (HADS‐anxiety), and patient age. Linear regression

models examining self‐reported adherence at baseline as a potential moderator showed a

significant interaction between group assignment and baseline self‐reported adherence in

predicting the MEMSCaps adherence rate (B = 26.04; SE = 9.65; p = 0.008; 95% CI, 6.97‐45.10;

Table 9). Further examination of the effect of group assignment on the MEMSCaps adherence

rate at levels of the moderator (good adherence versus poor adherence) revealed that among

patients with poor self‐reported adherence at baseline (Table 10), those who were

randomized to the mobile app intervention had improved adherence on the MEMSCaps (M =

86.23; SE = 7.72) compared with those in the standard care control (M = 63.94, SE = 6.46; B =

22.30, SE = 10.06, p = .034, 95% CI [1.78‐42.82]; Figure 4). Self‐reported adherence at baseline

was not a moderator of the other primary study outcomes (all ps > .10).

Linear regression models examining anxiety as a potential moderator showed a

significant interaction between group assignment and self‐reported anxiety (HADS‐Anxiety

subscale) at the baseline assessment in predicting the MEMSCaps adherence rate (B = 17.55;

SE = 8.84; p = .049; 95% CI, 0.08‐35.02; Table 11). Probing at the levels of this moderator (low

anxiety versus high anxiety) indicated that among patients with high anxiety at baseline (Table

12), those randomized to the mobile app intervention had improved adherence on the

49

MEMSCaps (M = 85.46; SE = 5.57) compared with those in the standard care control group (M

= 69.39, SE = 5.19; B = 16.08, SE = 7.76, p = .044, 95% CI [0.41‐31.74]; Figure 5). Baseline

anxiety was not a moderator of the other primary outcomes (all p‐values > .10).

Finally, in linear regression models to test whether age was a potential moderator, we

found a significant interaction between study group assignment and age predicting change in

overall QOL on the FACT‐G (B = 0.27; SE = 0.13; p = .041; 95% CI, 0.01‐0.52; see Table 13).

50

Table 9. Linear Regression Examining Baseline Self‐reported Adherence as a Moderator of the Effect of the Mobile App Intervention on the Objective Adherence Rate per the MEMSCaps (n = 158)

Adherence Rate With MEMSCaps

Predictor Unstandardized Standardized

B SE [95% CI] β p

Condition (mobile app) –3.67 4.46 [–12.48‐5.14] –0.07 .412 Perceived social support (MSPSS) –1.63 1.53 [–4.66‐1.40] –0.08 .290 Self‐reported poor adherence (MMAS‐4)

–19.92 6.31 [–32.38 to –

7.47] –0.33 .002

Interaction (condition X self‐reported adherence) 26.04 9.65 [6.97‐45.10] 0.30 .008a

Total model Adjusted R2 = 0.05, F = 3.03 (4153), p = .020

Effect of the mobile app on the adherence rate in patients with poor adherence B = 22.30, SE = 10.06, p = .034 [1.78, 42.82]

Abbreviations: CI, confidence interval; MEMSCaps, Medication Event Monitoring System Caps; MMAS‐4, 4‐item Morisky Medication Adherence Scale; MSPSS, Multidimensional Scale of Perceived Social Support; SE, standard error. a p < .05.

51

Table 10. Differences in Primary Outcomes by Study Group in Patients With Self‐reported Poor Adherence at the Baseline Assessment on the Morisky Medication Adherence Scale (MMAS‐4)a



Difference (SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 35 9/21 (40.9%) 7/14 (41.2%) ‐ 1.33 (0.70) .683 [0.34‐5.21]

Objective adherence rate (MEMSCaps) 34

63.94 (6.46) 86.23 (7.72) –22.30 (10.06) – .034b [–42.82 to –

1.78]


Δ Symptom interference (MDASI) 29 0.19 (0.49) 0.16 (0.58) 0.03 (0.77) – .970 [–1.55‐1.60]


Δ Physical well‐being 35 –0.85 (0.97) 1.46 (1.18) –2.31 (1.53) – .141 [–5.42‐0.81]

Δ Social/family well‐being 35 –1.69 (0.90) –0.53 (1.11) –1.16 (1.43) – .424 [–4.06‐1.75]

Δ Emotional well‐being 35 0.14 (0.74) –0.001 (0.90) 0.15 (1.17) – .902 [–2.23‐2.52]


Abbreviations: Δ, change from baseline to postassessment; CI, confidence interval; FACT‐G, Functional Assessment of Cancer Therapy‐General; MDASI, M.D. Anderson Symptom Inventory; MEMSCaps, Medication Event Monitoring System Caps; MMAS‐4, 4‐item Morisky Medication Adherence Scale; SE, standard error. a Self‐report adherence (MMAS‐4) analysis is controlling for baseline self‐reported adherence on MMAS‐4. All analyses are controlling for change in perceived social support (Multidimensional Scale of Perceived Social Support). b p < .05.

52

Figure 4. Differences in MEMSCaps Adherence Rates Between Study Groups Moderated by Self‐reported Adherence (Good Versus Poor) at Baselinea

a Model adjusts for perceived social support on the Multidimensional Scale of Perceived Social Support.

53

Table 11. Linear Regression Examining Baseline Anxiety as a Moderator of the Effect of the Mobile App Intervention on the Objective Adherence Rate Measured With the MEMSCap (n = 158)



B SE [95% CI] β p

Condition (mobile app) –2.63 4.72 [–11.96‐6.70] –0.05 .578 Perceived social support (MSPSS) –1.26 1.56 [–4.33‐1.82] –0.07 .422 High anxiety (HADS‐anxiety)

–12.93 6.06 [–24.90 to –

0.97] –0.23 .034

Interaction (condition X anxiety) 17.55 8.84 [0.08‐35.02] 0.24 .049a

Total Model Adjusted R2 = 0.02, F = 1.61 (4153), p = .175

Effect of the mobile app on the adherence rate in patients with high anxiety B = 16.08, SE = 7.76, p = .044, 95% CI [0.41, 31.74]

Abbreviations: CI, confidence interval; HADS, Hospital Anxiety and Depression Scale; MEMSCaps, Medication Event Monitoring System Caps; MSPSS, Multidimensional Scale of Perceived Social Support; SE, standard error. a p < .05.

54

Table 12. Differences in Primary Outcomes by Study Group in Patients With Self‐reported High Anxiety on the Baseline Assessment on the Hospital Anxiety and Depression Scale‐Anxiety Subscalea



Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 46 8/25 (32%) 1/21 (4.3%) – 0.06 (1.44) .047b [0.003‐0.96]

Objective adherence rate (MEMSCaps) 45 69.39 (5.19) 85.46 (5.57) –16.08 (7.76) – .044b [–31.74 to –0.41]


Δ Symptom interference (MDASI) 39 0.19 (0.51) –0.90 (0.55) 1.09 (0.77) – .168 [–0.48‐2.65]



Δ Social/family well‐being 46 –4.07 (1.07) –1.36 (1.18) –2.71 (1.62) – .102 [–5.97‐0.56]

Δ Emotional well‐being 46 0.63 (0.81) 0.62 (0.89) 0.01 (1.23) – .995 [–2.47‐2.49]


Abbreviations: Δ, change from baseline to post‐assessment; CI, confidence interval; FACT‐G, Functional Assessment of Cancer Therapy‐General; MDASI, M.D. Anderson Symptom Inventory; MEMSCaps, Medication Event Monitoring System Caps; MMAS‐4, 4‐item Morisky Medication Adherence Scale; SE, standard error. a Self‐report adherence (MMAS‐4) analysis is controlling for baseline self‐reported adherence on MMAS‐4. All analyses are controlling for change in perceived social support (Multidimensional Scale of Perceived Social Support). b p < .05.

55

Figure 5. Differences in MEMSCaps Adherence Rates Between Study Groups Moderated by Anxiety (Low Versus High) at Baselinea


56

Table 13. Linear Regression Examining Patient Age as a Moderator of the Effect of the Mobile App Intervention on Change in Quality

of Life on the Functional Assessment of Cancer Therapy‐General (n = 162)

Δ QOL (FACT‐G)


B SE [95% CI] β p

Condition (mobile app) –11.72 7.03 [–25.61‐2.18] –0.56 .098 Perceived social support (MSPSS) 1.33 0.62 [0.10‐2.56] 0.17 .035 Patient age –0.16 0.09 [–0.34‐0.03] –0.19 .094 Interaction (group X patient age) 0.27 0.13 [0.01‐0.52] 0.71 .041

Total model Adjusted R2 = 0.04, F = 2.62 (4132), p = .037

Effect of the mobile app on the adherence rate in older patients (> 55 years old) based on Johnson‐Neyman Technique: B = 5.84, SE = 2.57, p = .027, 95% CI [0.70, 10.98]

Abbreviations: CI, confidence interval; FACT‐G, Functional Assessment of Cancer Therapy‐General; MSPSS, Multidimensional Scale of Perceived Social Support; QOL , Quality of Life, SE, standard error. *p < .05.

57

Using the Johnson‐Neyman technique in the PROCESS macro for SPSS,61 we identified the

optimal cutoff of greater than 55 years of age versus 55 years of age or younger. Patients

greater than 55 years old (Table 14) who were randomized to the mobile app intervention

reported improved overall QOL (M = 1.93; SE = 1.93) compared with those in the standard care

control (M = –3.90, SE = 1.68), B = 5.84, SE = 2.57, p = .027, 95% CI [0.70‐10.98], Figure 6. Age

was not a significant moderator of the effect of group assignment on other primary outcomes

(all ps > .10).

Missing data analyses. The rate of missing data at the postassessment time point was

6.6% for the self‐report questionnaires and 6.1% for the MEMSCap data. To account for these

missing data, as well as the missing baseline MDASI data due to a clerical error, we conducted

all analyses in an identical fashion using Multiple Imputation.62 Tables 15 to 22 display the

findings, which generally corroborate the available case analyses. Specifically, the only

significant primary main effect of the intervention was the same: Participants in the mobile

app group reported a smaller reduction in social well‐being over time than did those receiving

standard care. In addition, the marginally significant group difference in satisfaction with

interpersonal treatment (secondary outcome) became statistically significant with Multiple

Imputation, favoring the intervention. Otherwise, the subgroup analyses were essentially

replicated for the effect of the intervention on objective (MEMSCap) adherence in patients

who reported adherence problems at baseline. However, the moderator effects of anxiety on

adherence and age on quality of life became marginally significant with Multiple Imputation.

58

Table 14. Differences in Primary Outcomes by Study Group in Patients > 55 Years of Agea



Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 66 8/36 (20.5%) 5/32 (13.2%) – 0.57 (0.73) .435 [0.13‐2.37]

Objective adherence rate (MEMSCaps) 66 85.24 (3.38) 86.59 (3.82) –1.35 (5.11) – .792 [–11.56‐8.86]

Δ Symptom severity (MDASI) 58 0.37 (0.17) 0.17 (0.20) 0.20 (0.27) – .447 [–0.33‐0.73]

Δ Symptom interference (MDASI) 57 –0.03 (0.29) 0.09 (0.34) –0.11 (0.44) – .797 [–0.10‐0.77]

Δ Quality of life (FACT‐G) 67

–3.90 (1.68) 1.93 (1.93) –5.84 (2.57) – .027b [–10.98 to –

0.70]



–2.65 (0.84) 0.22 (0.95) –2.87 (1.28) – .028b [–5.42 to –

0.32]

Δ Emotional well‐being 68 –0.33 (0.52) 0.35 (0.59) –0.68 (0.79) – .391 [–2.26‐0.90]



59

Figure 6. Differences in the Change in Overall Quality of Life Between Study Groups Moderated by Agea


60

Table 15. Differences in Primary Outcomes by Study Group Using Multiple Imputation (Pooled Results From 10 Data Sets)a



Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 181 22.4/90 (24.9%) 17.3/91 (19.0%) – 0.70 (0.45) .431 [0.29‐1.70]

Objective adherence rate (MEMSCaps) 181 78.40 (2.95) 78.78 (3.14) –0.38 (4.23) – .929 [–8.68‐7.92]


Δ Symptom interference (MDASI) 181 –0.08 (0.21) –0.10 (0.23) 0.02 (0.31) – .947 [–0.59‐0.63]

Δ Quality of life (FACT‐G) 181 –2.01 (1.15) –0.15 (1.21) –1.86 (1.64) – .258 [–5.08‐1.36]



–2.30 (0.50) –0.61 (0.51) –1.70 (0.72) –

.019b [–3.12 to –

0.28]

Δ Emotional well‐being 181 0.48 (0.39) –0.13 (0.40) 0.61 (0.57) – .285 [–0.51‐1.73]



61

Table 16. Differences in Secondary Outcomes by Study Group Using Multiple Imputation (Pooled Results From 10 Data Sets)a

Secondary Study Outcome N Mean (SE) or N (%)


Difference (SE) P Value 95% CI

Satisfaction with treatment (FACIT‐TS‐PS)

Δ Clinician explanations 181 –0.34 (0.19) 0.06 (0.20) –0.40 (0.28) .152 [–0.95‐0.15]

Δ Interpersonal treatment 181 –0.28 (0.13) 0.10 (0.14) –0.38 (0.18) .037a [–0.74‐0.02]

Δ Comprehensive care 181 –0.91 (0.50) 0.24 (0.56) –1.16 (0.74) .120 [–2.61‐0.30]

Δ Nursing communication 181 –0.25 (0.19) –0.46 (0.21) 0.21 (0.27) .444 [–0.32‐0.74]

Δ Trust and confidence in clinicians 181 –0.25 (0.12) –0.24 (0.13) –0.01 (0.18) .962 [–0.36‐0.34]

Emergency department visits (RUQ) 181 0.13 (0.04) 0.17 (0.05) –0.04 (0.06) .491 [–0.16‐0.08]

Hospitalizations (RUQ) 181 0.16 (0.07) 0.21 (0.07) –0.05 (0.10) .634 [–0.24‐0.15]

Abbreviations: Δ, change from baseline to postassessment; CI, confidence interval; FACIT‐TS‐PS, Functional Assessment of Chronic Illness Therapy–Treatment Satisfaction–Patient Satisfaction; RUQ, Resource Utilization Questionnaire. a All analyses are controlling for change in perceived social support (Multidimensional Scale of Perceived Social Support).

62

Table 17. Linear Regression With Multiple Imputation Examining Baseline Self‐reported Adherence as a Moderator of the Effect of the Mobile App Intervention on the Objective Adherence Rate per MEMSCaps (Pooled N = 181)


Predictor Unstandardized

B SE [95% CI] t p

Study group (mobile app) –5.89 4.67 [–15.04‐3.25] –1.26 .207 Δ Perceived social support (MSPSS) –1.21 1.93 [–5.07‐2.65] –0.63 .533 Baseline self‐reported poor adherence (MMAS‐4) –19.29 6.73 [–32.50 to –6.09] –2.87 .004 Interaction (group X baseline self‐reported MMAS‐4) 26.53 10.53 [5.83‐47.24] 2.52 .012b

Effect of the mobile app on objective adherence rate in patients with poor baseline adherence (per MMAS‐4): B = –20.63; SE = 10.04; p = .040; 95% CI [–40.41 to –0.94]

Abbreviations: CI, confidence interval; MEMSCaps, Medication Event Monitoring System Caps; MMAS‐4, 4‐item Morisky Medication Adherence Scale; MSPSS, Multidimensional Scale of Perceived Social Support; SE, standard error. a p < .05.

63

Table 18. Differences in Primary Outcomes by Study Group (Using Multiple Imputation) in Patients With Self‐reported Poor Adherence at the Baseline Assessment on the Morisky Medication Adherence Scalea

Primary Study Outcome Pooled

N Mean (SE) or N (%)


Difference (SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 40.2 9.6/22

(43.6%) 9.4/18.2 (51.6%)

– 1.38 (0.73) .657 [0.33‐5.77]

Objective adherence rate (MEMSCaps) 40.2

63.79 (6.40) 84.47 (8.07) –20.63 (10.04) – .040b [–40.41 to –

0.94]

Δ Symptom severity (MDASI) 40.2 0.002 (0.29) 0.04 (0.32) –0.04 (0.42) – .931 [–0.85‐0.78]

Δ Symptom interference (MDASI) 40.2 0.03 (0.43) 0.24 (0.54) –0.21 (0.68) – .754 [–1.55‐1.13]

Δ Quality of life (FACT‐G) 40.2 –2.96 (2.43) 0.35 (2.99) –3.31 (3.92) – .399 [–11.02‐4.40]

Δ Physical well‐being 40.2 –1.02 (0.97) 0.84 (1.24) –1.86 (1.55) – .230 [–4.90‐1.18]

Δ Social/family well‐being 40.2 –1.88 (0.91) –0.67 (1.25) –1.21 (1.50) – .419 [–4.17‐1.74]

Δ Emotional well‐being 40.2 0.04 (0.77) –0.36 (0.93) 0.40 (1.24) – .749 [–2.05‐2.84]

Δ Functional well‐being 40.2 –0.43 (0.80) 0.29 (1.02) –0.73 (1.27) – .568 [–3.23‐1.77]


64

Table 19. Linear Regression With Multiple Imputation Examining Baseline Anxiety as a Moderator of the Effect of the Mobile App Intervention on the Objective Adherence Rate per MEMSCaps (Pooled N = 181)



B SE [95% CI] t p

Study group (mobile app) –4.41 4.94 [–14.11‐5.29] –0.89 .373 Δ Perceived social support (MSPSS) –0.81 1.91 [–4.61‐2.99] –0.43 .672 High anxiety (HADS‐anxiety)

–13.51 6.44 [–26.12 to –

0.89] –2.10 .036

Interaction (group X baseline HADS‐anxiety) 17.11 9.35 [–1.24‐35.45] 1.83 .068

Effect of the mobile app on the objective adherence rate in patients with high baseline anxiety (per HADS‐anxiety): B = –14.47; SE = 8.11; p = .074; 95% CI [–30.36, 1.43]

Abbreviations: Δ, change from baseline to postassessment; CI, confidence interval; HADS, Hospital Anxiety and Depression Scale; MEMSCaps, Medication Event Monitoring System Caps; MSPSS, Multidimensional Scale of Perceived Social Support; SE, standard error. *p < .05.

65

Table 20. Differences in Primary Outcomes by Study Group (Using Multiple Imputation) in Patients With Self‐reported High Anxiety on the Baseline Assessment per the Hospital Anxiety and Depression Scale‐Anxiety Subscale (HADS‐Anxiety)a




Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 48.1

8/25 (32.0%) 2/23.1

(8.7%) – 0.11 (1.40) .110 [0.007‐1.67]

Objective adherence rate (MEMSCaps) 48.1 67.78 (5.51) 82.25 (5.62) –14.47 (8.11) – .074 [–30.36‐1.43]

Δ Symptom severity (MDASI) 48.1 0.04 (0.31) –0.25 (0.33) 0.29 (0.46) – .522 [–0.60‐1.19]

Δ Symptom interference (MDASI) 48.1 0.15 (0.47) –0.67 (0.52) 0.82 (0.71) – .250 [–0.58‐2.22]

Δ Quality of life (FACT‐G) 48.1 –4.67 (2.37) 1.81 (2.61) –6.48 (3.59) – .071 [–13.52‐0.57]

Δ Physical well‐being 48.1 –0.30 (0.83) 1.82 (0.90) –2.12 (1.26) – .092 [–4.58‐0.34]

Δ Social/family well‐being 48.1 –4.10 (1.08) –1.61 (1.18) –2.49 (1.63) – .128 [–5.69‐0.72]

Δ Emotional well‐being 48.1 0.60 (0.82) 0.47 (0.89) 0.13 (1.24) – .916 [–2.30‐2.56]

Δ Functional well‐being 48.1 –0.86 (0.85) 0.97 (0.94) –1.83 (1.28) – .153 [–4.34‐0.68]


66

Table 21. Linear Regression With Multiple Imputation Examining Patient Age as Moderator of the Effect of the Mobile App Intervention on Change in Quality of Life on the Functional Assessment of Cancer Therapy‐General (Pooled N = 181)

Δ QOL (FACT‐G)


B SE [95% CI] t p

Study group (mobile app) –11.46 6.97 [–25.14‐2.23] –1.64 .101 Δ Perceived social support (MSPSS) 1.31 0.61 [0.11‐2.52] 2.14 .032 Patent age –0.16 0.10 [–0.35‐0.03] –1.66 .098 Interaction (group X patient age) 0.25 0.13 [–0.003‐0.50] 1.94 .053

Effect of the mobile app on the adherence rate in older patients (> 55 years old) based on Johnson‐Neyman Technique: B = , SE = , p = , [95% CI: ]

Abbreviations: CI, confidence interval; FACT‐G, Functional Assessment of Cancer Therapy‐General; MSPSS, Multidimensional Scale of Perceived Social Support; SE, standard error. a p < .05.

67

Table 22. Differences in Primary Outcomes by Study Group (With Multiple Imputation) in Patients > 55 Years of Agea




Mean Difference

(SE)

Odds Ratio (SE)

P Value

95% CI

Self‐report poor adherence (MMAS‐4) 77

9.8/39 (25.1%) 8.5/38 (22.4%)

– 0.70 (0.67)

.591 [0.19‐2.58]

Objective adherence rate (MEMSCaps) 77 83.99 (4.20) 80.38 (4.34) 3.60 (6.02) – .550 [–8.20‐15.40]


Δ Symptom interference (MDASI) 77 0.02 (0.28) 0.03 (0.32) –0.004 (0.43) – .992 [–0.86‐0.85]




–2.75 (0.84) –0.02 (0.91) –2.74 (1.22) – .025b [–5.13 to –

0.34]

Δ Emotional well‐being 77 –0.39 (0.54) 0.07 (0.57) –0.46 (0.79) – .566 [–2.01‐1.10]



68

G. DISCUSSION

In this study, we developed an acceptable, patient‐centered mobile app for adherence

and symptom management for patients with diverse malignancies who were prescribed oral

chemotherapy. However, patients assigned to the intervention group did not meet the a priori

feasibility criterion for completing the weekly reports of adherence and symptoms. Moreover,

the mobile app did not lead to significant improvements in the primary and secondary

outcomes of adherence per MEMSCaps, symptoms, overall QOL, perceptions of quality of care,

and health care utilization as hypothesized. Patients in the mobile app intervention reported a

smaller reduction in social and family well‐being over the course of the study than did those in

the usual care control condition. It is possible that the app relieved some of the burden that

caregivers generally experience in the context of home‐based care, and this may have resulted

in improvements in QOL of the patient in the social domain. Furthermore, patients who are

struggling with medication adherence or have elevated anxiety may benefit from such an app

to improve oral chemotherapy adherence, which may, in turn, improve therapeutic efficacy

and influence treatment outcomes. Finally, older patients may find this mobile app helpful for

their overall QOL, potentially by connecting them with resources for managing symptoms and

by providing education about their illness and resources for improving health (eg, recipes,

activity tracking).

Decisional Context

The study results underscore the importance of clinicians proactively assessing

adherence to treatment in the modern era of oral cancer therapeutics—22.0% of patients

reported difficulties taking these medications at baseline. However, only 13.8% of patients in

the mobile app group reported adherence problems at postassessment, while 23.3% of

patients in the standard care group continued to report difficulties. Although not a statistically

significant change, proactive and systematic monitoring of adherence and symptoms through

mobile technologies not only emphasizes the value and importance of adherence for patients

but also may serve as an extra layer of support for the care team and patients to communicate

effectively about the administration of oral chemotherapy and management of symptoms.

69

Treatment adherence is significant for public health and is a challenge for the health

care system, which aims to optimize treatment outcomes. Health care decision makers may

find these study results beneficial in that they suggest the potential for improving treatment

outcomes for specific populations of patients who may be at greater risk. A mobile app

provides a minimally burdensome, cost‐effective, low‐resource approach for patients who are

receiving care outside of the hospital or infusion center. Such an approach could be offered to

patients who endorse difficulties taking oral chemotherapy as instructed, have anxiety

symptoms, or are older. The mobile app intervention reduces variation in practice by

administering validated instruments to assess adherence and symptoms in a systematic

manner. Moreover, using this intervention to target patients at greater risk for adherence

problems would ideally reduce variation in treatment outcomes across patient

subpopulations.

The study results in context

The current study results highlight the potential for an adherence intervention to

promote medication taking in patients at greater risk for nonadherence. However, we did not

observe a significant benefit of the mobile app intervention in improving the outcomes of

adherence, symptoms, QOL, or perceptions of quality of care overall between the 2 study

groups. Several factors may have contributed to the null findings: (1) most patients had high

self‐reported adherence at baseline and therefore had little or no room for improvement; (2)

the app included multiple features to enhance patient engagement, which perhaps diffused its

target focus on adherence and symptoms; (3) the sample was quite heterogenous with respect

to cancer types, stages, and oral treatments; (4) the use of MEMSCaps to assess adherence to

oral chemotherapy regimens that have multiple intermittent breaks between cycles is

challenging; and (5) oncology clinicians were not required to follow up with patients regarding

their weekly adherence and symptom reports but rather could respond based on their clinical

judgment. Further rigorous qualitative study with the intervention patients who participated in

this study and their oncology clinicians would help elucidate the possible reasons the mobile

app did not have its intended benefits on the outcomes.

70

In our recently published systematic review, we identified only 12 adherence

intervention studies for patients with cancer, and most of them had a high risk of bias due to

methodological limitations such as a small sample size or nonrandomized designs.18 To

overcome these prior limitations, we implemented an adequately powered, randomized trial

with 181 patients diagnosed with diverse malignancies. Furthermore, we employed a more

robust measure of adherence, including both objective monitoring with MEMSCaps and self‐

reported adherence, methods utilized in only 2 previous adherence intervention studies.41,63

Finally, in our clinical trial, we examined clinically meaningful outcomes relevant to the patient

experience, such as quality of life, symptoms and side effects, and satisfaction with

treatment18,64 in addition to adherence.

Studies to date have not targeted multiple adherence factors at the patient, provider,

and systems levels, with the few intervention trials mostly focused on reminder systems. For

example, a randomized 3‐group pilot study by Spoelstra and colleagues42 showed no

differences in adherence rates following an Automated Voice Response (AVR) system alone

compared with AVR combined with adherence management or with AVR combined with

adherence and symptom management. In another randomized 3‐arm trial, investigators did

find differences in adherence when effects from 2 patient information program interventions

were pooled in comparison with the control group.65 Otherwise, the few intervention studies

with improved outcomes for adherence were nonrandomized. Specifically, in one

nonrandomized study of patients with advanced non–small cell lung cancer who were

prescribed oral chemotherapy, participants in the treatment monitoring program had higher

rates of adherence as recorded by pill count and self‐report than did a retrospective standard

care control group.66 In addition, a nonrandomized study involving intensified multidisciplinary

pharmaceutical care showed that patients with colorectal and breast cancer had higher daily

adherence rates than did a standard care comparison group.41 Within this context, the

advances of our clinical trial are reflected not only in the randomized design, sample size, and

selection of outcomes, but also the intervention components. That is, we designed the mobile

app to target patient factors (ie, reminder system, education library, energy tracking),

71

treatment factors (ie, symptom monitoring and management strategies), and clinician factors

(ie, proactive communication with cancer care clinicians).

With respect to mobile health (mHealth) interventions, most studies have focused on

management of long‐term conditions such as diabetes, HIV, and asthma. For example, findings

from a recent meta‐analysis revealed improvements in treatment adherence following mobile

text messaging for patients with chronic illness, but this review did not include studies with

oncology patients and is therefore limited in generalizability.67 The authors of another meta‐

analysis of mHealth interventions concluded that certain mobile phone messaging

interventions may improve the self‐management of long‐term illness; however, significant

gaps exist in this work, requiring further research.68 While we observed no intervention effects

overall in our study sample, our findings extend the growing literature suggesting that an

adherence intervention delivered through mobile modalities may be beneficial for disease self‐

management among patients with cancer who are at greater risk for nonadherence. However,

prospective follow‐up study is needed to confirm that the app is indeed effective for those

with poor baseline adherence and higher anxiety prior to broader dissemination and

implementation of the intervention.

Implementation of study results

The key to successful implementation of the mobile app intervention in this study was

incorporating the voices of patients, family members, clinicians, cancer practice

administrators, and health care representatives throughout every phase of development and

testing. While stakeholder engagement is common in earlier stages of research, we

incorporated stakeholders throughout the research process, including dissemination and

implementation, which occurs less frequently.69 For example, our final stakeholder focus

groups proved to be instrumental in brainstorming the next steps for this study and how to

disseminate results to audiences outside the scientific community. Moreover, the stakeholders

were enthusiastic and willing to participate, and many did not want reimbursement but rather

sought to help make the research more meaningful, relevant, and feasible in real‐world care

settings. The positive experiences with our stakeholder engagement in this study informed the

development of a Patient/Family Advisory Council specifically for supportive care research at

72

the MGH Cancer Center, which meets 3 to 4 times per year. Investigators from our Cancer

Outcomes Research Program now present studies to and gather feedback from the council

about clinical relevance, whether interventions are timed effectively, and how to optimize

delivery to patients, families, and the care team.

As an example of how stakeholder feedback enhanced acceptability and

implementation of the mobile app intervention by MGH oncology clinicians, we drew on

qualitative feedback from clinician stakeholders. Specifically, we learned that the optimal

frequency for the care team to receive patient reports of adherence and symptoms would be

no more than weekly. To translate this intervention successfully in typical care settings, such

patient symptom reports would ideally be integrated seamlessly with the electronic health

record for ongoing tracking and documentation. Unfortunately, we were unable to deliver the

reports in this manner as our institution was in the process of converting to a new EHR system

at the time of study implementation. Regardless, it is also important to note that we did not

achieve our a priori feasibility threshold of most participants in the intervention group

completing 75% of the weekly adherence and symptom reports during the study period.

Perhaps the proposed feasibility criterion was not the most appropriate measure for how

patients engaged with the mobile app, as some patients may have chosen not to complete the

adherence and symptom reports on weeks they were feeling well or had nothing to endorse.

Follow‐up qualitative study is needed with patients assigned to the intervention, to discern the

optimal frequency of communication with the cancer care team and whether reports should

be recommended at certain time intervals, in tandem with clinically meaningful triggers, or

whether the reports should be delivered primarily at the patient’s discretion.

Another primary concern for implementation across care settings relates to

maintaining the mobile app functionality over time. Specifically, because mobile devices and

operating systems are constantly evolving and continuously upgraded, the application

software must also be adjusted and maintained. During our trial, we occasionally needed to

loan backup tablet devices to participants when we encountered glitches in the system due to

smartphone upgrades, for example.

73

A final key barrier to implementation of the intervention would be the extent to which

the patient populations, or those at risk for poor medication adherence, own smartphones

since the intervention was specifically designed for use on mobile technology. Given that many

people keep their smartphone on their person most of the day, this technology represents an

ideal modality, particularly for real‐time reporting of symptoms with responsive logic to teach

behavioral strategies for management. Although not all patients in the oncology setting have

access to smartphones, ownership continues to grow at a rapid pace across populations,

including older patients and those of lower socioeconomic status. Within our study, of the 500

patients approached to assess for eligibility, 322 (64.4%) had access to a smartphone.

Generalizability

As noted in the Results section, the patient sample was predominantly white and

married (or partnered), approximately equally distributed across genders, and very well

educated, with a wide range in age and representation of hematologic and solid tumor cancer

types. The study findings would therefore likely generalize to similar patients who seek care at

an urban comprehensive cancer center like Massachusetts General Hospital. Further study is

needed to test the efficacy of the intervention in larger, more racially and ethnically diverse

samples in both community and rural cancer care settings. Moreover, the extent to which

patients’ high education levels may have selected for a more knowledgeable and motivated

sample, and perhaps contributed to the high adherence rates and null findings overall,

requires further research.

Subpopulation Considerations

Although we must interpret subgroup analyses with caution given the reduction in

sample sizes, the mobile app intervention appeared to be more efficacious for patients with

certain risk factors. Specifically, based on prior theory and evidence, we examined particular

subgroups likely to have adherence problems, such as patients who reported poor adherence

or heightened anxiety at baseline. Analyses of these subgroups revealed that those assigned to

the mobile app intervention had significantly higher objective adherence estimates (per the

MEMSCaps) over the study period than did those who received standard care alone. Such

findings are theoretically consistent and would be meaningful to consider for potential

74

translation into patient care. We also observed that older patients who received the mobile

app intervention reported significantly higher quality of life (per the FACT‐G) than did older

patients in the standard care group. These findings certainly warrant further investigation for

confirmation prior to broader dissemination and implementation of the intervention.

Study Limitations

The study has several limitations that may have influenced the results. First, an

unfortunate clerical error resulted in loss of data on the MDASI, one of the main study

outcomes. However, analyses with imputed data on the full sample did not reveal a different

pattern of findings from the available case analyses with respect to intervention effects on

symptom severity or interference. In addition, although we used the current “gold standard”

for measuring medication adherence with the MEMSCaps as the primary outcome, such

monitoring in the control group likely raised awareness and improved adherence,70 potentially

diluting the effect of the intervention. Moreover, using MEMSCaps to monitor medication

adherence for patients with interval dosing schedules (eg, 2 weeks on, 1 week off) was

challenging, especially in defining critical periods for when the patient was supposed to be

taking the medication. The study team therefore had to compare data from the MEMSCaps

against EHR documentation of planned breaks in the medications to ensure patients were not

penalized for missing doses those days. Again, any error in adherence measurement would

likely bias against intervention effects. Finally, the study took place at an academic institution

with a fairly homogenous patient population with respect to race, ethnicity, level of education,

and socio‐economic status, which may limit generalizability of findings to other care settings

and populations.

Future Research

Follow‐up research is needed to test the benefit of the mobile app in populations at

high risk for poor adherence as well as across both academic and community oncology care

settings. Ideally, future studies should sample patients who are poorly adherent at baseline,

therefore minimizing type 2 error. A hybrid efficacy‐effectiveness study would be a useful

design for further intervention testing and implementation. In addition, to augment the utility

of the intervention, investigators may want to examine whether integrating patient‐reported

75

data from the mobile app into the EHR helps enhance communication with the care team

versus employing a primary triage clinician (eg, oncology nurse) to review and respond to the

patient reports versus having a completely stand‐alone app that records and stores data

natively on the smartphone, which patients can choose to share with clinicians at their

discretion. Moreover, a follow‐up study could explore how oncology clinicians utilized the

weekly adherence and symptom reports to inform care in their patient encounters as well as

patient perceptions of why the app did not affect the primary and secondary outcomes.

Expanding the mobile app to help patients monitor and manage multiple medications

simultaneously may also enhance its usefulness. Finally, in future studies, the inclusion of

multiple longitudinal assessments of adherence and symptoms would be needed to discern

the impact of the intervention over longer follow‐up periods, especially given prior research

showing that medication adherence tends to wane over time.18

H. CONCLUSIONS

To our knowledge, this study represents the first examination of the development and

testing of a mobile application to improve adherence to oral chemotherapy. With critical

feedback from key constituent stakeholders throughout every phase of the project, we first

successfully created a patient‐centered mobile application, incorporating features to support

adherence, symptom management, and communication with the oncology care team. We

then conducted a randomized clinical trial to test the benefits of the mobile app versus

standard care for improving symptoms and adherence to oral chemotherapy, in a sample of

181 patients with diverse malignancies. Although the mobile app did not have significant

effects on the primary and secondary outcomes in the entire sample overall, subgroup

analyses demonstrated that the intervention shows promise for patients who may be at risk

for poor adherence, such as those who report having problems with medication adherence or

anxiety. In addition, the mobile app may positively impact quality of life among older patients.

Further work is needed to confirm the effectiveness of the intervention in these

subpopulations across oncology care settings and to explore the utility of the mobile app in

sustaining optimal adherence over longer periods of time. A key factor to ensure successful

76

dissemination and implementation of the mobile app will be the seamless integration of

patient‐reported data with existing electronic health record systems. As cancer care continues

to evolve with orally administered agents, the innovative use of technology through this

mobile app may foster communication with the care team and serve as an extra layer of

support for patients to understand and adhere to their recommended treatments.

77

I. REFERENCES

1. Neuss MN, Polovich M, McNiff K, et al. 2013 updated American Society of Clinical Oncology/Oncology Nursing Society chemotherapy administration safety standards including standards for the safe administration and management of oral chemotherapy. Oncol Nurs Forum. 2013;40(3):225‐233.

2. Liu G, Franssen E, Fitch MI, Warner E. Patient preferences for oral versus intravenous palliative chemotherapy. J Clin Oncol. 1997;15(1):110‐115.

3. Borner MM, Schoffski P, de Wit R, et al. Patient preference and pharmacokinetics of oral modulated UFT versus intravenous fluorouracil and leucovorin: a randomised crossover trial in advanced colorectal cancer. Eur J Cancer. 2002;38(3):349‐358.

4. Banna GL, Collovà E, Gebbia V, et al. Anticancer oral therapy: emerging related issues. Cancer Treat Rev. 2010;36(8):595‐605.

5. Given BA, Spoelstra SL, Grant M. The challenges of oral agents as antineoplastic treatments. Semin Oncol Nurs. 2011;27(2):93‐103.

6. Barton D. Oral agents in cancer treatment: the context for adherence. Semin Oncol Nurs. 2011;27(2):104‐115.

7. Aisner J. Overview of the changing paradigm in cancer treatment: oral chemotherapy. Am J Health Syst Pharm. 2007;64(9)(suppl 5):S4‐S7.

8. Hartmann JT, Haap M, Kopp HG, Lipp HP. Tyrosine kinase inhibitors—a review on pharmacology, metabolism and side effects. Curr Drug Metab. 2009;10(5):470‐481.

9. Bedell CH. A changing paradigm for cancer treatment: the advent of new oral chemotherapy agents. Clin J Oncol Nurs. 2003;7(suppl 6):5‐9.

10. Al‐Barrak J, Cheung WY. Adherence to imatinib therapy in gastrointestinal stromal tumors and chronic myeloid leukemia. Support Care Cancer. 2013;21(8):2351‐2357.

11. Makubate B, Donnan PT, Dewar JA, Thompson AM, McCowan C. Cohort study of adherence to adjuvant endocrine therapy, breast cancer recurrence and mortality. Br J Cancer. 2013;108(7):1515‐1524.

12. Wu EQ, Johnson S, Beaulieu N, et al. Healthcare resource utilization and costs associated with non‐adherence to imatinib treatment in chronic myeloid leukemia patients. Curr Med Res Opin. 2010;26(1):61‐69.

13. Ganesan P, Sagar TG, Dubashi B, et al. Nonadherence to imatinib adversely affects event free survival in chronic phase chronic myeloid leukemia. Am J Hematol. 2011;86(6):471‐474.

78

14. Yoshida C, Komeno T, Hori M, et al. Adherence to the standard dose of imatinib, rather than dose adjustment based on its plasma concentration, is critical to achieve a deep molecular response in patients with chronic myeloid leukemia. Int J Hematol. 2011;93(5):618‐623.

15. Ruddy K, Mayer E, Partridge A. Patient adherence and persistence with oral anticancer treatment. CA Cancer J Clin. 2009;59(1):56‐66.

16. Partridge AH, Avorn J, Wang PS, Winer EP. Adherence to therapy with oral antineoplastic agents. J Natl Cancer Inst. 2002;94(9):652‐661.

17. Escalada P, Griffiths P. Do people with cancer comply with oral chemotherapy treatments? Br J Community Nurs. 2006;11(12):532‐536.

18. Greer JA, Amoyal N, Nisotel L, et al. A systematic review of adherence to oral antineoplastic therapies. Oncologist. 2016;21(3):354‐376.

19. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487‐497.

20. Verbrugghe M, Verhaeghe S, Lauwaert K, Beeckman D, Van Hecke A. Determinants and associated factors influencing medication adherence and persistence to oral anticancer drugs: a systematic review. Cancer Treat Rev. 2013;39(6):610‐621.

21. Balkrishnan R. Predictors of medication adherence in the elderly. Clin Ther. 1998;20(4):764‐771.

22. Vik SA, Maxwell CJ, Hogan DB. Measurement, correlates, and health outcomes of medication adherence among seniors. Ann Pharmacother. 2004;38(2):303‐312.

23. Jacobs JM, Pensak NA, Sporn NJ, et al. Treatment satisfaction and adherence to oral chemotherapy in patients with cancer. J Oncol Pract. 2017: 13(5):e474‐e485.

24. Noens L, van Lierde MA, De Bock R, et al. Prevalence, determinants, and outcomes of nonadherence to imatinib therapy in patients with chronic myeloid leukemia: the ADAGIO study. Blood. 2009;113(22):5401‐5411.

25. Partridge AH, Wang PS, Winer EP, Avorn J. Nonadherence to adjuvant tamoxifen therapy in women with primary breast cancer. J Clin Oncol. 2003;21(4):602‐606.

26. Lebovits AH, Strain JJ, Schleifer SJ, Tanaka JS, Bhardwaj S, Messe MR. Patient noncompliance with self‐administered chemotherapy. Cancer. 1990;65(1):17‐22.

27. Richardson JL, Marks G, Johnson CA, et al. Path model of multidimensional compliance with cancer therapy. Health Psychol. 1987;6(3):183‐207.

28. Lash TL, Fox MP, Westrup JL, Fink AK, Silliman RA. Adherence to tamoxifen over the five‐year course. Breast Cancer Res Treat. 2006;99(2):215‐220.

29. Lee CR, Nicholson PW, Souhami RL, Deshmukh AA. Patient compliance with oral chemotherapy as assessed by a novel electronic technique. J Clin Oncol. 1992;10(6):1007‐1013.

79

30. Levine AM, Richardson JL, Marks G, et al. Compliance with oral drug therapy in patients with hematologic malignancy. J Clin Oncol. 1987;5(9):1469‐1476.

31. Winterhalder R, Hoesli P, Delmore G, et al. Self‐reported compliance with capecitabine: findings from a prospective cohort analysis. Oncology. 2011;80(1‐2):29‐33.

32. Eliasson L, Clifford S, Barber N, Marin D. Exploring chronic myeloid leukemia patients' reasons for not adhering to the oral anticancer drug imatinib as prescribed. Leuk Res. 2011;35(5):626‐630.

33. Goodwin JS, Zhang DD, Ostir GV. Effect of depression on diagnosis, treatment, and survival of older women with breast cancer. J Am Geriatr Soc. 2004;52(1):106‐111.

34. Ayres A, Hoon PW, Franzoni JB, Matheny KB, Cotanch PH, Takayanagi S. Influence of mood and adjustment to cancer on compliance with chemotherapy among breast cancer patients. J Psychosom Res. 1994;38(5):393‐402.

35. Greer JA, Pirl WF, Park ER, Lynch TJ, Temel JS. Behavioral and psychological predictors of chemotherapy adherence in patients with advanced non‐small cell lung cancer. J Psychosom Res. 2008;65(6):549‐552.

36. Pirl WF. Evidence report on the occurrence, assessment, and treatment of depression in cancer patients. J Natl Cancer Inst Monogr. 2004(32):32‐39.

37. Walker J, Holm Hansen C, Martin P, et al. Prevalence of depression in adults with cancer: a systematic review. Ann Oncol. 2013;24(4):895‐900.

38. Wood L. A review on adherence management in patients on oral cancer therapies. Eur J Oncol Nurs. 2012;16(4):432‐438.

39. Quality Oncology Practice Initative. American Society of Clinical Oncology. http://www.instituteforquality.org/. Published 2016. Accessed May 25, 2017.

40. Schneider SM, Hess K, Gosselin T. Interventions to promote adherence with oral agents. Semin Oncol Nurs. 2011;27(2):133‐141.

41. Simons S, Ringsdorf S, Braun M, et al. Enhancing adherence to capecitabine chemotherapy by means of multidisciplinary pharmaceutical care. Support Care Cancer. 2011;19(7):1009‐1018.

42. Spoelstra SL, Given BA, Given CW, et al. An intervention to improve adherence and management of symptoms for patients prescribed oral chemotherapy agents: an exploratory study. Cancer Nurs. 2013;36(1):18‐28.

43. Steinhubl SR, Muse ED, Topol EJ. Can mobile health technologies transform health care? JAMA. 2013;310(22):2395‐2396.

44. Darlow S, Wen KY. Development testing of mobile health interventions for cancer patient self‐management: a review. Health Informatics J. 2016;22(3):633‐650.

80

45. Concannon TW, Meissner P, Grunbaum JA, et al. A new taxonomy for stakeholder engagement in patient‐centered outcomes research. J Gen Intern Med. 2012;27(8):985‐991.

46. Meurer LN. MCW population‐based model for patient‐centered care. http://www.mcw.edu/FileLibrary/User/facdev/PopulationHealthModelworksheetforTeamswebsite.pdf. Published 2008. Accessed May 9, 2017.

47. Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: examples from New Zealand. J Health Commun. 2012;17(suppl 1):11‐21.

48. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5(6):649‐655.

49. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377‐381.

50. Stanton AL, Petrie KJ, Partridge AH. Contributors to nonadherence and nonpersistence with endocrine therapy in breast cancer survivors recruited from an online research registry. Breast Cancer Res Treat. 2014;145(2):525‐534.

51. Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self‐reported measure of medication adherence. Med Care. 1986;24(1):67‐74.

52. Cleeland CS, Mendoza TR, Wang XS, et al. Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer. 2000;89(7):1634‐1646.

53. Cella DF, Tulsky DS, Gray G, et al. The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol. 1993;11(3):570‐579.

54. Peipert JD, Beaumont JL, Bode R, Cella D, Garcia SF, Hahn EA. Development and validation of the Functional Assessment of Chronic Illness Therapy Treatment Satisfaction (FACIT TS) measures. Qual Life Res. 2014;23(3):815‐824.

55. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361‐370.

56. Zimet GD, Dahlem NW, Zimet SG, et al. The Multidimensional Scale of Perceived Social

Support. J Pers Assess. 1988;52:30‐41.

57. Davis TC, Crouch MA, Long SW, et al. Rapid assessment of literacy levels of adult primary care patients. Fam Med. 1991;23(6):433‐435.

58. Brooke J. SUS‐A quick and dirty usability scale. Usability evaluation in industry. Vol 189 (194)1996:4‐7.

81

59. Bangor A, Kortrum P, Miller J. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24(6):574‐594.

60. Wang R, Ware JH. Detecting moderator effects using subgroup analyses. Prev Sci. 2013;14(2):111‐120.

61. Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis. New York, NY: Guilford Press; 2013.

62. Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York, NY: Wiley; 1987.

63. Krolop L, Ko YD, Schwindt PF, Schumacher C, Fimmers R, Jaehde U. Adherence management for patients with cancer taking capecitabine: a prospective two‐arm cohort study. BMJ Open. 2013;3(7).

64. Mathes T, Antoine SL, Pieper D, Eikermann M. Adherence enhancing interventions for oral anticancer agents: a systematic review. Cancer Treat Rev. 2014;40(1):102‐108.

65. Ziller V, Kyvernitakis I, Knöll D, Storch A, Hars O, Hadji P. Influence of a patient information program on adherence and persistence with an aromatase inhibitor in breast cancer treatment—the COMPAS study. BMC Cancer. 2013;13:407.

66. Gebbia V, Bellavia M, Banna GL, et al. Treatment monitoring program for implementation of adherence to second‐line erlotinib for advanced non‐small‐cell lung cancer. Clin Lung Cancer. 2013;14(4):390‐398.

67. Thakkar J, Kurup R, Laba TL, et al. Mobile telephone text messaging for medication adherence in chronic disease: a meta‐analysis. JAMA Intern Med. 2016;176(3):340‐349.

68. de Jongh T, Gurol‐Urganci I, Vodopivec‐Jamsek V, Car J, Atun R. Mobile phone messaging for facilitating self‐management of long‐term illnesses. Cochrane Database Syst Rev. 2012;12:CD007459. doi: 10.1002/14651858.CD007459.pub2

69. Concannon TW, Fuster M, Saunders T, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient‐centered outcomes research. J Gen Intern Med. 2014;29(12):1692‐1701.

70. Pagoto SL, McDermott MM, Reed G, et al. Can attention control conditions have detrimental effects on behavioral medicine randomized trials? Psychosom Med. 2013;75(2):137‐143.

82

J. PUBLICATION LIST

1. Fishbein JN, Nisotel LE, Macdonald JJ, et al. Mobile application to promote adherence to oral chemotherapy and symptom management: a protocol for design and development. JMIR Res Protoc. 2017;6(4).

2. Greer JA, Amoyal N, Nisotel L, et al. A systematic review of adherence to oral antineoplastic therapies. Oncologist. 2016;21(3):354‐376.

83

Appendix A: Phase 1 Semi‐structured Interview Guide for Pre‐Trial Focus Groups with Stakeholders Topic 1: Perceived importance of monitoring of adherence to oral chemotherapy

1) What are the potential problems and benefits of monitoring oral chemotherapy practices? 2) What do you believe are the patient, clinician, and healthcare system factors that impact

adherence to oral chemotherapy? 3) To what extent does the proposed study and intervention address those factors?

Topic 2: Barriers to communication between patients and the oncology team regarding management of side effects and medication adherence

1) What are the most important aspects of communication between patients and clinicians to ensure effective adherence to oral chemotherapy?

2) How might communication breakdown overtime between patients and the oncology team? 3) To what extent does the proposed study intervention address these communication barriers?

Topic 3: Potential role of the mobile application to address barriers to quality cancer care

1) What are your impressions of the three components of the mobile application to improve symptom monitoring and adherence to oral chemotherapy:

a. Creation of chemotherapy treatment plan b. Weekly self‐report surveys (via the app) of symptoms and medication adherence c. Immediate results feedback to patients and oncology team

2) Would you make any changes to any of these three components? In what ways could the app be improved to meet the needs of patients and clinicians?

3) What additional resources would be helpful for managing symptoms and medications at home? Topic 4: Feasibility, acceptability and utility of electronic intervention

1) When would be the ideal time to start the mobile app intervention during cancer care? 2) What types of problems, if any, do you foresee with the process of creating a chemotherapy

treatment plan? 3) To what extent does the results feedback seem personal, relevant, and helpful? 4) Under what circumstances do you think patients or clinicians would not want to use the app? 5) In what ways could we change or improve the app to help make it more user‐friendly and

acceptable to patients and clinicians? Topic 5: System barriers and facilitators to implementation

1) Given the components of the proposed mobile app, do you foresee any problems or barriers to implementing the intervention as we proposed?

2) Do you have any suggestions for changing the protocol to improve the study process and flow? 3) What recommendations do you have to ensure patient recruitment and retention?

Any final thoughts, comments or recommendations about the study that we have not yet discussed? This brings us to the end of the interview. We greatly appreciate your participation. Thank you.

84

Appendix B: Phase 1 Semi‐structured Interview Guide for MGH Patients & Oncology Clinicians Topic 1: Content of Mobile Application 1) What are your impressions of the three components of the mobile application to improve

symptom monitoring and adherence to oral chemotherapy: a. Creation of chemotherapy treatment plan b. Weekly self‐report surveys (via the app) of symptoms and medication adherence c. Immediate results feedback to patients and oncology team

2) Would you make any changes to any of these three components? In what ways could the app be improved to meet the needs of patients and clinicians?

3) Were there any parts of the app that didn’t make sense to you? 4) How likely would patients refer to educational links from organizations, like the National Cancer

Institute, or use suggestions from the mobile app for coping with symptoms and side effects? 5) What additional resources would be helpful for managing symptoms and medications at home?

Topic 2: Feasibility and Acceptability of the Mobile Application 6) When would be the ideal time to start the mobile app intervention during cancer care? 7) What types of problems, if any, do you foresee with the process of creating a chemotherapy

treatment plan? 8) To what extent does the results feedback seem personal, relevant, and helpful? 9) How comfortable would you feel using this mobile app (at home or in your practice)? 10) Under what circumstances do you think patients or clinicians would not want to use the app? 11) In what ways could we change or improve the app to help make it more user‐friendly and

acceptable to patients and clinicians?

Topic 3: Weekly Assessments 1) What are your thoughts about the app surveys of symptoms and medication adherence? 2) Are there any problems with the wording or was there anything you did no understand? 3) How often should patients complete the app surveys? 4) Are the surveys too long to complete on a weekly basis? 5) At what cutoff value for each scale would it make sense to notify the oncology team of the survey

results? 6) Should we add any other questions to the surveys to make the app more useful for patients and

clinicians?

Anything else you would like to add or suggest? This brings us to then end of the interview. Thank you very much for your time and participation.

85

Appendix C. Summary of Feedback from Phase 1 Interviews with MGH Patients, Clinicians,

and Stakeholders

Personalized Chemotherapy Plan/Medication Reminders

Identify patients who are taking the same oral chemotherapy/disease cohort and connect them

to serve a support system

Stratify patients by line of chemotherapy if variation exists in the patient populations

o Example: oral chemotherapy as a first line of therapy versus not first line or first time

taking oral chemotherapy versus treated with oral chemotherapy previously

Allow patients to create a window of time (i.e. 7am‐9am) during which they can take their

medication, rather than an exact time (on the hour)

Give patients the ability to alter the frequency of medication reminders (i.e. daily/weekly/x per

week)

Make sure that treatment plan is editable

Include patient/treatment information in treatment plan: primary clinician, NP, who to contact

on weekends, general contact info

Specify “since your last visit to the application...”

o Ask once every ~30 days – ask every time patient opens the app until they answer it

Symptom Management

One goal for this section should be to empower patients to report symptoms and hopefully

speak up more during clinic visits regarding symptom management

Establish a method of tracking patient phone calls to clinic regarding symptoms and compare

between study arms

List only the common/ “red flag” side effects

Use a slider scale for symptom support frequency and severity (0‐10 for each symptom)

Have the symptoms reorder according to endorsement

Give specific definitions and anchors for symptoms

Option for stable patients: “my symptoms have not changed over the last week/since my last

report”

Add sexual symptoms – sexual dysfunction

Notify clinician when a symptom is unacceptable

Daily Tips

Give patients the option to receive daily or weekly tips

Resources

Track patient hits on educational resources

Consider collaborating/partnering with a pharmaceutical company to provide relevant and

reliable information to patients – many companies have portals for patients and partners, as

well

86

Drive educational content based on common side effects of the patient’s specific disease/drug

Provide links to inspire.com for each disease message board(s)

Provide information for PFAC

Include link to website and location of Cancer Resource Room

Patient/Physician Communication

The study can serve as an opportunity to foster communication between the patient and the

care team (i.e. MD, PA, NP, etc...)

This could be an opportunity to measure patient satisfaction on an ongoing basis, as well as

satisfaction with care team communication

Provide patients with information on how to communicate with their care team effectively

Promote patient advocacy through this app

Capture patient‐physician communication as an outcome

Have a point person to triage patient messages via app – NP, RN, PA

Collect data on MDs who follow up on patient symptom reports and how (i.e. EMR)

Look at past information on how MDs act upon new or worsening symptoms

Make sure that communication is non‐judgmental in regards to feedback on adherence

Usability, Acceptability, and Feasibility

Ensure that this program is rewarding for patients

Collect baseline data on patient’s beliefs and expectations regarding oral chemotherapy, as

well as coping styles

Assess whether patients see oral chemotherapy as a quality of life treatment or burdensome

Inquire about patient’s unmet needs and gather them throughout the study

Provide patients with feedback on how their adherence is compared to other patients using the

mobile app

Examine subgroups/cohorts of patients who might benefit more than others – compare their

data – examine age/meds/age

It is important to recognize that patients’ symptoms can be debilitating so they might have a

hard time reporting some days

Pilot the actual application with patients and clinicians before the RCT

Assess how much the app has enhanced or burdened care for patients

o What proportion of patients in RCT liked or disliked the application – this section could

be called “Patient Engagement”

o Ask questions like “does reading about side effects make you more anxious?”

Incentivize patients – for every survey, give them $20

Find an alternative way to measure ED/Urgent Care visits because if patients do not go to MGH

ED/UC, then it will not show up in their electronic medical record

Flesh out “point person” for app in great depth – some barriers could arise if this is not

organized

87

o What will a patient do if they report a severe symptom on a Friday afternoon when no

one is there to receive the message?

o What is the point person is on vacation?

o Ensure that there is a clear disclaimer that the app does not replace when patient

should call their doctor regarding severe symptoms

Conduct clinician assessments on how acceptable the app is for them

o Clinicians might think it’s a great idea during a focus group, but once they actually have

a busy schedule, during the RCT, they could neglect it

o Clinicians who are not researchers might have a hard time accepting this

Stratify for different oral agents

Not necessary to exclude by line of chemotherapy, but worth collecting and noting/stratifying

Exclude patients who are enrolled in a clinical trial – clinical trial patients at MGH do not

receive “standard care”

Thus, exclude trial/experimental drugs

General Suggestions

Train Research Assistants and study staff in the app to give an instructional seminar for

participants using the app

Possibly include a training video in the app on how to use the app

Assess if patients are taking oral chemotherapy for a short period of time or indefinitely –

engage both of these populations and get feedback on how to make this app ongoing for both

populations

MD referrals are not ideal for RCT – query electronic medical records, then approach physicians

regarding eligible patients – develop a cross‐Cancer Center protocol for recruitment in various

disease groups

Involve clinicians throughout app development and RCT so that they are interested and

invested in the project, thus more likely to dedicate time to utilizing the patient/physician

features on the mobile app

Ensure that clinicians are addressing patient‐reported symptoms on the app during clinic

Make sure that patients are reminded to refill their medication ahead of time so they don’t run

out and miss doses because of this reason

Attempt to forward patient symptom reports to EMR

Gather information from pharmaceutical or insurance companies that might have their own

cell phone call system regarding medication adherence and instructions

Look into liability of symptom reporting – important for fever, neutropenia, etc...

Change exclusion criteria language from “owns a smartphone” to “uses a smartphone” – some

people will own an iPhone/Android, as well as an incompatible phone – could be confusing

Make certain parts of the app email/print friendly

Add a “notes” section for patients to store information regarding their next clinic visit, etc.

Best time to start app is at beginning of oral chemo prescription

88

Include a disclaimer that this app is not an emergency service app and that patients should call

911 if they feel their symptoms/situation is urgent

Provide patients with contact information if they are struggling with the app’s technical

features

Use the term “oncology clinician” rather than “doctor”

Utilize drop down menus when possible

89

Appendix D. Example Email Communication with Stakeholder Groups

Dear Stakeholders,

Thank you for providing feedback for Dr. Greer’s Oral Chemotherapy Mobile Application Study. Your

feedback is integral to the development and implementation of this study. We have consolidated your

feedback below and included plans of how we will incorporate your suggestions into this project.

1. We asked you…

What kinds of things could we highlight about the control arm to reassure those participants that their

participation is equally important, and to ensure that they stay motivated?

You suggested that we…

Emphasize the high‐tech characteristics of the pill bottles

Let participants know that in the future the app may be accessible to all patients

Be honest – the control group is important in order to see the effects of the mobile app

Schedule check‐ins with the control group participants to ensure that they feel appreciated

throughout their participation

What we have changed…

Thank you for your suggestions! We have begun to inform participants of the high‐tech aspects of the

pill bottles, and we now inform them that the app could eventually be available for all patients. We

emphasize the importance of the control group in research and communicate more frequently to

ensure that all control participants feel valued and appreciated.

2. We asked you...

We also want to encourage the oncology clinicians to be engaged with this study. When we send

clinicians these weekly reports, what kinds of messages would be helpful for getting the clinicians

motivated?

You suggested that we...

Include information about adherence in the weekly symptom reports

Include a brief report on symptoms that impact quality of life in between clinic visits

Set up a forum for clinicians to post comments about their experience receiving the reports

Disseminate information on how other clinicians respond to the reports

Ensure that the weekly symptom reports are simple, easy to interpret, and brief, with a way for

clinicians to provide feedback directly

90

What we have changed…

This is very helpful feedback! In the current weekly symptom report format, we disseminate

personalized information on the participants’ adherence and symptoms to clinicians. This is an effective

way for clinicians to be informed of patients’ symptoms in between clinic visits, particularly for

participants who come to clinic less frequently. We are currently restructuring our weekly symptom

report format to include easily interpretable and comprehensive graphics that will allow clinicians to

see what the participant endorses over a longer period of time, rather than only the current week’s

data.

We asked you...

Is there any one specific feature not currently in CORA that you think would greatly improve it if it were

added?

You suggested that we...

Create a competitive incentive by allowing participants to see how adherent they are

compared to other app users. This could work for both the Fitbit and mobile app.

Provide a section for participants to submit information about their emotional well‐being

Create a feature that allow participants to share comments on their experience with the app,

as well as tips on managing adherence and symptoms

Create a patient portal

Provide specific information about medication

These are excellent and thoughtful recommendations. In future iterations of the app, we hope to

create new features that engage the app users. Creating a method to allow participants to compare

their adherence with each other and share tips on their experience would be an excellent feature that

we will keep in mind for the future. Participants currently have the ability to record notes and

questions in the app and are encouraged to share this information with their clinician in clinic. This is a

great outlet for questions and comments about emotional well‐being. Participants are also given the

opportunity to submit information related to emotional well‐being in the adhoc and weekly symptom

reports. Additionally, the app includes an extensive education library with information on medication,

adherence, symptoms and side effects, as well as emotional well‐being and mood symptoms.

Thank you for providing us with your ideas! Your ongoing feedback is essential to the success of this

study and we are grateful for your support. Please do not hesitate to reach out with any additional

suggestions and feedback that come up in between Stakeholder newsletters.

Sincerely,

Joseph Greer and the Study Team

91

Appendix E. Mobile App Features

1. Treatment Plan: As part of routine clinical care, participants had an initial consultation with

their oncology clinician (i.e., oncologist or nurse practitioner) to review their personalized

chemotherapy treatment plans. These treatment plans were uploaded into the mobile

app so that patients had access to them throughout the study period. The chemotherapy

treatment plan included the patient’s medication name, dosage, administration schedule,

break schedule, and prompts for medication reminders.

2. Medication adherence: Within the app, participants could set up daily alerts to take their

medication. In addition, they were asked to complete a weekly, two‐item questionnaire

assessing how well they took their oral chemotherapy medication in the last week.

Specifically, these questions asked: 1) what percent of the time did you take your

prescribed oral chemotherapy medication(s)? (0%‐100%); and 2) on average, how would

rate your ability to take all of your oral chemotherapy medication(s) as your doctor

prescribed? (“very poor” to “excellent”). Patients were reminded to take their

medications and complete weekly adherence reports via push notifications sent directly

from the server. Push notifications are pop‐up messages that appear on the mobile device

and serve to remind the user to engage with the app. The patient could accept the

notification on the screen which directly opened the app to the adherence report page. In

addition, if the patient ignored the notification but entered the app at a later date, a

banner would appear on the home screen serving as a reminder to complete the weekly

report. Finally, a badge would display on the app icon itself (known as a badge app icon),

serving as an additional reminder for the patient to enter the app and complete the

weekly report. The adherence reports were then emailed to the patient’s oncology

clinicians on a weekly basis.

3. Symptom and side effect reporting: The mobile app contained features for symptom and

side effect reporting to participants’ care team. Patients completed an abbreviated

version of M.D. Anderson Symptom Inventory (MDASI) within the app. At a minimum,

patients were required to complete symptom reports on a weekly basis, and were

92

prompted with push notifications (as described above) to complete the symptom and side

effect survey on the app. Participants were also able to go into the app at any time and

report any bothersome symptoms or side effects. For any extreme, new (or worsening)

symptoms, patients were instructed through the app to call their oncology clinician

directly. The symptom reports were also displayed in a graph format showing symptoms

over time. Compiled results from these symptom reports were emailed to the patients’

oncology clinicians on a weekly basis.

4. Education library: To enhance patient engagement with the app, the study team compiled a

library of educational materials that could be accessed by patients within the app. The

library included descriptions of symptom self‐management strategies, skills for

communicating effectively with providers, and links to reputable websites (i.e., American

Society of Clinical Oncology and American Cancer Society websites) where educational

material about specific cancer types is available. There was also a page devoted to

connecting participants with reputable sources that provide advice about managing

finances and financial assistance during cancer care.

5. Social networking: The mobile app contained a social networking component that provided

patients with relevant websites with disease‐specific forums and support groups. In

addition to specific resources, all patients had access to general oncology forums and

support groups from reputable websites (e.g., inspire.com, cancer.net,

patientslikeme.com, etc.). Patients were also given contact information for the Cancer

Resource Center at the MGH and the Patient and Family Advisory Board to gain

information, resources, and support throughout their time at the MGH.

6. Nutrition education: The mobile app had a page that provided participants with helpful

nutritional information as well as suggestions for healthy recipes. This page contained

specific information on nutrition that work well for patients undergoing treatment for

cancer.

7. Fitbit integration: We provided intervention participants with Fitbit devices that connected

directly to the mobile app. Patients were able to keep track of steps taken each day and

create activity goals for themselves.

93

Appendix F. Summary of Protocol Changes

On March 6, 2015, our study team submitted an amendment to the IRB proposing to

restructure the timeline for study participants. We clarified that the 3‐month enrollment

period would begin on the date of orientation rather than the date of consent. We made this

change to accommodate patients with spaced out clinic schedules. The IRB approved this

amendment on March 16, 2015.

We submitted an amendment on March 20, 2015, proposing to add a resource

questionnaire that gathered information on emergency department visits outside of MGH. We

also added a psych resource questionnaire to inquire about patient's recent mental health

services and an ECOG Performance Status questionnaire so that patients could self‐report their

performance status if it is not listed in their electronic medical record. Lastly, this amendment

added a question to the patient qualitative interview to inquire if doctors brought up

adherence and symptom reports from the mobile application during clinic visits. This

amendment was approved by the IRB on April 14, 2015.

On July 12, 2015, our team submitted an amendment to replace the Memorial

Symptom Assessment Scale (MSAS) with the M.D. Anderson Symptom Inventory (MDASI) to

collect participant self‐report data at baseline and post‐assessment. Due to an administrative

error, we had not collected any data using the MSAS or full MDASI at the baseline or post

assessments prior to the submission of this amendment. Participants assigned to the

intervention group who utilized the mobile app had been completing an abbreviated MDASI

on a weekly basis. This amendment was approved by the IRB on July 30, 2015.

We submitted an amendment on August 14, 2015, proposing to add a pill diary that

would be given to all participants when they enrolled in the study. The pill diary was an

optional tool, and was not an official measure of adherence. Rather, it was given to

participants to use in the case that they had notes they would like to take regarding their

adherence on any particular day. This amendment was approved by the IRB on August 26,

2015.

94

On October 19, 2015, our study team submitted an amendment to add Mass General

West (MGH West) as a study site to aid in our enrollment efforts. Additionally, we proposed to

change our adherence monitor (GlowCap to MEMS). This amendment was approved by the

IRB on November 4, 2015.

On December 2, 2015, we submitted an amendment proposing to add an “app usability

questionnaire” to the post‐assessment with the intervention group. This questionnaire

gathered information about the usability of the app. This amendment was approved by the IRB

on December 4, 2015.

We submitted an amendment on March 23, 2016, proposing to increase the overall

study accrual from 180 to 200 participants. This amendment was approved on March 29, 2016.

On June 3, 2016, we submitted an additional amendment to increase the accrual once again

from 200 to 220 participants. This amendment was approved by the IRB on June 20, 2016. By

increasing accrual to 220 participants, we were able to enroll more than 180 participants to

account for those who dropped out or expired after randomization.

Appendix G. ClinicalTrials.gov Results Weblink:

https://clinicaltrials.gov/ct2/show/study/NCT02157519?term=greer+oral+chemotherapy&draw=1&ran

k=1

95

Appendix H. List of Generic Oral Chemotherapy and Targeted Therapy Drugs Afatinib Axitinib Bosutinib Capecitabine Ceritinib Crizotinib Dabrafenib Dasatinib Erlotinib Everolimus Gefitinib Ibrutinib Imatinib Nilotinib Lapatinib Lenalidomide Osimertinib Palbociclib Pazopanib Pomalidimide Sorafenib Sunitinib Temozolomide

96

Copyright© 2019. Massachusetts General Hospital (The General Hospital Corp.). All Rights Reserved.

Disclaimer:

The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent

the views of the Patient‐Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.

Acknowledgement:

Research reported in this report was [partially] funded through a Patient‐Centered Outcomes Research Institute® (PCORI®) Award (#IHS‐

1306‐03616) Further information available at: https://www.pcori.org/research-results/2013/does-smartphone-app-help-patients-cancer-take-oral-chemotherapy-planned

does a smartphone app help patients with cancer …...that interventions delivered via mobile...

Documents