A total of 42 patients enrolled (Table 1).

14 TB disease, 28 LTBI (n=42)

• Patients still on treatment, n=12 (28.6%).

• Completed treatment using AiCure app, n=22 (100.0% among those eligible to continue)

• Stopped treatment due to side effects, left the protocol, moved out of country, n=8 (19.0%)

• Mean cumulative average adherence is 94% based on visual confirmation by the software algorithms (range 60-100%).

–97% Active Disease

–92% Latent Infection

• 100% completion rates among those eligible.

• Baseline demographics were comparable between both groups (Active Disease and Latent Infection). The ratio or men to women was slightly higher in the outpatient portion of the study (Table 1).

• 5,881 adherence-related parameters collected on a pill by pill basis. 995 doses visually observed with AI application. 44 out of 1,057 doses (4.2%) were classified as: self-reported (on app or by phone), missed, or awaiting data. These comprised 299 individual pills.

–Visual data not successfully transmitted for 1.4% of all pills

• 40 seconds per pill (avg 2.5 minutes for 4 pills, TB disease).

• 84 interventions logged (84.5% phone calls, 10.7% in person visits, 4.8% SMS texts). (See Figure 3)

Value of Automated Real-time Monitoring

Unlike in-person DOT or vDOT (synchronous or asynchonous), software algorithms in the AI Platform automatically confirm correct administration (patient, medications, ingestion). The AI Platform works on an exception based rule, i.e. only those patients who require follow up will trigger alerts, allowing clinic staff to concentrate resources on those patients at most risk of nonadherence. Missed doses trigger automated alerts on the dashboard to allow for immediate intervention by health care workers (see Figure 4). In addition to dahsboard alerts, clinic staff may choose to receive alerts by SMS text or by email.

REFERENCES1. Garfein RS, Collins K, Muñoz F, Moser K, Cerecer-Callu P, 1Raab F, et al. Feasibility of

tuberculosis treatment monitoring by video directly observed therapy: a binational pilot study. Int J Tuberc Lung Dis. 2015;19:1057–64. http://dx.doi.org/10.5588/ijtld.14.0923

2. Hayward A, Garber E. TB Reach 5: to compare the efficacy of video observed treatment (VOT) versus directly observed treatment (DOT) in supporting adherence in patients with active tuberculosis. 2014 Apr 17 [cited 2015 Oct 30]. http://www.isrctn.com/IS-RCTN26184967

3. The Behavioural Insights Team. Virtually observed treatment (VOT) for tuberculosis patients in Moldova. ClinicalTrials.gov. https://clinicaltrials.gov/ct2/show/NCT02331732

4. Bain EE, Shafner L, Walling DP, Othman AA, Chuang-Stein C, Hanina A. Use of a Novel Artificial Intelligence Platform on Mobile Devices to Assess Dosing Compliance in a Phase 2 Clinical Trial in Subjects with Schizophrenia. JMIR Mhealth Uhealth (forthcoming).

5. Labovitz DL, Shafner L, Reyes Gil M, Virmani D, Hanina A. Using Artificial Intelligence to Reduce the Risk of Nonadherence in Patients on Anticoagulation Therapy. To be presented during Session S53: General Neurology: Mechanisms and Diagnosis in Overlapping Medical and Neurological Diseases; American Academy of Neurology, April 28, 2017.

6. Story A, Garfein RS, Hayward A, Rusovich V, Dadu A, Soltan V, Oprunenco A, Collins K, Sarin R, Quraishi S, Sharma M, Migliori GB, Varadarajan M, Falzon D. Monitoring Therapy Compliance of Tuberculosis Patients by using Video-Enabled Electronic Devices. Emerg Infect Dis. 2016 Mar;22(3):538-40. doi: 10.3201/eid2203.151620. PubMed PMID: 26891363.

7. Belknap R, Weis S, Brookens A, Au-Yeung KY, Moon G, DiCarlo L, Reves R. Feasibility of an ingestible sensor-based system for monitoring adherence to tuberculosis therapy. PLoS One. 2013;8(1):e53373. doi: 10.1371/journal.pone.0053373. PubMed PMID: 23308203.

8. Hu D, Liu X, Chen J, Wang Y, Wang T, Zeng W, Smith H, Garner P. Direct observation and adherence to tuberculosis treatment in Chongqing, China: a descriptive study. Health Policy Plan. 2008 Jan;23(1):43-55. PubMed PMID: 18156634.

AcknowledgmentThe authors acknowledge Stuart McMullen and Monica Rosales, PhD for their help in preparing the poster.

DisclosuresAlicia H. Chang, Ana Delia Hernandez, and Monica Rosales are employees of the Los Angeles County Department of Public Health – Tuberculosis Control Program. Los Angeles, CA, USA. Stuart McMullen is an employee of the U.S. Centers for Disease Control and Prevention assigned to work with the Los Angeles County – DPH.

Adam Hanina and Laura Shafner are employees of AiCure, New York, NY, USA, and consultants to LA County Department of Public Health – Tuberculosis Control Program.

Evaluating the Use of an Artificial Intelligence (AI) Platform on Mobile Devices to Measure and Support Tuberculosis Medication Adherence

CONCLUSIONS

METHODS

BACKGROUND• While innovative approaches such as vDOT are being deployed, the requirement to review

each video by health care workers and risks to patient confidentiality while data are transferred need to be carefully considered [6]. Other approaches such as the use of ingestible sensors and electronic medication packaging have also been tested. Ingestible sensors were validated for accuracy during DOT visits in patients with active TB but were not tested outside the clinic as an alternative to in-person DOT [7]. A recent RCT using electronic medication packaging that registers the date/time of opening with reminders incorporated demonstrated effectiveness [8], however the major limitation of electronic monitoring devices is that they do not verify drug administration.

• The Los Angeles County Department of Public Health (DPH) collaborated with AiCure to pilot the first use of an AI Platform AI in lieu of in-person DOT for TB disease and LTBI patients in one public health TB clinic.

• Completion rates of 100% among those eligible to continue, compared to 50-60% completion rates in patients receiving self-administration treatment (SAR).

• Mean cumulative adherence of 94% based on visual confirmation of drug ingestion by software algorithms.

• High patient likability and preference of the AI Platform over in-person DOT or vDOT.

• High adherence rates and minimal loss of data suggest that automated treatment monitoring using AI platforms is safe and feasible for active TB and LTBI patients.

• The artificial intelligence (AI) platform (AiCure, New York, NY) uses artificial intelligence to visually confirm medication ingestion (Figure 1) using software that can be downloaded as an application onto any mobile device. The platform has been clinically validated and demonstrated increases in drug concentration levels compared to other monitoring methods across different therapeutic areas [4,5].

• The pilot study has a target enrollment of 500 patients over the course of three years and multiple SPAs. To date, the technology is being used at one district public health center within one of the County’s eight Service Planning Areas (SPAs).

• Active TB patients in the continuation phase of treatment and new LTBI patients were eligible to participate. Patients were provisioned smartphones with the smartphone application was installed with Health Insurance Portability and Accountability Act (HIPAA)-compliant AI software (Figure 1).

• Unlike video-recorded DOT sessions that require 1:1 viewing, the AI platform relies on software algorithms to ensure correct dosing. Real-time data are encrypted and transmitted to secure centralized web-based dashboards, which the clinics and the county can access. If the study medication was not taken or was not taken correctly, DOT nurses were notified through the dashboard as well as by email or SMS text. Notifications were also transmitted if side effects were reported via the application. Subjects monitored themselves taking the medication on the application each time they dosed. At the required medication time, the software provided a medication reminder and walked the patient through a multi-step process: 1) patient identification; 2) medication identification; 3) confirmation that the patient had placed the medication in his or her mouth, and 4) empty mouth check No WiFi/3G/4G is needed: data are stored and will transmit upon signal. Videos of each dosing administration are stored and can be reviewed at any time.

• Adherence data are based upon visual confirmation of drug ingestion by the software algorithms (Figure 2).

• The total number of new TB cases reported in the U.S. in 2015 was 9,563, with LA County ranked within the top five areas reporting the highest number of cases. In addition, there are an estimated 11 million cases of latent TB infection in the US, of which 5-10% could progress to TB disease if left untreated.

• In-person directly observed therapy (DOT) is a global standard of care and a proven effective method to monitor patients during their course of TB treatment. DOT is resource intensive for TB programs and patients; universal DOT is challenging.

• Millions of patients begin TB treatment but face challenges in complying with treatment: logistical requirements of in-person DOT and complex regimens cause many patients to adhere inconsistently or stop treatment prematurely.

• Challenges to in-person DOT include logistical requirements of in-person DOT causing delays in completing the treatment regimen.

• Recently, innovative approaches to adherence monitoring have been piloted such as video-based DOT (synchronous or asynchronous video transmission requiring full review of each video). [1,2,3]. The Los Angeles County Department of Public Health (DPH) collaborated with AiCure to pilot using AI in lieu of in-person DOT for TB disease and LTBI patients in one public health TB clinic.

Objectives• Explore and understand the usability and feasibility of using an AI platform in measuring and

supporting tuberculosis medication adherence in patients with active disease and latent infection.

• The primary endpoint is to demonstrate that automated monitoring of treatment is equivalent to real-time observation. A secondary aim is to evaluate the platform’s cost-effectiveness compared to other monitoring methods.

Laura Shafner, MSc;1 Alicia H. Chang, MD, MS;2 Ana Delia Hernandez;2 Adam Hanina, MBA1

1AiCure, New York, NY, USA; 2Los Angeles County Department of Public Health, TB Control Program, Los Angeles, CA, USA

Figure 1. AI Platform Used in the Study Table 1. Enrollment Status

Dashboard web-based (roles-based access control)• Detailed dosing charts available for each

patient displaying data on a pill by pill basis• Data can be exported at any time for patient/

clinic performance• Computer vision and software algorithms

confirm patient ID, medication, and correct administration

• Real-time data are encrypted, (HIPPA-com-pliant) and transferred to cloud-based dashboards

• Real-time alerts of missed/incorrect doses• Patients able to report side effects

Figure 2. Majority of patients within 91-100% adherence range

RESULTS

Figure 3. Logged interventions by clinic staff based on real-time dosing data

Figure 4. Patient with Active Disease, assigned to twice weekly regimen, switched to daily dosing after 5 weeks following real-time notifications of missed doses.

FIGURES & CHARTS