fitnesse and error prevention - himss...dbfit (free) runs sql queries from test case expected output...
TRANSCRIPT
FitNesse and Error PreventionKrystal Baldwin
Mujeeb A Basit MD, MMSc
HIMSS Davies Enterprise Award Site Visit
2
Medical Errors
3
Problem
+ Frequent Enhancement = High Risk of ErrorComplex
4
Moncrief Medical Center at Fort Worth
UT Southwestern Health System
Academic Medical Center in Dallas
Partnered with Texas Health Resources
William P. Clements Jr. University Hospital
604Beds
1700Physicians
40+Clinics
23,132Admissions
40,813ED Visits
West Campus Professional Building 1
Radiation Oncology Facility
7947Inpatient Surgeries
5
Who we are
Krystal BaldwinDirector
HSIR Operations and Quality
Mujeeb Basit MD, MMScAssociate CMIO
Cardiologist
6
The Problem
7
Prior StateEr
rors
Time
User Reported Error
8
Example Error
As a charge master coordinator, I need a FitNesse test created to check the accuracy of the “Show on UB” item on hospital billing chargeable procedure
records and the HCPCS code is valid.
9
Physician Billing Process
Procedure
Charges (EAP)1. HTR2. HTR3. …
Claim File
Rev Code CPT / HCPCS Billed Amt Exp Reimbursement
0278 C1760 $200.00 $100.00
0278 ------- $200.00 $0.00
0481 93458 $1000.00 $500.00
…
10
Claim Window
11
Charge Configuration
12
0
20
40
60
80
100
120
140
2016-05 2016-07 2016-09 2016-11 2017-01 2017-03 2017-05 2017-07
Nu
mb
er o
f Er
ror
Rec
ord
s
Month
Charge Record Analysis
Additional Analysts
User Story
13
Physician Billing Process
Procedure
Charges (EAP)1. HTR2. HTR3. …
Claim File
Rev Code CPT / HCPCS Billed Amt Exp Reimbursement
0278 C1760 $200.00 $100.00
0278 ------- $200.00 $0.00
0481 93458 $1000.00 $500.00
…
Paid Less Work Queue Denied
14
Financial Impact
Before FIT(Six Months)
Transactions Affected 98
Expected Reimbursement per Transaction
$5,506.04
Self Pay $137.86
Total Per Transaction $5,643.89
15
Error Prevention
16
“ I conclude that though the individual physician is not perfectible, the system of care is, and that
the computer will play a major part in the perfection of future care systems.
~ Clem McDonald, MD NEJM 1976
17
Prior StateEr
rors
Time
User Reported Error
18
Solution Goals
Early Detection
No repeat failures
Automated
Reduced Time to Resolution
Traceability
HighReliability
19
Design, Implementation, and Governance
20
Uses of Automated Testing
Reactive Preventative
Erro
rs
Failure in Operation
ServiceNow Ticket
Error Correction and Automated Monitoring
Erro
rs
Failure during Development or Testing
Internal Request
Monitoring in Production System
21
FIT and FitNesse
FIT = Framework for Integrated Testing
FitNesse is a HTML wiki based open-source “front-end” to FIT
Guarantee business rules
22
Automated Acceptance Testing and FitNesse
Failure story
ServiceNowTicket
FIT Table (Test Case)
http://www.fitnesse.org/FitNesse.FullReferenceGuide.UserGuide.FitNesseWiki.MarkupLanguageReference.MarkupPicture
Claritydatabase
dbFit(free)
runs SQL queries from
Test Case
Expected Output = Actual OutputExpected Output ≠ Actual Output
23
28 Hours
IT Infrastructure
FitNesse Server
dbFit
SQL Server Driver
Epic
Clarity KB_SQL
EpicChronicles
Data Warehouse
Dev3 Prod SQL Prod SSAS
5 Minutes
24 Hours
Real-Time
Time to Detection
24
Start of a FIT Test
http://www.fitnesse.org/FitNesse.FullReferenceGuide.UserGuide.FitNesseWiki.MarkupLanguageReference.MarkupPicture
Committee
Failure story
ServiceNowTicket
Erro
rsEr
rors
ServiceNow
Reactive
Preventative
25
Fault Handling
Parent Incident FIT Name Initial Records 2017,06 2017,07 2017,08 Total Trend
INC0653546 FIT0653546:OR Procedure records without SHx/EAP value 2012 0 0 0 2012
INC0656674 FIT0656674:Test patients but do ont conform to ZZZTEST as the last name 88 0 0 0 88
INC0656678 FIT0656678:Patients with a last name like test but not marked as test patient 1 0 0 0 1
INC0657263 FIT0657263:Individual statin medication records not in groupers that identify whether med is high/moderate/low intensity statin3 0 0 0 3
INC0657288 FIT0657288:E-Prescribe flag set to "Y" for ordering 25 or more prescriptions in Epic for last 150 days18 3 1 0 47
INC0678844 FIT0678844:Radiology departments missing vital items - facility, center, rev location, report grouper 7, or default POS ID2 0 0 0 3
INC0683631 FIT0683631:Test of POS Business Rule 30 0 0 8 51
INC0683633 FIT0683633:Test to find out if 205 = 14 or if 5012 = 76 or 99 for Radiology procedures 1 0 0 0 1
INC0683635 FIT0683635:Test to find Visit Types that are active but not attached to a correct EAP record 0 0 0 0 0
INC0683641 FIT0683641:Test of PRL Business Rule 114 0 0 0 114
INC0683645 FIT0683645:Test of POS Business Rule 27 0 0 0 32
INC0683647 FIT0683647:Test of PRL Business Rule -HCM Metadata records 0 0 0 0 0
INC0699493 FIT0699493:MSDRG Grouper 1 check 0 0 0 0 0
INC0700559 FIT0700559:Billing providers with grouper 9 0 0 0 0 0
26
Governance
FitNesse Automated Test Committee
• Director HSIR Operations and Quality• CMIO • Imaging Team• Quality and Testing Team• Application Infrastructure• Business Analytics• EMR• Hospital Business Systems
Enterprise Data Warehouse
Steering Committee
Health System Business Rules
Workgroup
FitNesse Governance
Health System Information Resources Steering
Health System EHR
Governance
Ambulatory EHR Governance
Oncology EHR Governance
Revenue Cycle Governance
Imaging EHR Governance
Clinical Decision Support
Governance
Hospital EHR Governance
Pharmacy EHR Governance
Emergency Department
EHR Governance
Neonatal Pediatrics EHR
Governance
Laboratory EHR Governance
Obstetrics Gynecology EHR
Governance
Medical Executive
Committee
Health Information
Management Committee
27
Results
28
Automated Tests
0
20
40
60
80
100
120
140
2015-11 2016-02 2016-05 2016-08 2016-11 2017-02 2017-05
Nu
mb
er o
f Te
sts
Date
29
0
20
40
60
80
100
120
140
2016-05 2016-07 2016-09 2016-11 2017-01 2017-03 2017-05 2017-07
Nu
mb
er o
f Er
ror
Rec
ord
s
Month
Charge Record Analysis
User Story
Additional Analysts
30
Financial Impact
Before FIT(Six Months)
After FIT(Six Months)
Change
Transactions Affected 98 64
Expected Reimbursement per Transaction
$5,506.04 $6,317.65 12.8%
Self Pay $137.86 $0
Total Per Transaction $5,643.89 $6,317.65 $673.76
31
Anesthesiologist
As an Anesthesiologist, I need a FitNesse test to track the physical location of the Tangent device and DIM for each Anesthesia cart in the OR. Each Tangent Device is programmed to stay together
with a specific DIM and, if separated, vitals for one patient could flow to a different patient’s chart in
Epic
32
Avoiding Data Errors
33
Data Transfer
Serial Port(RS232)
Network(Ethernet)
34
Mechanism of Error
Room 1 Cart
Room 2 Cart
Data Routed
Incorrectly
35
Matching Hardware
Room 2 CartRoom 2 Data Module
Asset Tag Mini
CetaniDatabase
15Mins
36
Cumulative Records in Error from all FIT Tests
14,692
0
2000
4000
6000
8000
10000
12000
14000
16000
2015-11 2016-02 2016-05 2016-08 2016-11 2017-02 2017-05 2017-08
Nu
mb
er o
f R
eco
rds
Iden
tifi
ed
Date
37
Return on Investment
38
Financial ROI – Meaningful Use
1. FIT0749762 Providers without NPI but have ordering authorizing capabilities
2. FIT0657288 Meaningful Use EP (Electronic Prescribing) flag set to 'Y'
3. FIT0739097 SER Provider to License Type mismatch
4. FIT0683591 Primary Scheduling department for MU Provider
5. FIT0964302 Active Provider Inactive User MyChart Settings Check
Compliance for MU is our greatest hurdle.
$275,199.79 for 35 eligible clinicians who would have failed ePrescribe
39
What if we didn’t have FitNesse?
Manual Test Execution Monthly Weekly Daily
Test Runs 12 52 245
Number of Tests Executed 100 100 100
Hourly Rate $55 $55 $55
Labor Cost $6,600 $28,600 $134,750
40
Additional Uses of Automated Testing
41
Uses of Automated Testing
Reactive FIT
Preventative FIT
Test Driven Development
Predictive FIT
42
Test-Driven Development
Test
Test
Build
Test
43
Test Driven: CKD CDS FitNesse
Pre
-Bu
ildIn
itia
l Bu
ildFi
nal
Bu
ild
No BPA
BPA with Errors
Correct BPA
44
CDS Monitoring
Overexpression Under expression
CDS displaying for patient that it is not supposed to
(false positive)
CDS not displaying for patient that it is supposed to(false negative)
Date
Nu
mb
er o
f V
iew
s
Date
Nu
mb
er o
f V
iew
s
45
Predictive CDS Monitoring
46
Conclusion
47
Summary
High ReliabilityFIT
15Mins
24Hrs
48
Summary
Plan
DoStudy
Act
Modeling
AgileData Warehouse
Error Detection
As a ___, I want to be able to ____ so that ____.
49
Questions
• Email us at: