healthcare software testing strategies · keeping up with regulations and compliance, medical...

HEALTHCARESOFTWARE TESTINGSTRATEGIES

M edical professionals, as well as patients, are using

healthcare software applications on mobile devices.

Portable access improves care and convenience. But the

increased complexity raises critical issues about maintaining

security, integration, and interoperability across platforms.

Laptops, smartphones, and tablets add entry points to the

mobile network that increase the risk of loss or unauthorized

access to Patient Health Information (PHI). Testing permutations

are increased several fold with new ways data is stored and

transmitted between various devices via Bluetooth, WIFI, and

cellular communications. Developing a test strategy to not only

cover functionality, but also non-functional requirements in

security, data integrity, display, and performance, requires

careful thought with a systematic approach.

requirements is not met, then it can’t satisfy the needs of a

healthcare application that’s HIPAA compliant. Mobile

security involves many more pieces than we are accustomed

to including in the network infrastructure, hardware, and

operating system. Additionally, with most healthcare

applications providing interoperability amongst medical

devices and information sources via HL7 messages, any

system sending and receiving messages needs to be added to

the test matrix.

Testing Patient Mobile Apps Security and Data Integrity

W ith the cost of healthcare skyrocketing, there is a drive

toward increased e�ciency while raising the level of

patient care and striving for better outcomes. This can only lead

down the path to increasing the scope of software in automating

numerous procedures and processes in healthcare. The

healthcare IT market continues to grow1, supported by

expanding requirements for more sophisticated software across

all categories, including:

• Patient mobile apps

• Medical device software

• Electronic Health Record (EHR)

• Electronic Medication Administration Record (EMAR)

• Billing and claims processing

• Pharmacy Management and dispensing

• Physician Practice Management

There are many more types of software, including: healthcare

speci�c CRM, Health Information Systems, health analytics and

imaging systems. But the above represents the major categories.

When testing healthcare applications, we have to be very

cognizant of the domain and speci�c application usage contexts,

including government regulations and compliance issues, but

especially those regarding data security and privacy. In this white

paper, we discuss the major types of healthcare software and the

issues related to ensuring their quality.

11XBOSOFT.COM CONTACT US AT [email protected]

TESTING HEALTHCARE SOFTWARE

1 "Healthcare IT Market Size To Reach $104.5 Billion By 2020." https://www.grandviewresearch.com/press-release/global-healthcare-it-market. Accessed 20 Mar. 2017.2 "Testing your health care application for use on mobile devices." http://searchsoftwarequality.techtarget.com/tip/Testing-your-health-care-applica. N.p., n.d. Web.

D ata must be accurate and consistent, complete, timely,

interoperable, auditable and accessible.2 If any of these

T he way that your application is rendered probably has

the biggest impact on whether or not the application is

H ealthcare apps on mobile devices must be reliable.

You can’t have an incident in the middle of collecting

patient data or doing a vital reading. Therefore, testing

functionality alone isn’t enough. We need to test in the �eld.

Simulating real conditions, such as syncing of healthcare data is

crucial when a practitioner is with a patient and needs to

upload data or receive it from another source. With a

sub-optimal connection, what happens? In a typical medical

building, signal strengths vary based on location, construction

materials as well as interference from other devices. Ensuring

the application can recover and adapt to suboptimal network

conditions is critical. Con�ict resolution for data access is also

critical. What happens when the same patient record is

accessed from another location?

We all get frustrated if any of the apps on our mobile devices

don’t provide good performance. The frustration is exacerbated

with healthcare apps. Even though they may be more

complicated and involve many more moving pieces, user

expectations don’t take that into account. For testing, this

means having a coordinated performance testing strategy that

not only tests in the lab, but can prioritize and simulate

situations in the �eld that would a e�ect performance. This

means you’ll need automated performance tests that simulate

real connectivity issues as well as real work �ows.

Display

ultimately accepted as a replacement for whatever is currently

in place. When it comes to displaying on di�erent devices,

you’ll have to ensure that you have tested all ‘supported

platforms’ covered. There are no excuses for �eld names not

displaying correctly or being cut o�. Unnecessary scrolling

and clicking with a small screen won’t be tolerated. You’ll have

to make sure your application is optimized for a mobile device

and conduct usability assessments from the very beginning.

Reliability Performance


T o continue to innovate and deliver more features while

keeping up with regulations and compliance, medical

device manufacturers are continuously developing devices

with the help of software. This process is similar to testing

mobile application software, except even more complex. Not

only are testers looking for defects, they must ensure that the

software meet the stringent requirements of the IEC 62304

standard, which applies to all software for medical devices. Thus

they must possess a thorough understanding of regulation and

compliance matters related to testing medical devices. Our

challenge is twofold; We have to test for explicit requirements

in data accuracy and reliability, while being aware of the

security concerns raised by the many new permutations of

connectivity. Therefore, testing medical devices requires a

thorough approach and strategy that covers many

requirements and is designed to reveal defects where they are

most likely to occur.

Testers need expertise and experience of the domain, device

design, and connected infrastructure to support their testing

in such areas as performance, interoperability, security, user

interface (UI), and reliability amongst others. You can’t

tolerate a reboot in the middle of surgery when a patient’s life

is at stake. Testing the UI is critical so that data is presented in

an understandable way to avoid misinterpretation and

confusion. Accurate and timely transmission of the data via

HL7 to various interfaces is imperative.

Behind all this we have increasingly sophisticated malware

trolling healthcare databases with the intent of stealing that

data. Frequent device connections, the transfer of data, and

even just being on the network exposes PHI vulnerabilities. To

thwart this threat, new software must allow medical devices to

integrate into hospital networks and various data sources

safely--and testers must be able to determine if the software

meets such a standard.

Testing Medical Device Software


E nhanced e�ciency, cost containment, and improved

patient outcomes are the primary drivers for EHR software

implementation. Accuracy and completeness of work�ows are

essential to accomplishing these objectives. For testing, you

should have established work�ows or user stories where each

individual and departmental role is de�ned regarding patient

input, clinical documentation, billing, scheduling and patient

follow-up. Thorough testing is needed to ensure that all

processes work in the software including usability issues to

identify areas of confusion. Avoiding confusion is perhaps the

most overlooked element in testing healthcare software; it

requires an eye for detail, mapping the existing processes to the

software work�ows. You want to �nd issues or breakdowns and

correct them before going live with real patients.

When testing, you’ll need to create many test patients to

practice every aspect of clinical documentation, billing, ordering

prescriptions, printing patient education handouts, scheduling,

etc. In doing so, you’ll need to create many user stories. For

example, a checkup for a patient with a broken arm, checkup

visit for a senior citizen, or surgery follow-up.

In your user stories, you’ll need to include every component of

documentation that would be required for that patient, from

patient personal data to medications, vital readings,

prescriptions and future appointments. For each role, step

through each step in seeing the patient. For example, the

receptionist will obtain identi�cation and consent forms, and

input the information into the system. Next, the nurse will

examine the entered information and then collect vitals

including temperature, oxygen level, weight, height and blood

pressure. Physicians will do their physical exam and input their

data and diagnosis, followed by a plan of care and scheduling

for future appointments. By stepping through each of these

roles, you’ll be able to identify possible gaps in understanding or

work�ow. The more variation in your user stories the better. You

can’t test all scenarios, so you’ll need a well-prioritized test plan.

Testing Electronic Health RecordSoftware (EHR)


Testing Electronic Medication Administration Record Software


Reduced medication errors and better outcomes are the primary

objectives of implementing Electronic Medication

Administration Record (EMAR) software. According to the FDA

Adverse Events Reporting System (FAERS), the number of

serious incidents and deaths continues to grow. In a 2016 report,

Johns Hopkins patient safety experts have calculated that more

than 250,000 deaths per year are due to medical error in the U.S.

EMAR systems are being designed and implemented to alleviate

this situation by replacing paper and other non-digital records

that are prone to being incomplete, misread, or even misplaced.

The goal is to ensure that nurses can properly administer and

track medication. This also reduces the chances of ‘lost

medicine’ and provides relevant drug knowledge at their

�ngertips essential to understanding drug interactions and

e�ective integrated case management. This is especially critical

given today's opiate crisis where organizations can be held

legally responsible for those accessing 'lost medicine'. Again,

documenting and understanding work�ows and processes are

critical in testing EMAR software. User stories need to step

through each role. Their part in the overall process of

administering medication and interfaces with other systems that

dispense the medication needs to be examined, including

inventory tracking and knowledge bases that keep up-to-date

drug interaction information.

From FDA (https://www.fda.gov/Drugs/)

Medication Error-Related Deaths and Serious Incidents in the US


E lectronic prescribing of controlled substances (EPCS) is

now permitted in all states, so we can expect it to gain

considerable momentum. Replacing paper prescriptions, EPCS

can reduce fraud/forgery and secure electronic records, while

improving safety and patient care. However, there are

challenges slowing down its adoption. Examples include

pharmacy and prescriber applications of EPCS that need to talk

to each other and ill-de�ned implementation standards. No

standards exist for interoperability other than HL7 for

information transfer. Thus is it necessary to test each system, its

behavior and its interaction with connected systems. This will

require attention to detail and an understanding of work�ows

that cross system boundaries. Additionally, when it comes to

testing, the same issues that surround HL7 (see page 8) are

applicable speci�cally on HL7 compliance.

For physicians ‘writing’ prescriptions, proving identity is required

using 2-factor authentication (TFA) credentials. TFA credentials

can be implemented via tokens or biometrics such as �ngerprint

scans. A physician might need a code and a �ngerprint scan to

write an EPCS. As TFA may be implemented in a variety of ways,

a deep understanding of TFA implementation is essential to

develop scenarios that not only test if it works (positive), but also

to verify that it doesn’t work (negative testing).

Regulatory changes related to EPCS are becoming more

prevalent. Some states are requiring e-prescriptions for

opioids or other speci�c drug classes. They require pharmacies

and physicians to use only electronic prescriptions for

controlled substances. As these new regulations become

implemented in software, knowledge of domain business

rules will be critical to developing accurate user stories.

Testing Electronic Prescribing ofControlled Substances (EPCS) Software


TESTING FOR COMPLIANCE

G overnment regulations apply to all of these software

types. These regulations can be assumed as stringent

F or the healthcare industry in the U.S., the Health Insurance

Portability and Accountability Act of 1996, or HIPAA,

requirements, even if not explicitly stated, so it is critical for any

testing organization to both thoroughly understand these

regulations and ensure that the test strategy and test plan

accommodates them.

represents one of the main regulatory pieces of legislation. It

has two major components:

• Health insurance coverage is protected for workers and

their families when they change or lose their jobs.

• National standards are established for the Security and

Privacy of private health data while allowing the �ow of

health information needed to provide and promote

high-quality healthcare, and to protect the public’s

health and well-being.

Therefore, from a software testing perspective regarding

HIPAA, we are concerned mostly with Security and Privacy of

health data. All healthcare applications must meet HIPAA

standards; ensuring that this happens can be very arduous, as

data crosses many disparate systems and follows a patient

from system to system. Data and information must be kept

safe from unauthorized access, and when it is accessed, it must

be kept private to only those given access. The data’s

aggregation level must be properly followed to prevent an

individual’s data from exposure.

The �ve following areas must be covered when testing a software

product for HIPAA compliance3:

1. User Authentication – Using veri�cation methods to

ensure that those logging on are who they say they are,

and to deny access to all others.

2. User Authorization – Authorizing access to

information is based on user role and patient limitations.

3. Audit Trail – All transactions and all attempts of

data access with a proper set of audit trail information

are recorded.

4. Data Transfers – Ensure data encryption at all transfer

points according to ANSI 5010.

5. Help Information – Help information on the correct

and incorrect uses of data.

There are many other speci�c regulations that may or may not apply depending on your software product and its users. A few of these requirements include3:

1. When a breach of security is discovered, noti�cation to

the consumer a�ected must be within 60 days.

2. Rules for pre-existing conditions are numerous and

complicated. For example, the exclusion period for

pre-existing conditions can be 6, 12, or 18 months,

depending on the circumstances. These rules have many

variables and many test cases are needed to ensure

compliance. Having a thorough understanding of these

rules is paramount.

3. Rules concerning family members are also varied and

complicated. You’ll need to understand the coverage

variables for each type of situation.

Testing for HIPAA Compliance


3 Healthcare Software Testing Strategies for HIPAA ...(n.d.). Retrieved from https://xbosoft.com/healthcare-software-testing-strategies-for-hipaa-compliance/

https://xbosoft.com/software-testing-services/security-testing/

H ealth Level Seven International (HL7) is a standards body

providing “a comprehensive framework and related

standards for the exchange, integration, sharing, and retrieval

of electronic health information”4 to support health services.

In order to test HL7 standards compliance, it’s important to

understand its functionality as well as its limitations. The HL7

framework is very �exible, which means that every single

implementation will be di�erent. Two di�erent environments

will have di�erent data exchange rules. Therefore, testing HL7

interfaces has several associated challenges:

1. Each system needs to interpret the meaning of each

data item. This means you need to understand the

semantics of the data in di�erent contexts and how

each system receives the data and deals with it.

2. Data items can be exchanged to achieve di�erent

multiple goals in di�erent contexts. Some �elds may be

documented as optional but for speci�c work�ows, there

may be data constraint rules. In some contexts, the rules

will be removed. This means you’ll have to analyze and

test each exchange on a case-by-case basis.

3. HL7 provides suggested values for some �elds, but

you’ll need to understand how systems implement them

and the mapping between di�erent systems.4

4. Segments and �elds require customization:

Field length, data types and other attributes can

be customized, which means you’ll have to

understand the data structure and di�erences

between interfacing systems.

HL7 interface testing typically includes:

1. Unit testing to con�rm that HL7 messages sent and/or

received from each software application conform to the

HL7 interface speci�cation.

2. Integration testing with user stories that ensure that

information �ows correctly between applications.

3. System testing consisting of end-to-end scenario

testing to ensure all modules of all relevant applications

are able to correctly exchange data. This means you’ll

need to design scenarios that integrate EHRs, Billing

Systems, HIEs and other healthcare applications

requiring in-depth knowledge of each system.

Any and all testing needs to ensure the protection of privacy

regarding PHI. This means any information in a medical record

that can be used to identify a speci�c individual, including

information about health status, provision of health care, or

payment for health care.

Testing For HL7 Compliance

4 Health Level Seven International - Homepage. (n.d.). Retrieved from http://www.hl7.org/


16XBOSOFT.COM CONTACT US AT [email protected] CONTACT US AT [email protected]

H ealthcare software continues to grow and encompasses

many di�erent technologies and environments, expanding

from a typical hospital acute care setting to long-term care

facilities, home care, and more. As such, the drive is to allow the

�ow of information to improve e�ciencies and medication

decision making while improving outcomes. Achieving this

utopia, however, can involve over a dozen software components

that need to talk to and integrate with one another.

Testing all of these pieces and components needs to focus at

each level of aggregation. First, on each individual unit test,

then combining related units into a component test, and then

�nally combining components into a system where complete

user scenarios can be executed from beginning to end. The

test plan should be composed of each function and feature

used in practice. This includes all scenarios: visit types, roles

and actions/information entered. Test cases should include

expected results as well as deliberate negative tests to quickly

detect defects and system logic de�ciencies or errors.

Regarding regulations and compliance, it’s important for the

test strategy and plan to include which parts of the

regulations apply and to ensure that the right rules are

included in test case design to guarantee proper coverage.

This all requires very specialized expertise in the software

being tested combined with experience in the healthcare

domain - a body of knowledge that can only be accumulated

over time. This should be facilitated by implementing a test

case management system with procedures within each

scenario for tracking issues, requirements, expected and

unexpected results as well as data needed for analysis of root

causes. Without learning, and managing the speci�c domain

knowledge, it’s di�cult to test healthcare software and ensure

its quality.

Lastly, in addition to knowledge of the regulatory

environment, domain knowledge, and understanding

interconnectivity complexities, we must always keep security

in mind. Testing for vulnerabilities should take place at all

phases of development to ensure PHI is protected to prevent

unauthorized access.

SUMMARY - ACCUMULATING DOMAIN KNOWLEDGE IS CRITICAL

https://xbosoft.com/test-strategy-test-plan/

ABOUT XBOSOFT

As an experienced software testing and QA company, our team of nearly 100 has hundreds of years of cumulative work experience

serving clients, from Fortune 500 to small ISV’s, in North America, Europe, and Asia. Through long term multi-year relationships with

our clients, we’ve developed deep technical expertise in software quality assurance, covering web, mobile, embedded and desktop

apps, and spanning compliance-heavy industries, such as �nance and healthcare.

XBOSoft’s software quality processes are designed to provide con�dence every step of the way. We ensure that your software not only

performs optimally, but keeps your users coming back.

PHONE: +1 703 995 9896 EMAIL: [email protected] XBOSOFT


healthcare software testing strategies · keeping up with regulations and compliance, medical...

Documents