greater healthcare demands

Greater healthcare demands !!

Healthcare meltdown

Aging and nursing shortage

400,000 shortag

e by 2020

1:20 vs ideal of

1:10

INCREASING DEMANDS on SENIOR NURSESRetention of Workers Strategy

At least 1

adverse effect

70% of deaths

INCREASED INCIDENCES of ERROR

WORK ABILITY

Physical MentalHeterogeneous concept that must be broken down

There is no specialized concept or metric for the cognitive aspect of work ability and there are

also no methods at present to measure / assess this.

Measure Cognitive Work Ability

Job reassignment & interventions

Predicting stress levels and likelihood of adverse effects (ADEs)

Existing methods are insufficient & subject to bias.

Automated and Performance-Based

CASNUR: Developing a software to assess cognitive work ability among aging nursesby Maria Dominique Rustia & Rosemary SevaHuman Factors and Ergonomics Center, De La Salle University, Manila, [email protected]

May 28, 2012IFA 2012 Global Conference on Ageing

mailto:[email protected]

PROBLEM STATEMENT

•Existing assessment methods for work ability among nurses are subject to response bias and fail to consider the cognitive dimension of tasks.

•There is a need for a performance-based assessment method for cognitive work ability (CWA) to be established, especially in the context of ICU nursing.

1Background Method Concept & Design ConclusionTesting

OBJECTIVES, SCOPE & LIMITATIONS

• to provide a reliable and valid measure of CWA among hospital nurses by developing a software application that can assess CWA among them.


1. To test the validity of the CWA assessment software2. To perform an initial evaluation of the usability of the software3. To determine the reliability & effectiveness of software in capturing data for CWA metric derivation4. To identify areas for improvement in the software

SCOPE & LIMITATIONS Hospital nurses, ages 40-65, Intensive Care Unit (ICU) setting Philippine setting Will only test variables of CWA model of Rustia & Seva (2011) Will not propose any interventions Conceptualization and design of software Gender, affective & physical needs not considered

OBJECTIVES

COGNITIVE WORK ABILITY 1

Cognitive Work Ability (CWA) the ability of a person to fulfill mentally demanding tasks efficiently and

effectively

Background Method Concept & Design ConclusionTesting

Work Ability Index (WAI) “how good workers are at present and in the near future, and how they will be able to do their work

with respect to work demands, health and mental resources” (Ilmarinen & Tuomi, 2003)

speed accuracy

EXISTING ASSESSMENT METHODS 1

• Reliability and validity have been verified but coverage of components of CWA is poor

• Questionnaire-based, risk of biased and inaccurate answers

• No performance-based evaluation methods to date

SIMULATION OF THE NURSING

ENVIRONMENT(low cost software)


Work Ability Index

(Ilmarinen & Tuomi, 1992)

WHO Quality of Life (WHO,

1991)

Expanded Nursing Stress Scale (French et al., 2007)

TISS-28 (Miranda et al., 1996)

Revised Nursing Work Index (Aiken &

Patrician, 2000)

CASNUR

Cognitive work ability Assessment Software for NURses

METHODOLOGY 2Background Method Concept & Design ConclusionTesting

METHODOLOGY 2Systems Development Life

Cycle


SOFTWARE DESIGN- Software Concept3

SIMULATIONDATA-

GATHERINGEVALUATION

1

2

3


SOFTWARE DESIGN – Requirements Planning

• Applied Cognitive Task Analysis (ACTA) applied to determine cognitive processes in regular ICU nursing tasks.

3

ACCURACY SPEED

PERFORMANCE QUALITY


SIMULATION INTERVIEW &

KNOWLEDGE AUDIT

MONITOR and ALARM hard to say immediately where the problem is

EMERGENCIES lead to improvising

presence of mind and alertness, prioritization, good memory, speed

checking, finding discrepancies in medicine

arrangement and labeling of medicine, better systems



3

SIMULATION1DATA-

GATHERING2

Abnormal event generation

Errors, Actions, Responses



Other Characteristics:• Simulation must also have the same layout as actual ICU in the Philippines• Emergency ICU case feeling of “time pressure” and urgency

Constantly changing

E-cart to concentrate on individual skill

Computerized to remove bias related to

handwriting & technique

SOFTWARE DESIGN - Conceptual Model3Background Method Concept & Design ConclusionTesting

SOFTWARE DESIGN - Software Features3Background Method Concept & Design ConclusionTesting

SOFTWARE DESIGN – Software Features 3Background Method Concept & Design ConclusionTesting

EXPERIMENTATION- Factor Selection 3

Measurable by CASNUR

Measurable by CASNUR and eye-tracker software

Measurable by CASNUR and eye-tracker software

Measurable by CASNUR


PILOT TEST – Subjects & Assumptions4

ASSUMPTIONS The participants selected are representative of the targeted users of CASNUR. The medications are assumed to be correct even if they medically are not. The frequency of errors is assumed to be correct. The subjects were physically, mentally and emotionally fit before testing.

• Filipino nurses, ages 45+ • Fluent in English and Filipino• Capable of using computers• No physical or mental diseases / impairments• Must have had experience in ICU / emergencies• Day shift to avoid influence of night shifts• Pre-scheduled appointments to avoid rushing

SUBJECTS & ENVIRONMENT


PILOT TEST - Methodology 4

CASNUR software

Usability Plan, CASNUR

Usability Questionnaire

Dikablis Eye-Tracker & SoftwareCASNUR + Eye-Tracker +

MoraeSimulation Evaluation

Survey

Subject Profile Survey


PILOT TEST - Methodology 4

RELIABILITY VALIDITY USABILIT

Y

Testing of successful generation of

variables and events

How close the simulation is to real-

life situations

EFFICIENCY

EFFECTIVENESS

SATISFACTION


PILOT TEST - Results and Analysis 4

RELIABILITY

SIMULATION Vital Signs Monitor changes, alarms and indicates abnormality

• Emergency cart with full functionality of selecting and adding medicine to table

• Medicine Table with maximum capacity of 3 medications

• Administration of oxygen, dextrose and medicine (input of dosage)

DATA-GATHERING Error identification, counting and recording

Response / action identification, counting and recording

Visual timer on interface

Export feature to EXCEL



RELIABILITY


10 variables measured


VALIDATION OF CWA MODEL FROM RESULTS

Attentive Resources

Perception

Working Memory Capacity

Responsiveness

Task structure

TESTS CONDUCTED:

Collinearity

Heteroskedasticity Normality



VALIDITY

Shapiro-Wilk W-test for normality

Correlation analysis

Simulation evaluation survey

Rating of how realistic simulation is

Mostly normal in distribution

4 / 5 (5 = very realistic)

Number of wrong medicine types = positive correlation with unnecessary

actionsResponse time = +correlation with

suffering durationCorrect responses = -correlation w/

wrong type errors



USABILITY

EFFECTIVENESS High Learnability (12.03% unnecessary actions)

• 40.66% of abnormal events solved by the users

EFFICIENCY “Normal” task completion

times



USABILITY

SATISFACTION 50% satisfaction rating 5.25 / 6 in ease of use Second trial slightly easier than

first



DEBRIEFING RESULTS



AREAS FOR IMPROVEMENT

PREPARATION1. Only qualitative testing for

presence of mental impairments2. Inconsistencies in pacing and

tone of briefing voice.

Use of COGLAB Memory Test Design of a briefing video for

CASNUR

NORMALITY OF DATA

SOFTWARE1. Unstoppable alarms make nurses more irritable or tense.2. Nurses tend to forget to select the medicine first before clicking on GIVE

MEDICINE.3. Confusion with color on vital signs monitor.4. Nurses forget to click DONE / CANCEL to return to main interface from e-cart.5. Scattering of physician’s orders during experiment6. Too many abnormal events occurring within the span of 5 minutes.

More runs / replications




SOFTWARE ON/OFF toggling of alarm Reprogramming of functions Use of brighter color (YELLOW) Placing of “GO BACK TO

PATIENT” button in e-cart and changing CANCEL to “CANCEL addition”




EXPERIMENT1. Difficulty moving mouse

around.2. More than one run per setting

is needed.3. Subjects tend to get

significantly better after 4th / 5th try.

4. Need a basis / ideal performance value for the task times.

5. Usability testing needs more standardization.

Touchscreens More subjects and runs Do not use beyond 3 tries

per subject. Compute for performance

value through control limit computation & interviews with nursing educators. Inclusion of NASA TLX test.


CONCLUSION & RECOMMENDATIONS5

• Cognitive Work Ability (CWA) is a specialized concept for the ability of a person to fulfill mentally demanding tasks efficiently and effectively.

• Simulation as performance-based method for assessing CWA among ICU nurses.

• A simulation software has been successfully designed with acceptable usability ratings, reliability in data collection & simulation, and validity (face and construct).

• Recommended use of software by the academe, health professionals, nurses, hospitals.

• Larger experiment with more samples and actual derivation and evaluation of CWA index.

• Incorporation of eye-tracking and screen-recording features in system to reduce CASNUR’s usage costs.

• Testing by wider demographic (young and old).

• Longitudinal testing of software and CWA index.

CONCLUSION NEXT STEPS


“With society becoming older and more active, will we change the way we work…? Does it

make any sense to stop being productive at a particular age?”

– Ken Dychtwald (1990)

TRY OUT CASNURLook for me after the presentation /

session / anytime during the IFA 2012 Conference.

Thank you.Questions / Comments? Feel free to e-mail them to:

[email protected]

greater healthcare demands

Documents