potenzialità dell’intelligenza artificiale come · the biorobotics institute 2019 8 professors 3...

Potenzialità dell’intelligenza artificiale come

strumento di pre-screening di massa: applicazioni wearable nello studio del movimento

umano

Andrea Mannini, PhD

Statistics for Health and Well-beingUniversità di Brescia, Dip. Economia e Management

25 – 27 Settembre 2019

www.bioroboticsinstitute.eu

Pisa: a “melting pot” of students, professors and scientists

Source: MIUR, data collected oct 2011

National Institute

of Nuclear Physics

300 Researchers

National Research Council

15 Research Institutes

1.500 Researchers

1810

87

450

1343

1450

53.000

1987

103

800

Founding date

Faculty staff

Students

The BioRobotics Institute 2019

8 Professors

3 Associate

Professors

15 Assistant Professors (4 in tenure track)

9 Administrative Assistants

7 Technical Assistants

8 Tecnologists

75 Post Doc Research Assistants

90 PhD Students

250+ people

40+ % women

0

5

10

15

20

25

30

2011 2012 2013 2014 2015 2016 2017 2018 2019

Assistant Associate Professor Total

Volterra annual meeting July 5-7, 2018

About me

Biomedical Engineer (2009)

PhD in BioRobotics (2013)

Assistant professor in

Bioengineering at Scuola

Sant’Anna (2017)

Work on machine learning

enabled solutions for wearable

sensors (mainly inertial sensors

and electromyography)

AI-based preliminary-diagnosis using wearables

AI momentum

Why is AI living a new explosion?

High computational power

available in our pockets

Amazing outlooks for costs

reduction using AI apps

150 Billion$ saving thanks to top AI

applications by 2026 (Accenture)


The Iron Triangle in healthcare

In healthcare management, trying to

improve one factor usually harms

another…

More affordable, less effective

More effective, less accessible

…The Iron

Triangle

Access

Cost

Quality

AI appears as the short-circuit solution:

It can cut costs, improve treatment

quality and bolster accessibility


How is this possible?

Transferring time consumining

human tasks to machines

Enabling patients to self service

their care needs when possible

Objection 1: we are talking about

healthcare, I do not trust robots!

Objection 2: robot will steal our

jobs!


Breaking the Iron Triangle

What is really changing right now?

Most of the short-term investments are

about the operational/administrative

side rather than the clinical one;

The keyword of current solutions is:

Data-driven products and services

i.e. use data and previous experience

to improve quality, personalization and

reduce final costs

AI in Healthcare

Robotic surgery is shifting

towards automatic surgery

Nursing assistance

Administrative and

technical optimizations

Diagnostics


AI in Healthcare for diagnostic applications

AI diagnostics

Image based

Genomeinterpretation

Biomarkerdiscovery

Clinicaloutcomeprediction

Wearables

OftalmologyDermatology

RadiologyPathology

Identify gene mutations related to pathologies

Identify molecular patterns associated with

pathologies by merging information using ML

Predict hospital stay length,

readmission rate, mortality

AI for prevention: wearables

Movement (accelerometer,

gyroscope, magnetometer, GNSS)

Heart rate,

Respiratory rate,

Oxygen levels,

Blood pressure,

Body temperature,

Galvanic skin response, …

The vocabulary of measurements that

are possible via smart devices or

connected intrumentation is huge


Providing healthcare

before you knew you

even need it!

Do that ecologically, with

no time/space restrictions

AI for prevention: wearables

Providing healthcare before

you knew you even need it!

We can track health status and

flag abnormal events

We can identify changes

Early detection in ecological

settings


Evaluate the health

status ecologically

Wearables has no space limitations

Current golden standard requires

dedicated facilities and

instrumentation or invasive analysis

They are super accurate but do

they alter the quantity they are

measuring?


Evaluate the health

status ecologically

Wearables has no time limitations

They can be used to induce and

track long term behavior changes

They can be used to evaluate

alterations, building on previous

«history»

World Health Organization: 60% of related factors to individual health and quality of life are

correlated to lifestyle choices, including taking prescriptions such as blood-pressure medications

correctly, getting exercise, and reducing stress.


The Christmas tree effect

AI for prevention using wearables: limitations

The accuracy of wearable solutions

is often lower than that of classical

solutions

The user comfort is not always ideal,

and then the user’s adherence to

long term monitoring studies can

be limited

One third of US costumers who have owned wearable devices stopped using them within six

months of receiving them, foreshadowing the utility of the devices in fostering long term

behavioral change

[Source: Wearables are totally failing the people who need them most. Wired, June 2014]

AI for prevention using wearables

Search string «wearable AND prevention»

Numbers of studies targeting prevention using wearables are significantly

growing


Let us see an example in detail…

Wearable sensor systems

Motion sensors

Computational methods for human motion analysis

Movement assessment


Body Sensor

Networks

Activity

recognition

Activity

assessment

Alteration

recognition

A. Mannini, S. S. Intille, M. Rosenberger, A. M. Sabatini, W. Haskell, "Activity recognition using a single accelerometer placed at the

wrist or ankle", Medicine & Science in Sports & Exercise, vol. 45, no. 11, pp. 2193-2203, 2013.

Wrist Ambulation Cycling Other Sedentary

Ac

tua

l Ambulation 2121(87.2 %) 112 (4.6 %) 74 (3.0 %) 126 (5.2 %)

Cycling 86 (8.3 %) 650 (62.9 %) 16 (1.5 %) 281 (27.2 %)

Other 59 (6.2 %) 16 (1.7 %) 778 (81.6 %) 100 (10.5 %)

Sedentary 36 (1.2 %) 156 (5.2 %) 70 (2.3 %) 2722(91.2 %)

Overall accuracy = 84.7%

Ankle Ambulation Cycling Other Sedentary

Ac

tua

l

Ambulation 2481(99.5 %) 10 (0.4 %) 2 (0.1 %) 0 (0 %)

Cycling 13 (1.3 %) 975 (93.9 %) 21 (2.0 %) 29 (2.8 %)

Other 6 (0.6 %) 12 (1.2 %) 802 (81.6 %) 163 (16.6 %)

Sedentary 0 (0.0 %) 16 (0.5 %) 105 (3.5 %) 2879(96.0 %)

Overall accuracy = 95.0%

Ambulation Cycling Other activities Sedentary

walking. carrying-load

stairs: inside and down

stairs: inside and up

treadmill: 3 mph 0% incline





walking, natural

70rpm. 50W. 0.7kg

cycling outdoor level

cycling outdoor uphill

cycling outdoor downhill

painting: roller

painting: brush

sweeping with broom

sitting, internet search

sitting, computer typing

sitting: writing

sitting: reading

sorting files / paperwork

lying: on back

lying on left side

lying on-right-side

sitting: legs straight

standing still

Activity recognition

Features

evaluation

(time and

frequency

domain)

Classifiers

cross-

validation

(SVM +

leave-1-

subject-out)

Translate windowed accelerometer data in

activity information

2 h of annotated data for each participant

Leave-one-subject-out cross-validation

A. Mannini, M. Rosenberger, W. Haskell, A.M. Sabatini e S. S. Intille, Activity recognition in youth using a single accelerometer placed at the wrist or ankle, Medicine and Science in Sports and Exercise, 49 (4), 801-812, 2017.A. Mannini et al., A smartphone-based implementation based on the embedded accelerometer of this method is also available, Bodynets 2015.

It can be implemented on mobile devices

using their embedded sensors!Wrist activity recognition Accuracy

LOSO CV on Adults (original feature set) 84.7 🗸

LOSO CV on Youths (original feature set) 85.9 🗸

LOSO CV on Adults (new feature set) 87.0 🗸

LOSO CV on Youth (new feature set) 91.0 🗸

Crossed tests:

Training on Adults testing on Youths (original

feature set)

58.8 ✘

Training on Adults testing on Youths (new feature

set)

69.8 ✘

Training on Youths testing on Adults (original

feature set)

71.7 ✘

Training on Youths testing on Adults (new feature

set)

71.8 ✘

LOSO CV on Adults +Youths 88.5 🗸

(Youth data contribution only) 90.7 🗸

(Adult data contribution only) 86.7 🗸

Activity recognition

It works across different age groups

A. Mannini, V. Genovese and A. M. Sabatini, "Online decoding if hidden Markov models for gait event detection using foot-

mounted gyroscopes", IEEE Journal on Biomedical and Health Informatics, vol. 18, no. 4, pp. 1122-1130, 2014 .

It works on pathological gait too! (Mannini et al. IEEE-EMBC 2015)

Activity assessment: gait

Hidden Markov

Model (HMM)

Early detect gait alterations in everyday life

Healthy (EL) Post-stroke (PS) Choreic (HD)

Differentpathological

conditions alter differently the way

we move

A. Mannini, , O. Martinez-Manzanera, T.F. Lawerman, D. Trojaniello, U. Della Croce, D. Sival, N. Maurits and A.M. Sabatini, Automatic classification of gait in

children with early-onset ataxia or developmental coordination disorder and controls using inertial sensors, Gait & Posture, vol. 52, pp.287--292, 2017.


Healthy (EL)Post-stroke (PS)

Choreic (HD)

Model #1, EL group Model #2, PS group Model #3, HD group




Values in the upper part of the plots refers to relevant alterations

By observing classification outputs in relation to clinical scales we see that

misclassified patients are mild PS patients or severe HD patients. No

misclassification between healthy and pathological occurs

Parkinson’s Disease (PD), (1)

Palmerini et al, IEEE Trans Neur Sys Rehab Eng, 2013

Salarian et al, IEEE Trans Neur Sys Rehab Eng, 2010

TUG test

Since 2010-2011: correlations between measured gait and signal

parameters and clinical scales

Gait and TUG-test has been studied (Salarian et al, Weiss et al.)

Studies used single sensor solutions (lower back) or even more

complex setup with up to 7 inertial measurement units

Since 2013: Proper classification studies including PD-altered gait

recognition obtained accuracies >80% in discriminating between

PD and controls

Such studies used a single sensor (lower back, Palmerini et al

2013) or two shoe-fixed sensors (Klucken et al. 2013)

Large dataset evaluated (173 subjects in Klucken et al. 2013)

Klucken et al, PlosOne, 2013


Parkinson’s Disease (2)

Since 2012-2013: PD symptoms affecting gait were also studied.

Several methods to automatically recognize freezing of gait

and levodopa induced diskynesia were proposed

Studies adopted complex sensing setups

(6 sensors at wrists, waist, trunk and ankles)

Control populations were involved

very high accuracies in the recognition of

such symptoms were obtained (>88%)

The features adopted spanned from entropy

of signals to signal energy

Several automatic classifiers were compared in cross-

validation.

Tripoliti et al, Computer Methods

in Biomedicine 2013


Hemiplegic patients (HP)

Recognizing hemiplegic gait from controls is a much

simpler problem due to the asymmetry and highly

altered gait in most HP patients

Scheffer et al (2012) proposed a complex approach

involving a full Xsens MTV system (17 IMUs)

28 HP patients and 30 CTRL

24 features from gait temporal parameters and range

of motion, including left-right cross-correlations

Artificial Neural Network classification (no cross-

validation reported)

>99% recognition accuracy


Alzheimer’s Disease (AD)

A study by Hsu et al, (2014) started

the evaluation of variables extracted from

wearable sensors to identify AD symptoms

in gait automatically

A 3 IMUs system was proposed

21 AD and 50 CTRL tested

Features included postural sway parameters, gait temporal parameters and stride length

Tests included single and dual task gait (walking while counting down)

Most gait parameters differ across the two groups, especially in dual task gait

Hsu et al, IEEE J Biomed Health Inf,2014


Cognitive decline, frailty and fatigue

Frailty: Greene et al (2014) observed 184 non-frail, 185 pre-frail and 30 frail elderly doing a TUG test and

designed a frailty classifier based on logistic regression using 44 features (temporal gait params,

spatial gait params, angular velocity parameters, "turn" params) and including gender, age, heigth,

weigth. The study involved a 10-fold cross validation and achieved an accuracy of 78%

Cognitive decline: TUG test was also adopted by the same group (Greene and Kenny 2012) to

discriminate cognitive declined and intact subjects. 16 declined and 120 intact subjects were

tested achieving good classification performances. Discriminant analysis was done and a cross-

validation was performed.

Fatigue: Zhang et al (2014) used shank and sternum IMUs to discriminate normal and fatigue walk on

17 subjects. Stride-by-stride classification. 96% accuracy obtained, without running a proper cross-

validation.


Gait is the most important movement, it is related to main aspects of quality

of life and its assessment has main relevance for clinical reasons.

But we also want to have fun…

But… what about other movements?


Yoga & sports assessment

Recent collaboration with

TuringSense company and

Università di Roma Foro Italico

IMUs to recognize and analyze

movements and postures in sports

Recognize specific Yoga postures to

reduce errors in kinematic

estimation

Identify skill-related features in the

sport gesture

Fatigue assessment in soccer players&referees

Data acquisition involved referee from the

Florence section in Coverciano, April 2019

Data are being processed to extract fatigue

predictors

Target exercises were the Counter-movement

Jump and Gait

Other ongoing projects:

PRIN project that will start in late 2019:

TRAINED: "mulTifeature analysis of heaRt rate

variability and gaIt features in cliNical Evaluation of

Depression“, budget 450 k€ in 3 years.

PNRM project that will start in late 2019:

WAVE: “Wearable Assistant for VEterans in sport",

budget 1.2 M€ in 3 years.


So, keeping focused

on the main topic …


“A total of 1183 patients (51% response rate) were

enrolled between May and June 2018. Overall, 20%

considered that the benefits of technology (e.g.,

improving the reactivity in care and reducing the

burden of treatment) greatly outweighed the dangers.

Only 3% of participants felt that negative aspects

(inadequate replacement of human intelligence, risks

of hacking and misuse of private patient data) greatly

outweighed potential benefits.”

“We found that 35% of patients would refuse to

integrate at least one existing or soon-to-be

available intervention using biometric monitoring

devices and AI-based tools in their care.”


Conclusion

AI will be one of the keywords of

reserach, in healthcare too

(Horizon EU)

Wearables and personal devices

will play a significant role in our

daily monitoring for pre-diagnostics

Ethical concerns and user

perception are central


Thank you!2029 ?

potenzialità dell’intelligenza artificiale come · the biorobotics institute 2019 8 professors 3...

Documents