mobilizing mhealth: interdisciplinary computer science and engineering

eric c. larson | eclarson.com

interdisciplinary computer science and engineering mobilizing mhealth

Assistant Professor Computer Science and EngineeringWednesday, September 11, 13

databases

algorithms

arch

AI

OS

CompSci

Wednesday, September 11, 13

mobile

databases

algorithms

arch

AI

OS

CompSci


mobile

databases

algorithms

arch

AI

OS

CompSci

mhealth


mobile

databases

algorithms

arch

AI

OS

CompSci mhealth


what is mhealth?


stress check

glucose buddyfitness trainer

heart rate

zombie run


consider physician’s need


compliance?

cost?

doctor patient?


cost


compliance


what can the mobile phone already sense ubiquitously?


accelerometergyroscopemagnetometer /compassdual camera / flash1+ microphonesproximity sensorcapacitive sensorgpsmotorized actuator


accelerometergyroscopemagnetometer /compassdual camera / flash1+ microphonesproximity sensorcapacitive sensorgpsmotorized actuator

compliance++;cost--;dr_pat *= 10;


what can the mobile phone sense with clinical utility?


futureresearch

lungfunction jaundice


spirometer lung function??Wednesday, September 11, 13

spirometer

device that measures amount of air inhaled and

exhaled.


lung function

asthmaCOPDcystic fibrosis

evaluates pulmonary impairments


using a spirometer

flow

volume

volum

e

timeWednesday, September 11, 13

volume-time graphvo

lume

time


volume-time graphvo

lume

time

FEV1

FVC

FEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity


volume-time graphvo

lume

time1 sec.

FEV1

FVC



volume-time graphvo

lume

time1 sec.

FEV1

FVCFEV1% = FEV1/FVC




FEV1% = FEV1/FVC



FEV1% = FEV1/FVC

> 80% healthy60 - 79% mild40 - 59% moderate

< 40% severe


flow-volume graphflo

w

volume


flow

volumeFEV1 FVC

1 sec.

PEF

PEF: Peak Expiratory FlowFEV1: Forced Expiratory Volume in 1 secondFVC: Forced Vital Capacity

flow-volume graph


flow

volume

normal

flow-volume graph


flow

volume

normalobstructive

flow-volume graph


obstructive diseases

resistance in air path leads to reduced air flow


restrictive diseases

lungs are unable to pump enough air and pressure


flow-volume graphFlo

w

Volume

normalobstructive


flow-volume graphFlo

w

Volume

normal

restrictiveobstructive


clinical spirometry


home spirometry


home spirometry

faster detectionrapid recovery

trendingWednesday, September 11, 13

home spirometry

high cost barrierpatient compliance

less coachinglimited integration

challenges with


flow ratevolume

lung functionairflowsensor


flow ratevolume


soundpressure microphone


flow ratevolume


soundpressure microphone processing

estimated


SpiroSmart

availabilitycostportabilitymore effective coaching interfaceintegrated uploading


Using SpiroSmart


Using SpiroSmart

]


Using SpiroSmart

]Wednesday, September 11, 13

Using SpiroSmart


study design

x 3

x 3Wednesday, September 11, 13

study enrollment


study enrollment

participants 5218-75 years old, mostly healthy18-75 years old, mostly healthy

study a


study enrollment


study a

participants 1012-17 years old, mixed healthy/abnormal12-17 years old, mixed healthy/abnormal

study b


study enrollment


study a

participants 1012-17 years old, mixed healthy/abnormal12-17 years old, mixed healthy/abnormal

study b

participants 5610-69 years old, mostly abnormal10-69 years old, mostly abnormal

study c

enrolled bypulm

onologists


audio


audio

flow features


audio

flow features

measuresregression

FEV1FVCPEF


0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measuresregression

curveregression

FEV1FVCPEF


0 1 2 3 40

5

10

15

Flow

(L/s

)

Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

0 1 2 3 40

5

10

15

Flow

(L/s

)Volume(L)

0 2 4 6 8 100

1

2

3

4

time(s)

Volu

me(

L)

audio

flow features

measuresregression

curveregression

lung functionFEV1FVCPEF


0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features


0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

vocal tractsource output

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500


0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

flow features

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

envelope detection

0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1

time(s)

amplitude

resonance tracking0 1 2 3 4 5 6 7

−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

flow estimation features

vocal tractsource output 0 1 2 3 4 5 6 7−1

−0.5

0

0.5

1lpc8raw

time(s)

amplitude

auto-regressive estimate

time(s)

frequency(Hz)

1 2 3 4 5 60

500

1000

1500

2000

2500


curveregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature N

window

ed machine

learning regression

...


−1 0 1 2 3 4 50

2

4

6

8

time(s)

flow(L/s)

curveregression

−1 0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

0 1 2 3 4 50

0.1

0.2

0.3

0.4

time(s)

feat

ure

valu

e

feature 1

feature N

curveoutput

window

ed machine

learning regression

...


resultsmeasuresregression


curveregression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


curveregression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


curveregression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


curveregression

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−2 0 2 4 60

5

10

15

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)

−1 0 1 2 3 40

2

4

6

8

volume(L)

flow(L/s)


can SpiroSmart curves be used for diagnosis?


survey


• 10 subjects curves

survey


• 5 pulmonologists


survey


• 5 pulmonologists


• unaware if from SpiroSmart / spirometer

survey


results

normalminimal obstructivemild obstructivemoderate obstructivesevere obstructive

restrictive

inadequate


results


restrictive

inadequate

identical64%


results


restrictive

inadequate

one off10%

identical64%


results


restrictive

inadequate false positive14%

one off10%

identical64%


results


restrictive

inadequate

false negative4%

false positive14%

one off10%

identical64%


results


restrictive

inadequate

error8%false negative

4%

false positive14%

one off10%

identical64%


results


restrictive

inadequate

error8%false negative

4%

false positive14%

one off10%

identical64%

abnormal vs normal96%


appropriate for screening and early diagnosis


appropriate for screening and early diagnosis

in the home? in the developing world?


in 10 years, COPD will surpass AIDS/HIV as cause of death in developing nations


futureresearch



neonatal jaundice in the US

kernicterus: 21

hazardous jaundice: 1158

extreme jaundice: 2,317

severe jaundice: 35,000

phototherapy: 290,000

visible jaundice: 3.5 million

births/year: 4.1 million


Method Accuracy Disadvantages

TSB Gold standard (r=1.0)

Painful, costly, inconvenient, delayed

TcBAccurate

(r= 0.75 -0.93)Meter = $7000

tips $5 unavailable in most

physician offices

Visual assessment (provider or parent)

not accurate (r= 0.36 - 0.7), underestimates

severity

No standardization lighting, pigmentation


bilicam


participants 400-4 days old, collected in nursery0-4 days old, collected in nursery

study a


Color Linearization

• Camera Settings Adjustment

• Light Source Estimation

Image Segmentation

• Quality Control for Distance, Lighting, and Shadow

• Sternum, Forehead, Card Segmented

Color Calibration

• Dynamic Least Squares Regression

• Automatic Feature Selection

Neonatal Skin Response to Bilirubin

• Skin Independent Color Transformations Applied

• Multivariate Machine Learning Regression

bilicam


05

10

15

TcB

va

lues

0 5 10 15bilicam values

Fitted values bilicam

biliru

bin

level

bilicam estimation0

15

155 10

5

10

mg/dlr=0.91

bilicam initial results


• near term: screening


• near term: screening• medium term: TSB study


• near term: screening• medium term: TSB study• long term: developing world


• near term: screening• medium term: TSB study• long term: developing world

“in many resource poor nations, hyperbilirubinemia is the second or

third leading cause of infant mortality and disability”


futureresearch



oxygen volume, VO2


blood pressureWednesday, September 11, 13

intra ocular pressure


activity detectionelder care

daily activity logscongestive heart failure

health markers via phoneblood pressurepulse oximetrystress

opportunistic sensingpain managementdetecting circulation


activity detectionelder care

daily activity logscongestive heart failure

health markers via phoneblood pressurepulse oximetrystress

opportunistic sensingpain managementdetecting circulation

developing worldWednesday, September 11, 13

take away

• CSE has a large role in the coming evolution of

mHealth, equal to doctors

• sensing and clinical utility

• deploy in clinic, deploy in world, then release

• ... look for a new mobile application sensing

course ...


eric c. larson | eclarson.com

interdisciplinary computer science and engineering mobilizing mhealth

Assistant Professor Computer Science and Engineering

eclarson.com [email protected]@ec_larson Thank You!


mailto:[email protected]

mailto:[email protected]

mobilizing mhealth: interdisciplinary computer science and engineering

Technology