diem diatec 2014 · cv tools -recognition conceptual architecture and visual data set web dataset:...

Automatische

Kohlenhydratabschätzung:

Stand der Dinge

Peter Diem

Universitätspoliklinik für Endokrinologie,

Diabetologie und Klinische Ernährung

Inselspital, Universität Bern

Towards the Artificial PancreasARTORG: Diabetes Technology Research (Univ. Bern)

Glucose Insulin

2

Develop alternative, "learning" algorithms

• Prediction modules

• Taking in account changes of insulin sensitivity

• Improved bolus calculation

• Carbohydrate intake (food recognition)

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0%

80,0%

90,0%

+/- 10% +/- 20% +/- 30% +/- 40% +/- 50% > +/- 50

"Fehlschätzung"

���� - 2 E

���� + 2 E

5 E ���� 3.5 E

���� 6.5 E

6 E ���� 4.2 E

���� 7.8 E

7 E ���� 4.9 E

���� 9.1 E

8 E ���� 5.6 E

���� 10.4 E

9 E ���� 6.3 E

���� 11.7 E

3

Schätzung der angebotenen Kohlenhydrate bei T1DM

GoCARB Project

Captured Image

Food type Volume Carbs

Breaded 110 ml 11 g

Rice 120 ml 16.4 g

Salad 80 ml 2.6 g

3D Model

80 ml

120ml

110 ml

Segmentation

CHO Estimation

Recognition

SaladRice

Breaded

Bolus Calculator

4

CV Tools - SegmentationPyramidal mean-shift filtering

Mean-shift filtering• Clusters pixels which are close both in terms of color and spatial distance

• The Perceptually uniform CIELAB color space is used

• The used distance weakens the significance of Lightness - eliminates the shadow effect

• Smoothes the fine-grain texture without losing the dominant color edges

5

CV Tools - RecognitionConceptual architecture and visual data set

Web Dataset: for training and testing the food recognition module.�3500 images belonging to 10 classes were collected and annotated

Carrot Salad Breaded Red Sauce Bread Rice

Green Salad Pasta Meat Potatoes Legumes

6

CV Tools - RecognitionImage segment description - Classification

2

4

1

3

1

23

4Breaded

SVM Classification

Training Images Feature space

Describe

Tra

inin

gTe

stin

g

Describe

Salad RiceBreaded

Training of SVM Classifier

Color/texturefeatures

Color/texturefeatures

Breaded

7

CV Tools - RecognitionEvaluation

Confusion matrix

A cross-validation approach

with 10 folds was adopted.

The overall recognition

accuracy was 87%.

8

Volume estimation

9

Reference object:• Credit card size• Strong texture• Easily matched

Volume estimation3D shape reconstruction

Rectification and stereo matching

3D shape dense reconstruction

Two input images

Features extraction and matching

Relative camera pose estimation

Image size – 1MPx, computational time – less than

12 seconds on desktop with i7 CPU

10

Volume estimation

11

Integrated system

Captured Image

Food type Volume Carbs

Breaded 110 ml 11 g

Rice 120 ml 16.4 g

Salad 80 ml 2.6 g

3D Model

80 ml

120ml

110 ml

Segmentation

CHO Estimation Recognition

SaladRice

Breaded

Bolus Calculator

smartphone server side (laptop)

Plate Detection

12

Volume estimationEvaluation with dummy food items

Several dummy foods with known volume and CHO content were

placed separately and in combinations on the plate

Food items

Volume(ml)Absolute error in CHO

estimation (g)Real Computed

mean ± std

125 131+ 15 2.7

375 373 + 21 2.9

150 179+ 24 8.1

210 223+ 19 2.4

131 145+ 18 3.2

88 79+ 9 1.9

302 312+ 16 1.7

320 305+ 31 3.6

75 72+ 8 0

Mashed potatoes

Spaghetti with tomato sauce

Wholemealbread roll

Danish pastry

Boiled potatoes

Slice of toasted bread

Muffin

Sandwich

Chicken thigh

13

Volume estimationEvaluation with real food items (preliminary results)

Food items

Volume(ml)Absolute error in CHO

estimation (g)Real Computed

mean ± std

790 823 ± 95 14.4

712 700± 63 8.6

462 501± 85 7.6

14

Integrated systemFuture work

smartphoneserver side

Data Transfer

Update

15

Clinical evaluationDesign – Pilot study (without bolus estimator)

• 20 patients

• cross over design (see attachment)

• patients use the GoCARB-application

during 3 months and adjust the prandial

insulin according to the carbohydrate

estimation of the GoCARB-application

• Frequent patient monitoring

1. Primary outcome: definition of the

variability (get numbers to conduct a

power analysis for a clinical trial)

2. Secondary outcomes: AUC, postprandial

blood glucose concentration,

carbohydrate counting accuracy,

diabetes specific quality of life…

3. Practicability

4. Safety

16

17

AP

GoCARB team

University of Bern

Agbesi Prosper

Anthimopoulos Marios

Botwey Ransford

Cann Keith

Daskalaki Elena

Dehais Joachim

Diem Peter

Gianola Lauro

Loher Hannah

Mougiakakou Stavroula

Scarnato Luca

Shevchik Sergey

Stettler Christoph

Wenger Christine

Züger Thomas

Roche Diagnostics Operations IncDuke David

Greenburg Alan

Soni Abhishek

Weinert Stefan

Roche Diagnostics GmbHGerber Martin

Lodwig Volker

Weissmann Joerg

22 scientists, 9 nationalities, 3 partners

18

Fragen? Alles klar?

Fragen? Alles klar?