diem diatec 2014 · cv tools -recognition conceptual architecture and visual data set web dataset:...
TRANSCRIPT
Automatische
Kohlenhydratabschätzung:
Stand der Dinge
Peter Diem
Universitätspoliklinik für Endokrinologie,
Diabetologie und Klinische Ernährung
Inselspital, Universität Bern
Towards the Artificial PancreasARTORG: Diabetes Technology Research (Univ. Bern)
Glucose Insulin
2
Develop alternative, "learning" algorithms
• Prediction modules
• Taking in account changes of insulin sensitivity
• Improved bolus calculation
• Carbohydrate intake (food recognition)
0,0%
10,0%
20,0%
30,0%
40,0%
50,0%
60,0%
70,0%
80,0%
90,0%
+/- 10% +/- 20% +/- 30% +/- 40% +/- 50% > +/- 50
"Fehlschätzung"
���� - 2 E
���� + 2 E
5 E ���� 3.5 E
���� 6.5 E
6 E ���� 4.2 E
���� 7.8 E
7 E ���� 4.9 E
���� 9.1 E
8 E ���� 5.6 E
���� 10.4 E
9 E ���� 6.3 E
���� 11.7 E
3
Schätzung der angebotenen Kohlenhydrate bei T1DM
GoCARB Project
Captured Image
Food type Volume Carbs
Breaded 110 ml 11 g
Rice 120 ml 16.4 g
Salad 80 ml 2.6 g
3D Model
80 ml
120ml
110 ml
Segmentation
CHO Estimation
Recognition
SaladRice
Breaded
Bolus Calculator
4
CV Tools - SegmentationPyramidal mean-shift filtering
Mean-shift filtering• Clusters pixels which are close both in terms of color and spatial distance
• The Perceptually uniform CIELAB color space is used
• The used distance weakens the significance of Lightness - eliminates the shadow effect
• Smoothes the fine-grain texture without losing the dominant color edges
5
CV Tools - RecognitionConceptual architecture and visual data set
Web Dataset: for training and testing the food recognition module.�3500 images belonging to 10 classes were collected and annotated
Carrot Salad Breaded Red Sauce Bread Rice
Green Salad Pasta Meat Potatoes Legumes
6
CV Tools - RecognitionImage segment description - Classification
2
4
1
3
1
23
4Breaded
SVM Classification
Training Images Feature space
Describe
Tra
inin
gTe
stin
g
Describe
Salad RiceBreaded
Training of SVM Classifier
Color/texturefeatures
Color/texturefeatures
Breaded
7
CV Tools - RecognitionEvaluation
Confusion matrix
A cross-validation approach
with 10 folds was adopted.
The overall recognition
accuracy was 87%.
8
Volume estimation
9
Reference object:• Credit card size• Strong texture• Easily matched
Volume estimation3D shape reconstruction
Rectification and stereo matching
3D shape dense reconstruction
Two input images
Features extraction and matching
Relative camera pose estimation
Image size – 1MPx, computational time – less than
12 seconds on desktop with i7 CPU
10
Volume estimation
11
Integrated system
Captured Image
Food type Volume Carbs
Breaded 110 ml 11 g
Rice 120 ml 16.4 g
Salad 80 ml 2.6 g
3D Model
80 ml
120ml
110 ml
Segmentation
CHO Estimation Recognition
SaladRice
Breaded
Bolus Calculator
smartphone server side (laptop)
Plate Detection
12
Volume estimationEvaluation with dummy food items
Several dummy foods with known volume and CHO content were
placed separately and in combinations on the plate
Food items
Volume(ml)Absolute error in CHO
estimation (g)Real Computed
mean ± std
125 131+ 15 2.7
375 373 + 21 2.9
150 179+ 24 8.1
210 223+ 19 2.4
131 145+ 18 3.2
88 79+ 9 1.9
302 312+ 16 1.7
320 305+ 31 3.6
75 72+ 8 0
Mashed potatoes
Spaghetti with tomato sauce
Wholemealbread roll
Danish pastry
Boiled potatoes
Slice of toasted bread
Muffin
Sandwich
Chicken thigh
13
Volume estimationEvaluation with real food items (preliminary results)
Food items
Volume(ml)Absolute error in CHO
estimation (g)Real Computed
mean ± std
790 823 ± 95 14.4
712 700± 63 8.6
462 501± 85 7.6
14
Integrated systemFuture work
smartphoneserver side
Data Transfer
Update
15
Clinical evaluationDesign – Pilot study (without bolus estimator)
• 20 patients
• cross over design (see attachment)
• patients use the GoCARB-application
during 3 months and adjust the prandial
insulin according to the carbohydrate
estimation of the GoCARB-application
• Frequent patient monitoring
1. Primary outcome: definition of the
variability (get numbers to conduct a
power analysis for a clinical trial)
2. Secondary outcomes: AUC, postprandial
blood glucose concentration,
carbohydrate counting accuracy,
diabetes specific quality of life…
3. Practicability
4. Safety
16
17
AP
GoCARB team
University of Bern
Agbesi Prosper
Anthimopoulos Marios
Botwey Ransford
Cann Keith
Daskalaki Elena
Dehais Joachim
Diem Peter
Gianola Lauro
Loher Hannah
Mougiakakou Stavroula
Scarnato Luca
Shevchik Sergey
Stettler Christoph
Wenger Christine
Züger Thomas
Roche Diagnostics Operations IncDuke David
Greenburg Alan
Soni Abhishek
Weinert Stefan
Roche Diagnostics GmbHGerber Martin
Lodwig Volker
Weissmann Joerg
22 scientists, 9 nationalities, 3 partners
18
Fragen? Alles klar?
Fragen? Alles klar?