alex edgcomb frank vahid university of california, riverside department of computer science
DESCRIPTION
Feature Extractors for Integration of Cameras and Sensors during End-User Programming of Assistive Monitoring Systems. ?. Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science. Motion sensor. Sensors and actuators in MNFL [1] for end-user programming. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/1.jpg)
Feature Extractors for Integration of Cameras and Sensors during
End-User Programming of Assistive Monitoring Systems
Alex EdgcombFrank Vahid
University of California, RiversideDepartment of Computer Science
1 of 16
?Motion sensor
![Page 2: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/2.jpg)
Sensors and actuators in MNFL [1] for end-user programming
Alex Edgcomb, UC Riverside 2 of 16
“Person at door”
LED lights in house
“Person at door”
Outdoor motion sensor
Doorbell
• Assistive monitoring• User customizability essential [2][3]
[1] Edgcomb, A. and F. Vahid. MNFL: The Monitoring and Notification Flow Language for Assistive Monitoring. Proceedings 2nd ACM International Health Informatics Symposium, 2012. Miami, Florida.[2] Philips, B. and H. Zhao. Predictors of Assistive Technology Abandonment. Assistive Technology, Vol. 5.1, 1993, pp. 36-45.[3] Riemer-Reiss, M. Assistive Technology Discontinuance. Technology and Persons with Disabilities Conference, 2000.
![Page 3: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/3.jpg)
Porch light
LED lights in house
Expanding the previous example
Alex Edgcomb, UC Riverside 3 of 16
“Person at door”
“Person at door”
Outdoor motion sensor
Doorbell
Light sensor
![Page 4: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/4.jpg)
Webcams are cheap
4 of 16Alex Edgcomb, UC Riverside
![Page 5: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/5.jpg)
Webcams can do more than sensors
Fall down at home
In room for extended time
Can do same as some sensors
Motion sensorLight sensor
5 of 16Alex Edgcomb, UC Riverside
Identify personat front door
![Page 6: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/6.jpg)
Problem: Integration of webcams and sensors
6 of 16
Homesite
Commercial approach:
Alex Edgcomb, UC Riverside
?
Outdoor motion sensor
![Page 7: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/7.jpg)
Solution: Feature extractor
7 of 16
92Integer stream
output
0
100
Alex Edgcomb, UC Riverside
Extract some feature
Video stream input
![Page 8: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/8.jpg)
Identify person at door in MNFL
Alex Edgcomb, UC Riverside 8 of 16
Outdoor motion sensor
![Page 9: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/9.jpg)
Person in room for extended period of time in MNFL
9 of 16Video’s YouTube link
Alex Edgcomb, UC Riverside
![Page 10: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/10.jpg)
Many feature extractors are possible
10 of 16Alex Edgcomb, UC Riverside
![Page 11: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/11.jpg)
Are feature extractors usable by lay people? Two usability trials.
• 51 participants• Trials required as 1st lab assignment• Non-engineering/non-science students at UCR
1 2 3 4 50
10
20
30
40
50
60
70
80
90
100
Programming experience(1- little/none; 5- a lot)
Perc
enta
ge o
f par
ticip
ants
11 of 16Alex Edgcomb, UC Riverside
![Page 12: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/12.jpg)
Participant reference materials
• One-minute video showing how to spawn and connect blocks.
• Overview picture
12 of 16Alex Edgcomb, UC Riverside
![Page 13: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/13.jpg)
Example challenge problem
13 of 16Alex Edgcomb, UC Riverside
actual participant solution
![Page 14: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/14.jpg)
Trial 1: Increasingly challenging feature extractor problems
25 participants14 of 16
10 9 7-8 5-6 3-4 1-2 00%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Fall detector (easy) Visitor detector (medium)
Leave at night detector (hard)
Rubric score
Perc
ent o
f sub
ject
s
Alex Edgcomb, UC Riverside
![Page 15: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/15.jpg)
Trial 2: Feature extractor vs logic block
26 participants15 of 16Alex Edgcomb, UC Riverside
10 9 7-8 5-6 3-4 1-2 00%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Visitor detector (feature)
Emergency button (logic)
Rubric score
Perc
enta
ge o
f sub
ject
s
![Page 16: Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e8550346895ddb29ab/html5/thumbnails/16.jpg)
Conclusions
• Feature extractors– Elegant integration of cameras and sensors– Quickly learnable by lay people
• Future work– Develop additional feature extractor blocks– Trade-off analysis between privacy,
communication, and computation16 o f 16Alex Edgcomb, UC Riverside