lab-forming fields and field-forming labs
TRANSCRIPT
国立研究開発法人
Lab-Forming Fields (LFF)and
Field-Forming Labs (FFL)Takeshi Kurata1, 2
1 Human Informatics Research Institute, AIST, Japan
2University of Tsukuba, JapanE-mail: [email protected]
1
Takeshi Kurata, Ph.D.• Position:
– Research Group Leader, Service Sensing, Assimilation, and Modeling Research Group, Human Informatics Research Institute, AIST
– Professor (Cooperative Graduate School Program), Faculty of Engineering, Information and Systems, University of Tsukuba
• Professional Experience:– 2011‐2014 Doctoral co‐supervisor, Joseph Fourier University, UJF‐
Grenoble 1, France– 2012‐ ISO/IEC JTC 1/SC 24 Member– 2003‐2005 Visiting Scholar, HIT Lab, University of Washington
• Education:– 2007 Ph.D. (Eng.) from Doctoral Program in Graduate School of
Systems and Information Engineering, University of Tsukuba– 1996 M.E. from Doctoral Program in Engineering, University of Tsukuba
• Research Interests:– Service Research, Assistive technology, Wearable/Pervasive Computing,
Mixed and Augmented Reality, Computer Vision2
AIST http://www.aist.go.jp/
PresidentDr. Ryoji Chubachi
AIST, Tsukuba1 h drive from Tokyo
National Institute of Advanced Industrial Science and Technology
• One of the largest national institute in Japan– The independent agency of the Ministry of Economy, Trade and
Industry– The mission of AIST is advanced research and development for
industry– Over 2,300 permanent researchers– Over 50 research units cover various research fields
3
Research Fields and Staffs of AIST
4
Human Informatics Research Inst.
• History– Established in April, 2015– Main department is located in Tsukuba
• Organization– 85 permanent researchers– 10 research groups
• Brain science• Human factors engineering• Digital human modeling• Service engineering
5
Framework of Human Informatics
Human and Society Service
Presentation
AnalysisUnderstanding
Sensing
ParticipationSocial cognition
HealthcareWellness
SafetyComfort
Deep Data (High Quality Reference Data)
Big Data
ITIoT/Cyber-Physical
6
国立研究開発法人
Comb Data: Big + Deep in LFF & FFL
7
SFS
Dollhouse VR
CCE Lite
WearableRGB-D sensing
PDR
Handheld AR
Result/Behavior/Environment and LFF/FFL8
国立研究開発法人
Lab-Forming Fields & Field-Forming Labs
• Borrowing from “Terraforming”• Lab-forming Field: Transforming a real
field into a lab-like place. (IoT/G-IoT)• Field-forming Lab: Transforming a
laboratory into a field-like place. (VR)9
国立研究開発法人
Service design loop
10
国立研究開発法人 11
測って 図るHakatte Hakaru
MeasureWeighSurvey
PlanDesignAttempt
国立研究開発法人
測って図るHakatte Hakaru
12
ASPR Technologies for Multi-Stakeholders
13
国立研究開発法人
Efficient interactive label attaching for supervised Service Operation Estimation
14
So many kinds of positioning methods
15
PDR(Pedestrian Dead-Reckoning)Estimates velocity vector, relative altitude, and action type by measurements from a wearable sensor module.
Wearing a sensor module on waist (2D SHS (Steps and Heading Systems) PDR) Easy to wear and maintain Easy to measure data for action recognition Relatively easily apply for handheld setting compared to shoe-mounted PDR
(3D-INS (Inertial Navigation System) PDR)
16
Handheld PDR From PDR to PDRplus
10-axis sensors• Accelerometers• Magnetic sensors• Gyro sensors• Barometer
Shoe-mounted PDR
Waist-worn PDR
AR by PDR + Image registration(1999-2003)
Panorama-based Annotation: IWAR1999, ISWC2001,
ISMAR2003
G
Environmental mapA
B C D
E
A
B
C
F
Input frames
Position at whicha panorama is taken
PositionDirection
235 [deg]
5 [deg]From the user’s camera
Located Orientated
17
Frontier of PDR: Walking direction estimation
18
• Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.
Frontier of PDR: Walking direction estimation
19
• Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.• Long Paper: Christophe Combettes, Valerie Renaudin, Comparison of Misalignment
Estimation Techniques Between Handheld Device and Walking Directions, IPIN 2015.• FIS was proposed by Kourogi and Kurata in PLANS 2014.
“Globally, the FIS method provides better results than the other two methods.”
Frequency analysis of Inertial Signals
Forward and Lateral Acc. Modeling
Principal Component Analysis
Overview: History of our PDR20
ISWC2001
IWAR1999
ISMAR2003
PLANS2014
PLANS2010 ICServ2013
Docomo map navi(500 areas as of March, 2017))
Image registration + Gyro
Panorama-based annotation (Image-registration-based positioning)
Image registration + PDR
PDRplus (PDR + Action recognition)
Handheld PDR(Walking-direction estimation)
2015- 2015-
PDR module
2011-
Academia
Industry
Before PDR
ICAT2006 PDR + GPS + RFID
Global Trend on PDRPDR R&D players have rapidly indicated their presence all over the world on and after 2010.
Movea (France)
Sensor Platforms (USA)
CSR (UK)
TRX (USA)
Trusted Positioning (Canada)
21
Acquired by QualcommAcquired by InvenSenseAcquired by InvenSense
Acquired by Audience
Indoo.rs (USA)
SFO
Standardization on PDR Benchmarking• PDR related R&D is highly active worldwide: Necessity for sharing
common measures.• Description of the performance should be unified in spec sheets
and scientific papers.• Different measures from absolute positioning methods such as
GNSS, Wi-Fi, and BLE are required for PDR, which is a method of relative positioning.
• PDR Benchmark Standardization Committee was established in 2014 as a platform of the grassroots activity.
22
https://www.facebook.com/pdr.bms
Support Organizations• Asahi Kasei Corporation, Asia Air Survey Co., Ltd. (Y. Minami), INTEC Inc.,
MTI Ltd., KDDI R&D Laboratories, Inc., KOKUSAI KOGYO CO., LTD.,SHIBUYA KOGYO CO., LTD., Koozyt, Inc., GOV Co., Ltd., SITESENSING, inc., Sharp Corporation, Sugihara Software and Electron Industry Co., Ltd. (SSEI), ZENRIN DataCom CO., LTD., Information Services International-Dentsu, Ltd. (ISID), Hitachi, Ltd., IBM Japan, Ltd., Frameworx, Inc. (S. Watanabe), MULTISOUP CO.,LTD., Milldea, LLC, Murata Manufacturing Co., Ltd., MegaChips Corporation, Recruit Lifestyle Co., Ltd. (K. Ushida), RICOH COMPANY, LTD., Rei-Frontier Inc.,
• Aichi Institute of Technology (K. Kaji), NARA Institute of Science and Technology (NAIST) (I. Arai), Kanagawa Institute of Technology (H. Tanaka), Keio University (S. Haruyama, N. Kohtake, M. Nakajima), University of Tsukuba (T. Kurata), Tokyo Institute of Technology (S. Okada), Nagoya University (N. Kawaguchi), Niigata University (H. Makino), Ritsumeikan University (N. Nishio), National Institute of Advanced Industrial Science and Technology (AIST) (T. Kurata, M. Kourogi), Human Activity Sensing Consortium (HASC), Location Information Service Research Agency (LISRA)
• 36 organizations in Japan as of March, 2017
23
Scene in data collection25
PDR Challenge Series• Ubicomp/ISWC 2015 PDR Challenge
– Scenario: Indoor Navigation– On-site– Continuous walking while keeping watching the
navigation screen by holding the smartphone– Several minutes per trial
• IPIN 2017 PDR Challenge in Warehouse Picking– Scenario: Picking work in a warehouse– Off-site– Not only walking but various actions including picking
and carrying– Several hours per trial
26
IPIN2017
27
• User Requirements• Hybrid IMU Pedestrian Navigation & Foot Mounted
Navigation• Human Motion Monitoring• High Sensitivity GNSS, Indoor GNSS, Pseudolites• RTK GNSS with handheld devices• Mitigating GNSS errors prior to moving indoors• Self-contained sensors• Signal Strength Based Methods, Fingerprinting• UWB (Ultra-wideband)• Passive & Active RFID• Optical Systems• Ultrasound Systems• TOF, TDOA based Localization• Localization, Algorithms for Wireless Sensor Networks• Frameworks for Hybrid Positioning• Industrial Metrology & Geodetic Systems, iGPS• Radar Systems• Mapping, SLAM• Indoor Spatial Data Model & Indoor Mobile Mapping• Novel uses of maps and 3D building models• Magnetic Localization• Innovative Systems• Location Privacy• Applications of Location Awareness & Context
Detection• Health and Wellness ApplicationsRegular Papers Due: April 30, 2017
Integrated Positioning (SDF)
28
Sub-meter indoor positioning: Visible Light Communication (VLC) & PDR
• Less density of infrastructure installation by SDF combining VLC and PDR
• Reduction of initial/running cost of sensing by Replacement demand of lighting
29 Collaboration with Panasonic
RGBD (Depth) sensor & PDR• Error compensation of PDR with precise
trajectories obtained from surveillance (RGBD) cameras
• Coverage compensation of surveillance cameras with continuous measurement of PDR
30
国立研究開発法人
VDR (Vehicle/Vibration-based DR)
31
国立研究開発法人
Whole-body posture estimation and precise positioning
32
Many sensors for heterogeneous and more precise real-world capturing (position, orientation, posture, physical load, etc.) and deep-data gathering
CSQCC (Computer-supported QC Circle)
33 Staying-time rate at each dinning area per personSales at each dinning area per employee
Visualization tool combining human-behavioral and accounting history
Employee taking order while cleaning up the
guest room
Icons showing the number of customers at each table
POS data log
Service Characteristics1. Intangible2. Heterogeneous3. Inseparable4. Perishable
Alleviate the issues due to IHIP
QCC in manufacturing industryPurpose: Productivity improvement
Conventional QCC in service industryPurpose: Productivity improvement
Subjective QCC in service industryPurpose: Improvement of CS/ES
w/ reasonable ways to gather objective data in plants
In 1980s, applying QCC for service industry
w/o reasonable ways to gather objective data in service fields
In 1990s, Service industry lost interest in QCC
In 2000
QCC in the Service Industry in Japan
34
Computer-supported QCC (CSQCC)Purpose: Productivity improvement
In 2010
CSQCC in the futureProductivity improvementImprovement of CS/ES
w/ reasonable ways to gather subjective data continuouslyw/ reasonable ways to
gather objective data in service fields
1950~ Deming Award
3rd CSQCC for newly open
35
Yamano Aiko-tei, Shinjuku: Mansion style restaurant
(2014.12.23)
2014.10.18 (Sat) 2014.11.08 (Sat)
Case studyin Japanese Restaurant “Ganko”
• Objectives1. (for AIST) to test the CSQCC
(Computer-Supported QCC) suites in a real service field.
2. (for the restaurant) to observe effects of process improvement planned by CSQCC.
• Place– Japanese cuisine restaurant
GANKO Ginza 4-chome (Tokyo)
• Term– 1st term
• January 12 to 18, 2011– 2nd term
• February 3 to 9, 201136
Dining area Course dishes
1st term(Jan. 12-18, 2011)
for observing ordinary operations
QC circlefor making improvement plans
2nd term (Feb. 3-9, 2011)for observing improved operations
37 B2
B1
Dinning Area
Kitchen
Office room
Pantry
During Discussion in CSQCC38
Trajectory of a wait staff in lunch time: 12:00-14:00
Fact: Going in and out of the kitchen/office to no small extent.Possible result: Difficulty in concentrating on guest service.Cause: Cell phone everywhere, but reservation book only in the office room.Possible improvement: e-reservation book
Dinning Area
Kitchen
Office room
T. Fukuhara, R. Tenmoku, T. Okuma, R. Ueoka, M. Takehara, and T. Kurata, "Improving Service Processes based on Visualization of Human-behavior and POS data: a Case Study in a Japanese Restaurant“, ICServ2013, pp.1-8.
Summary of 1st CSQCC for Wait Staff
39
Grasp of actual condition Shorter stay in dinning area than the manager assumed
Kaizen plan development (1) Re-composition of service processes (SP)(2) Thoroughly obeying each division’s roll, (3) Guts
Direct effect Stay ratio in dinning area at dinner time: UP ↑Spillover effect Number of additional orders at dinner time: UP ↑
Side effect(Trade-off)
(1) Work load (walking distance): No difference →(2) Number of additional orders at 3pm: No difference →
Stay ratio in dinning areas
30%
35%
40%
45%
50%
55%
11 12 13 14 15 16 17 18 19 20 21 22
Walking Distance [m]
1,000
1,500
2,000
2,500
11 12 13 14 15 16 17 18 19 20 21 22Num. of additional orders per customer
0.0
0.4
0.8
1.2
11 12 13 14 15 16 17 18 19 20 21 22Hour Hour Hour
BeforeAfter
Down: Due to SP re-comp. for preparation
of dinner/partyUP: Much more than time
decreased in Tea hour
No diff.: Due to no SP re-comp.
No diff.: Despite SP re-comp. for preparation of dinner/party
UP: due to reduction of opportunity loss
No diff. on workload
Lunch Tea Dinner Lunch Tea Dinner Lunch Tea Dinner
Walk distance of waiting staff per customer (meters / hour / person)
40
***
* p < .05, ** p < .01, *** p < .001
******
They were able to reduce walking distance while not reducing staying time in the dining area!
Indicators for position keeping
41
B2
B1
Zone Dedication Rate=Orange/RedZone Order Defense Rate =Orange/Blue
All of orders in the staffʼs zone
# of accepted orders by a staff in the staffʼs zone
The total # of accepted orders by the staff
Relation between skill level andZone Defense/Dedication
42
IV. ExpertThey take all of orders in their zone while taking orders in other zone for helping others.
II. Fully occupiedThey take orders in his/her own zone but it is not enough for covering the zone. Support by other staffs is needed.
III. Well organizedThey take all of orders in his/her zone, but they don’t help other zones.
I. PurposelessThey fail to take orders in his/her zone and take orders in other zones. Training is required.
Zone
ord
er d
efen
se ra
tio (Z
OD
):Th
e ra
tio o
f # o
f acc
epte
d or
ders
by
a st
aff i
n hi
s/he
r ow
n zo
ne o
ut o
f all
of o
rder
s in
the
zone
Zone dedication ratio (ZD):The ratio of # of accepted orders by a staff in his/her own zoneout of the total # of accepted orders by the staff
Precision
individual skill Teamworkperformance
Before
43
Precision
After
44
Improved coverage of each zone by each staff
Less need for helping other staffs (zones)
Precision
Pre-evaluation of Kaizen PlanConsidering Efficiency and Employee Satisfaction
by Simulation Using Data Assimilation
45
Sensing ModelingPicking work model of employee
Action
HT
WMSSimulation
・Analyze・Visualize
EmployeeCart
Receiver
VL with IDVLC
Evaluation
Kaizen Support FrameworkSimulator
Planning
・Man-hour Productivity・Worktime・Time to spare ・Evenness of work rate
traffic line
Subject Extraction
Kaizen of the kaizen activity
46
Simulation To-Be
Pre-evaluation
Sensing analyzevisualize As-Is
Understand current status
KaizenPlan C
KaizenPlan B
KaizenPlan A
Action
It is possible to quantitatively decide Kaizen planand to apply KSF to several warehouses
Overview of the measurement field
25m
50 meters
54 meters
D ABC
47
Wide passage
Narrow passage
25m
48
When many employees conduct picking work,Zone A become crowded.
Items in A zone were picked frequently.
Overview of the measurement field
P1P2
P3HT
Measurement method :Warehouse Management System (WMS)
WMS manages items and provides information. When employees pick an ordered item, they are required to scan a barcode with a hand terminal.We can estimate positions from scan data 49
Measurement method : Visible Light Communication System
Receiver
50
Measure and record the positions of the employee and carts that are equipped with a receiver
P1P2
P3
HT
Measurement method :Warehouse Management System
and Visible Light Communication System
Receiver
Estimate route during pickings for combination with timestamp
51
Picking work model constructed and verification of reproduction
AB
C
Order
HT
52
CartEmployee confirm the orders using hand terminalMove toward shelf and into the wide passage with cart
Place cart near shelf on wide passage and leave and enter narrow passage
Pick up items and read a barcode with hand terminalReturn to cart and place picked items on the cart
In this model essentially handles plural orders by repeating these steps
Sorting place
2. Distribute Shelves equally for every zone
1. Divide one floor with some zones
Zone Picking
3. Employee takes charge of only one zone
5. Processes a zone package and brings it to the sorting place
4. Created by a combination of the same zone’s sub‐orders.
53
54
Simulated trajectoriesActual method (Single picking) Kaizen plan (Zone picking)
Trajectory distinguished by color for each employee
Actual method(Single picking)
Kaizen plan(Zone picking)
# of zone -
# of employeesN-7
3-4
4-4
5-5
6-7
7-7
EFMan-hour productivity M H H H H H
Work time as a team M H M M L L
ES Time to spare L L L L M M
Evenness of workload M M L L M L
The result of the best combinations of efficiency and employee satisfaction
EF (Efficiency), ES (Employee Satisfaction)
55
Interview with FPV
Passage of Time
+ Over 50% cost reduction on labor cost and preparation time compared with existing time studies+ Consideration of customer privacy by not using cameras+ FPV with less motion sickness+ Effective in episodic memory retrieval for retrospective interviews considering bounded rationality
Worker’s trajectory
3D model built from a set of photos
First-person view (FPV)
CCE (Cognitive Chrono-Ethnography) Lite
Japanese-style hotel at Kinosaki Onsen (hot spring)
56
国立研究開発法人
キャビンアテンダントのおもてなし分析
57
東⼤・ANA総研の共同研究、及び東⼤・産総研の共同研究の事例⽇経情報ストラテジー12⽉号
• 飛行中の機内でCAの動線を計測• PDR+BLE+マップマッチング• BLEは機内持ち込み荷物の中でラピッド設置・撤去
国立研究開発法人
PDR+BLEを⽤いた達⼈CAと新⼈CAの⽐較
58
CA1業務内容
CA2業務内容
行き 帰り 計
達人CA 15分 6分 21分
新人CA 17分 3分 20分
• ドリンク提供の帰り時間を多く作る• おかわりを申告してもらいやすい• 乗客の変化へ対応がしやすい
1. 乗客の変化に気づき対応するという受動的な行動
2. 乗客の申告を促す能動的な行動
2種類の行動メカニズムの存在の示唆
得られた知見
[日経情報ストラテジー2015年12月号より]
Service Field Simulator•Supporting service design using VR technology
– Evaluating service environment and its process in advance by sensing and analyzing human behavior in virtual environment
Risk reduction by evaluation of the new service in advancecomparison between • current layout and new layout plan• current process and new process
Acquiring more detail and reliable data• Various sensors are available because of
limited sensing area• Easy to control the condition
As is New plan
With EEG
With Eye-Tracker
59
Continued improvementSFS Ver. 1.0• Low resolving power: 0.2• Short of vertical FOV
SFS Ver. 2.0• 24 Full-HD 27-inch LCD: Resolving
power is improved to 0.7
SFS Ver. 2.1• 40 Full-HD 24-inch LCD: Vertical
FOV is improved (Upper 35°, Lower 58.5°)
60
Case studies for verifying efficiency•Gaze point analysis using combination of eye-
tracking device and SFS
– Hypothesis•we can do the same investigation using an eye-tracker and the SFS
as real in-store marketing
in-store marketing experienced person(subjective opinion):"the motion of the gazed point in the virtual environment is similar to that in the real store especially from the entrance to in front of the shelf where target products are laid out"
61
Case studies for verifying efficiency•Investigation for a method for measuring human interest
using EEG and the SFS
62
Example of Analysis and Future Work
63
To compare the shopping behavior in detail, we made heat-map visualization of thestay time for each 50 cm grid in the real and virtual store. The read area indicatessubjects spent longer time than other area. Because position data of the real storesituation is recorded by hand, we only have the discrete position and timestampdata. Therefore, we could not compare both of them strictly, but we found out wecould get the similar results.
Comparison of heat-map visualization of stay
Virtual store in SFSReal store
Dollhouse VR: An Asymmetric Collaborative System for Architectural-scale Space Design
64 提供︓慶応⼤ 杉浦先⽣
Collaborative system for multi-stakeholders
国立研究開発法人
Research cases on LFF and FFL in AIST
65
国立研究開発法人
Thank You!!
66
SFS
Dollhouse VR
CCE Lite
WearableRGB-D sensing
PDR
Handheld AR