closed-loop evaluation of an embedded visual servo system
TRANSCRIPT
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
1/32
Closed-Loop Evaluation of
an Embedded Visual
Servo SystemZhenyu Ye
Prof. Pieter Jonker
Biorobotics
TU Delft
Prof. Henk Corporaal
Electronic Systems
TU Eindhoven
Prof. Henk Nijmeijer
Dynamics & Control
TU Eindhoven
Advisers:
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
2/32
Page 2
Vision Enabled Mechatronics
Example: Vision-Enabled Inkjet Printing
(Source: OTB, Eindhoven)
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
3/32
Page 3
Vision Enabled Mechatronics
Example of an OLED Display:
Inkjet Printing Process:
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
4/32
Page 4
Existing System:
Vision for Offline Alignment
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
5/32
Page 5
Limits of Offline Visual Alignment
Difficult to compensate online dynamics:
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
6/32
Page 6
Limits of Existing Metrology
Techniques
What you measure is not what you get.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
7/32Page 7
Limits of Existing Metrology
Techniques
It is like (according to ASML) locating an apple in the
map of the Netherlands.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
8/32
Page 8
Vision-In-the-Loop Approach
What you see is (almost) what you get.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
9/32
Page 9
Vision-In-the-Loop Case Study
Inkjet printing on an OLED wafer.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
10/32
Page 10
Challenges:
Precision Visual Servo Control
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
11/32
Page 11
Requirements for Electronic
Systems
Typical Requirements for electronic systems:
Low latency: less than 1ms end-to-end delay
Time-predictability: visual feedback in closed loop.
Embedded Systems Needed!
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
12/32
Page 12
FPGA-Based Visual Servo System
(Click to play movie)
http://youtube.com/v/EuOeBHqmt5M -
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
13/32
Page 13
System Overview
This talk focuses on the electronic system.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
14/32
Page 14
Vision Algorithm
Goal: detect the centers of the OLED structures.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
15/32
Page 15
Step 1: Projection
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
16/32
Page 16
Step 2: Filtering
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
17/32
Page 17
Step 3: Detect Segment
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
18/32
Page 18
Step 4: Image Moment
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
19/32
Page 19
Computational Complexity
For an image of width W and height H.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
20/32
Page 20
Architecture and Program
Mapping
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
21/32
Page 21
Example: Parallel Projection
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
22/32
Page 22
Timing Analysis
Throughput: 1600 frames-per-second (fps).End-to-end Delay: 1 ms
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
23/32
Page 23
Timing Analysis
Throughput: 1600 frames-per-second (fps).End-to-end Delay: 1 ms
Bottleneck: Image Readout.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
24/32
Page 24
How Far Can We Go?
What if we have a camera of 10 KHz?
Fraction of resource used on an Virtex-5 FPGA:
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
25/32
Page 25
How Far Can We Go?
What if we have a camera of 10 KHz?
Fraction of resource used on an Virtex-5 FPGA:
Scalable to 10 KHz.
May hit Amdahls Law at 100 KHz.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
26/32
Page 26
Experiment: Velocity Control
Overview of the movement (click to play movie).
Microscopic view at 100x slow motion (click to play movie).
http://youtube.com/v/c-8ytK8HuDohttp://youtube.com/v/vp6iptElS-Y -
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
27/32
Page 27
Evaluation
Velocity error: 0.5 pixel per frame.
Position error: less than 5 micrometer.
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
28/32
Page 28
Conclusions
The visual servo system achieves:
Low latency: less than 1 ms, throughput of 1600 fps.
High precision: position error less than 5 micrometer.
Time-predictable visual processing in closed loop.
High-speed high-precision
visual servoing feasible!
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
29/32
Page 29
Future Work
Towards 10 KHz nano-scale visual servoing
Cyber-physical modeling
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
30/32
Page 30
10 KHz Nano-scale Visual
Servoing
Commercial off-the-shelf cameras reach 10 KHz.
(Source: Mikrotron EoSens 3CL)
C b Ph i l M d li
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
31/32
Page 31
Cyber-Physical Modeling
Th k Y ! Q ti ?
-
7/29/2019 Closed-Loop Evaluation of an Embedded Visual Servo System
32/32
Page 32
Thank You! Questions?