applied scientific research - egypt scholars...civil engineering (intelligent transportation...
TRANSCRIPT
Applied Scientific Research:From Lab to Field
Samah El-Tantawy, PhDAssistant Professor
Faculty of Engineering, Cairo University
2/07/2018
Applied Scientific Research: From Lab to Field
My Academic Career Journey
Electronics and Communications
Engineering, B.Sc.
Engineering mathematics,
M.Sc.
Civil Engineering (Intelligent
Transportation Systems), Ph.D.
AI Applications (Transportation,
Education,…)
2
Basics of Scientific Research
Agenda
• I: What is research? The Scientific Research Process
• II: Why is Research?
• III: A Journey of An Applied Scientific Research Example
From Lab to Reality
• IV: When to Start Research?
• V: What do you need to know before starting your
research?
3
Applied Scientific Research: From Lab to Field
I. Scientific Research Process
4
Applied Scientific Research: From Lab to Field
Have you conducted scientific research before?
5
6
Applied Scientific Research: From Lab to Field
Research is a Key Driver of Innovation
7
II. Why Do We Need to Conduct Research?
Why should we conduct Research?
• To develop knowledge for professions.
• To develop effective policies.
• To solve practical problems.
• To make informed decisions.
• To increase the knowledge base of larger society.
Huge amounts of daily life and experience in our society are based on what we have learned using the logic and evidence involved in scientific research.
8
Applied Scientific Research: From Lab to Field
III. Taking Research From Lab to Field
9
Smart Traffic Lights that Learn !
Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers
M A R L I N
Smart Traffic Lights that Learn !
Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers
M A R L I N
Baher Abdulhai, Ph.D., P.Eng.
Director, ITS Centre and Testbed, University of Toronto
Co-founder of Pragmatek Transport Innovations Inc.
Samah El-Tantawy, Ph.D.
Post Doctoral Fellow, University of Toronto
Co-founder of Pragmatek Transport Innovations Inc.
Applied Scientific Research: From Lab to Field 11
1. Make An Observation
Traffic Congestion: Urban CANCER
0
5
10
15
20
25
0
2
4
6
8
10
12
Jakarta Sydney Chicago Area NY-NJ Cairo Los Angeles Toronto
Pp
op
ula
tio
n in
Mill
ion
s
Co
st in
$ B
illio
ns
Cost of Congestion Population
Applied Scientific Research: From Lab to Field
Traffic Congestion Problem
Tahrir Square, 1914
Tahrir Square, 2014
13
2. Define a Problem Statement and
Formulate a Research Question
Applied Scientific Research: From Lab to Field 15
Applied Scientific Research: From Lab to Field
Traffic Congestion Solutions
More Supply
Less Demand
Intelligence
16
Applied Scientific Research: From Lab to Field 17
Intelligent Transportation Systems
Intelligent Transportation Systems (ITS)
Advanced Travelers Information Systems
Advanced Public Transit Systems
Advanced Traffic Management Systems
Ramp Metering Signal Control Route Divergence
Fixed-Time Control Actuated ControlAdaptive Traffic Signal
Control (ATSC)
Applied Scientific Research: From Lab to Field 18
3. Gather InformationLiterature Review
Applied Scientific Research: From Lab to Field 19
Applied Scientific Research: From Lab to Field 20
Applied Scientific Research: From Lab to Field
Research Database
21
Applied Scientific Research: From Lab to Field
ATSC in North America
22
MARLIN-ATSC: Level 4
Evolution of “Adaptive” Traffic Signal Control
Level 0• Fixed-Time
and Actuated Control
• TRANSYT• 1969, UK
Level 1• Centralized
Control, Off-line Optimization
• SCATS• 1979,
Australia • >50
installations worldwide
Level 2• Centralized
Control, On-line Optimization
• SCOOT• 1981, UK • >170
installations worldwide
Level 3 • Distributed
Control, Model-Based
• OPAC, RHODES• 1992, USA• 5 installations in
USA
Level 4• Distributed
Self-Learning Control
• MARLIN-ATSC• 2011, Canada
23
23
Issues with Leading ATSC Technologies
• Expensive• Not scalable • Not robust
Centralized
• Relying on an accurate traffic modelling framework
• the accuracy of which is questionableModel-Based
• Increasing the complexity of the system exponentially with the increase in the number of intersections/controllers
Curse of Dimensionality
• Requiring highly skilled labour to operate due to their complexity.
Human Intervention
Requirements
24
24
4. Hypothesis Construction
Applied Scientific Research: From Lab to Field
In a Nutshell• Grand objective
– Intersections "talk to each other"
– Each is affected by what is happening upstream
– Each affects what is happening downstream –
– Whole network control in one shot from a grand brain is the dream
• Issue
– Intractable theoretically
– Too complex practically
– Requires massive and very expensive communication
• Solution??
– Decentralized
– Self learning: agents learn to control their local intersection
– Game theory based: agents learn to collaborate
26
5. Define Methodology and Do Experiments
Applied Scientific Research: From Lab to Field
Traffic Signal Control ProblemMarkov Decision Process
28
Current State
Next State ??
Applied Scientific Research: From Lab to Field
Markov Decision ProcessDynamic Programming
29
• The Transition Probability Matrices (TPM)
• (-1)* Delay
•Extend Green (a1)•Switch to Minor Street (a2)
• Queue Length• Short (S1)• Long (S2)
State Action
Transition ProbabilityReward
Queue Length
Optimal Control Policy that
maximizes the expected long-term
rewarda1
a2Curse of Modeling Curse of Dimensionality
Applied Scientific Research: From Lab to Field
Smart / Intelligent Machine?
SMA
RT
can
:
Sense
Interpret
Learn
React
Optimize
Adapt
Help
30
Applied Scientific Research: From Lab to Field
The Most Intelligent Machine Known to Man?
31
Artificial Intelligence MethodReinforcement Learning
Reward (savings in delay)
environment
action
observation (queue Lengths)
MARLIN
32
Q a1 a2
s1 -10 -5
s2 -3 -15
Q Table
Model-Free Algorithms
Model-Based Algorithms
Solution Form
Theoretical Framework
Problem Identification
Stochastic Control Problem
Single Agent (Stationary Environment)
Markov Decision Process
Optimal Control Policy
Dynamic Programming Methods
Reinforcement Learning
Multi-Agent (Non-Stationary Environment)
Stochastic Game
Optimal Joint-Policy
Game Theory Methods
Multi-Agent Reinforcement Learning
Single vs. Multiple Agents:MARLIN Independent vs Integrated33
33
Corridor Synchronization Vs. Network-Wide Coordination
Collaboration with each adjacent intersection in the
neighborhood
Collaboration with each adjacent intersection in the
neighborhood
34
Corridor SynchronizationCorridor Synchronization
34
Applied Scientific Research: From Lab to Field
MARLIN-ATSC Framework
Agent(Learning and
Decision Making)
Environment(Observation and Acting)
Environment-Agent Interface
(Synchronized Interaction)
RL
In
pu
t P
ara
me
ters
RL
In
pu
t P
ara
me
ters
Sig
na
l D
ata
P
ara
me
ters
Sig
na
l D
ata
P
ara
me
ters
35
Applied Scientific Research: From Lab to Field 36
6. Data Collection and Analysis
Applications Simulation Testbeds on Toronto
Applications Simulation Testbeds on Toronto
37
Applied Scientific Research: From Lab to Field
Traffic Simulator
38
Applied Scientific Research: From Lab to Field
Testbed Networks and Applications
Isolated Int.
→ Different RL Design Parameters
5- Int. Network
→ Transferabilityand Different CoordinationApproaches
Large-Scale
→ Scalability and Network-wide
Coordination Effect
39
Applied Scientific Research: From Lab to Field
Isolated IntersectionBay and Front (Downtown Toronto)
Zone 1
Zone 2
Zone 3
Zone 4
0%
5%
10%
15%
20%
25%
1 2 3 4 5 6
Dem
an
d A
rriv
al P
erce
nta
ge
Time Interval (10min)
Uniform Profile For All Phases Phase 1 (NBL-SBL) Profile Phase 2 (NB-SB) Profile Phase 3 (EBL-WBL) Profile Phase 4 (EB-WB) Profile
40
Applied Scientific Research: From Lab to Field
5-Intersections Network
4 Bay & Front
2 University/
York & Front
3 Wellington
& Bay
1 Bay & Lake
Shore
5 Yonge &
Front
4 Bay & Front
2 University/
York & Front
3 Wellington
& Bay
1 Bay & Lake
Shore
5 Yonge &
Front
IN
Wellington & BayIE
Yonge & Front
IW
University/
York & FrontIW
Bay & Lake Shore
IC
Bay &
Front
41
Applied Scientific Research: From Lab to Field
Large-Scale Application in Toronto
42
Applied Scientific Research: From Lab to Field 43
7. Make Conclusions
Applied Scientific Research: From Lab to Field
Large-Scale ApplicationAverage Delay % Improvements
MARLIN-IC vs BCMARLIN-IC vs BC
% ImprovementArea 1
Area 2
Area 3
44
Applied Scientific Research: From Lab to Field
Large-Scale ApplicationAverage Route Travel Time for Selected Routes
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12
Av
era
ge T
ra
vel
Tim
e (
min
)
Time Interval (5 min)
Gardiner EB
BC MARL-TI MARLIN-IC
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8 9 10 11 12
Average T
ravel
Tim
e (
min
)
Time Interval (5 min)
LakeShore EB to Spadina NB
BC MARL-TI MARLIN-IC
0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8 9 10 11 12
Average T
ravel
Tim
e (
min
)
Time Interval (5 min)
LakeShore EB to Spadina NB
BC MARL-TI MARLIN-IC0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8 9 10 11 12
Average T
ravel
Tim
e (
min
)
Time Interval (5 min)
LakeShore EB to Spadina NB
BC MARL-TI MARLIN-IC0
2
4
6
8
10
12
14
16
18
20
1 2 3 4 5 6 7 8 9 10 11 12
Average T
ravel
Tim
e (
min
)
Time Interval (5 min)
LakeShore EB to Spadina NB
BC MARL-TI MARLIN-IC
45
Hardware-in-the-Loop Simulation (HILS)
Hardware-in-the-Loop Simulation (HILS)
Samah El-Tantawy Kasra Rezaee Hossam Abdelgawad Tamer Abdulazim Baher Abdulhai
Applied Scientific Research: From Lab to Field
Controller Interface Device(CID)RS485 to USB
Traffic Signal Controller
RS485 -SDLC protocol
USB -SDLC protocol
Industrial Computer
Ethernet -NTCIP protocol
Paramics Modeller
MARLIN-HILS Architecture
47
Applied Scientific Research: From Lab to Field 48
8. Communicate Results
Applied Scientific Research: From Lab to Field
Writing a Research Proposal/ Publishing a Paper
7/23/2018 49
Applied Scientific Research: From Lab to Field 50
Applied Scientific Research: From Lab to Field
MARLIN RecognitionIEEE ITSS Best Ph.D. Dissertation Award, 2013
Canadian Institute of Transportation Engineers Scholarship, 2010
INFORMS George B. DantzigDissertation Award, 2013
ITS Canada Scholarship, 2011
51
Applied Scientific Research: From Lab to Field
MARLIN Featured in Media!
52
52
Towards System Implementation Towards System Implementation
53
MARLIN Field Implementation
Ethernet
Switch
Industrial Computer
running MARLIN Traffic Controller
54
Lessons Learned for Future Consideration
Applied Scientific Research: From Lab to Field 56
Applied Scientific Research: From Lab to Field
IV. When to Start?
57
Applied Scientific Research: From Lab to Field 58
Applied Scientific Research: From Lab to Field
IV. What do you need to know before starting your research?
7/23/2018 59
Applied Scientific Research: From Lab to Field7/23/2018 60
Applied Scientific Research: From Lab to Field7/23/2018 61
Applied Scientific Research: From Lab to Field
Lessons Learned
62
• Research Never Ends– Stay on Top of the Literature
• Passion and Handwork are the Keys
• Adapting toYour Audience Academia Vs. Industry Vs. Government
“If Opportunity Doesn’t Knock, Build A Door “ Milton Berle
Being Out of Your Comfort Zone
MARLIN is One Piece of the Puzzle
Samah El-Tantawy, [email protected]
Assistant ProfessorFaculty of Engineering, Cairo University
27/07/2017