doctoral defense application of agent-based approaches to enhance container terminal operations
DESCRIPTION
Doctoral Defense Application of Agent-Based Approaches to Enhance Container Terminal Operations by Omor Sharif Director: Dr. Nathan Huynh In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Civil Engineering College of Engineering and Computing - PowerPoint PPT PresentationTRANSCRIPT
1
Doctoral Defense
Application of Agent-Based Approaches to Enhance Container
Terminal Operations
byOmor Sharif
Director: Dr. Nathan Huynh In Partial Fulfillment of the Requirements for the Degree of
Doctor of Philosophy in Civil EngineeringCollege of Engineering and Computing
Summer, 2013
2
A Very Brief Review of Container Terminal
• An interface between ocean and land transport• Containers are loaded/unloaded to/from a ship• Temporary storage of export and import
containers• Operation involves a large number of decisions
Land Side
Water Side
Container Yard
3
Growth in Container Traffic, Major Challenges
• Limited capacity issues, Congestion• Environmental concerns, emission from
congestion• Increased complexity in planning• Lack of optimized decision making tools• Competition among terminals has considerably
increased• Provide satisfactory customer service, reliability• Tools that reduce cost, increase productivity of
operationsOptimization | Automated Equipment | Information Technology
World container volumes have increased from 113 million TEU to more than 572 million TEU in the period from 1993 to 2011. An annual average growth rate of nearly 10% per year.
1993
1995
1997
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
0100200300400500600700
Total Container Traffic (Million TEU) By Year
Flow of Containers in a Container Terminal
CUSTOMER/DEPOTS
TERMINALGATE
STORAGEAREA
SHORE/BERTH
VESSEL/ SHIP
XTs XTs AGVs /ITs/SCs QCsYCs
ARRIVAL AND STORAGE RETRIEVAL AND LOADINGRETRIEVAL AND PICKUP UNLOADING AND STORAGE
Quay Cranes
Internal Trucks
Yard Cranes
ExternalTrucks
ExternalTrucks
Customers
Storage
Area
Berth
Vessel
Intern
al Tru
ck
5
Decision Problems in Container Terminal
× Berth Allocati
on
× Quay Crane Scheduling
Transport
Containers
Storage Assignment
Yard Crane Scheduling
Gate Operations
Three Research TopicsResearch Focus Areas
Queuing of Trucks at
GatesYard Crane Operations
Container Storage Problem
6Reduce truck processing time | Reduce congestion | Reduce emission Optimize available resources | Improve efficiency of related processes
7
Journal Article I
Application of El Farol Model for Managing Marine Terminal Gate
Congestion
B
byOmor Sharif, Nathan Huynh and Jose Vidal
Research in Transportation Economics, 32 (1), 81-89 (2011)
Excessive Waiting of Trucks at Terminal Gates
• Documentation processing, security checks at gate• At peak hours demand greatly exceeds supply at
gate• Long waiting time results in high truck cost• Source of significant emission, unhappy truckers
Figure: Port Elizabeth, New Jersey - Peak hour congestion at gate
How to Reduce Queuing Time? Experimental Solutions to Reduce Fluctuating
Arrivals
Appointment System with Time
WindowsLive View of GateUsing Webcams
9
10
Agent-based Model for Dispatch Decisions Send
Truck Now???
4 miles
11
El Farol Bar Problem (Arthur, 1994)• A bar in Santa Fe, New Mexico• Every Thursday night, everyone wants to go to the
El Farol• But it's no fun to go there if, say, more than 60 go
to the bar, they'll all have a worse time than if they stayed at home.
• It is necessary for everyone to decide at the same time whether they will go to the bar or not
• If agents use different strategies to decide, the overall attendance fluctuates at 60, which is the Nash Equilibrium
Fig: A simulated 100-week record of attendance at El Farol
Principal Parameters of Model
12
N ≡ Set of Depots (n ∈ N), T ≡ Set of Trucks (t ∈ T)
Tolerance, L ≡ Maximum time the truck willing to waitExpected wait, E (W) ≡ The time truck thinks it will waitMake a Decision, SEND? (n, t) ≡ YES if E (W) ≤ L
NO otherwise
Wait at gate, W(t) = Waiting in Queue + Waiting in ServiceDiscretization interval, I ≡ How often to make a decision?Historyx = Waiting time at terminal gate for last m intervals
Estimate E(W) using ‘predictors’• Predictors are simple rules or
strategies/hypotheses• Agents use predictive strategies to estimate E(W)• Sample predictors-
A random number Positive trend of last 10 values A periodic function Moving average of past 5 values
13
14
Solution Steps
Create: A large number of ‘predictors’ to estimate E(W)Choose: Agent chooses k predictors from predictor space, S Estimate: Estimate E(W) using each predictor
Learn: Agents learns about how predictor has performed in past Rank: Agents assign scores and rank the predictors
Use Best: Use ‘best performing predictor’ from predictors set
Model Implementation in Netlogo
• Simulation model, coded in Netlogo• A modeling environment for agent-based systems• Many useful primitives that expedite the
implementation• Extensive documentation and tutorials, model
libraries• Updated versions are continuously released 15
Experiment Parameters (Process 1440 Trucks)
Parameter Value UnitNumber of Depots 10 NosDispatch rate (θ) 12 trucks/
depot/hrMean transaction time (μ)
3, 4, 5, 6, 7 and 8
minutes
Tolerance (L) 15, 20, 25 and 30
minutes
Total predictors 200 NosPredictors per depot (k)
12 Nos / depot
Update interval (I) 5,10 and 15 minutesMaximum memory (m)
20 intervals
Predictor scoring policy
Original precision
n/a
Alpha 0.5 n/a
16
Results (Impact of Tolerance, Interval on Wait)
Fig: Impact of tolerance on mean wait time of trucks
Fig: Impact of tolerance on max waiting time. 17
Increases fast at low I
Mean Wait Time Fluctuates Around ToleranceFig: Mean wait time of trucks (I =15 minutes, L = 15 minutes)
Fig: Mean wait time of trucks (I =10 minutes, L = 10 minutes)
18
Significantly Better Than ‘Do Nothing’
• 43% and 63% lower mean wait time for I = 5 and 10 mins
• 22% and 40% lower maximum wait time for I = 5 and 10 mins
19
Key Findings and Contributions• Provides steady truck arrival, less queuing time at gate
• Adopt higher ‘I’ for distributing peak hour demand
• Good amount of emission (PM2.5, NOx, GHG) reduction
• First study on gate congestion using real-time information
• Examines how dispatchers can benefit from real-time information.
• Offers many experiments for choosing suitable parameters
20
21
Journal Article II
Inter-Block Yard Crane Scheduling at a Marine Container Terminal
byOmor Sharif, Nathan Huynh, Mashrur Chowdhury, Jose
VidalTransportation Science and Technology, 1 (2), 109-130
(2012)
22
Efficiency of Container Yard Operations is Crucial
• Container yard is the interface of land and waterside operations
• Containers are temporarily stored in Container Yard • A container yard is made up of several blocks of
containers• Workload varies (1) among blocks (2) over operational
hours
Land/ Gate Side
Water/ Vessel Side
23
Yard Cranes Influence Truck and Ship Waiting Time
• Yard Cranes are deployed for container handling• Rubber Tired Gantry Cranes are most popular Yard
Crane• Cranes support the ship loading and unloading
operations• Cranes support external truck operations
24
Objective, Assumptions, Constraints
• Total operational hours is divided in several planning periods
• A workload forecast for blocks is known at start (time-units)
• At most two cranes work at a block at the same time• At most one transfer per planning period• 10 – 15 minutes transfer time for each block traveled
Given: A limited number of cranes and workload at blocksGOAL: Assign and relocate cranes among blocks to minimize the workload that remains unfinished at the end of a period
25
Rules for Initial Assignment of Cranes to Blocks High to low work volume• Sort blocks in decreasing order of initial work
volume (IWb)• Assign NCb
max cranes to each block starting from topmost item
Crane at each block• If nc = nb, assign one crane per block• If nc < nb, sort blocks in decreasing IWb , assign
cranes• If nc > nb, first assign one crane per block, then sort
in decreasing IWb
Reduce transfers • Assign cranes to reduce number of future transfers• A four step approach• Based on pass/fail of a series of inequalites
26
Three Principal Parameters of Model Extra capacity of a crane• E (c) = Tc × NCb
initial − IWb
• Time Period X Initial Number of Cranes – Workload of block
Amount of help needed by a block• H (b) = IWb − Tc × NCb
current
• Workload of block - Time Period X Current Number of Cranes
Transfer time matrix• TTc
od
• Different transfer time for longitudinal and lateral movement
27
Matching using Deferred Acceptance Algorithm• Assigning cranes are viewed as matching crane and
blocks• DAA is a market matching model by Gale and Shapley• Numerous follow up studies in Economics, Computer
science• Find a match between two set of agents• Each agent has a preference order over other set of
agents • ‘One to one’ matching – e.g. Marriage Problem• ‘Many to one’ matching – e.g. College Admission
28
Preference Functions For Crane And Block AgentsMinimum transfer time • Transfer time of a crane from origin to destination
blockPositive difference • Extra Capacity of a crane – Help needed by a
block – Transfer timeAbsolute difference • abs (Extra Capacity of a crane – Help needed by a
block – Transfer time)
Absolute difference squared distance• abs (Extra Capacity of a crane – Help needed by a
block – Transfer time2)
29
Algorithm to assign cranes to blocks1. Each crane j send match request to first block i from its preference list
2. Each block i receiving more than qi requests, ‘holds’ the most preferred qi cranes and rejects all others.
n. Each crane j rejected at step n − 1 removes the block i rejecting the crane from its preference list.
Then the rejected crane j makes a new match request to its next most preferred block i who hasn’t yet rejected it.
Go to step n − 1. (n = 3, 4, 5, …)
30
Implementation Snapshot from Netlogo
• Multi-agent simulation GUI
Setup block layout
Create workload
Create cranes
Initial assignment
Inter-block transfer matrix
Pre-analysis steps
Find Matching
Reassign/update results
31
Test Parameters Based on Real-World ProblemsParameter Value UnitNumber of blocks 10, 20, 30 NosNumber of cranes 1 or 1.5 × Number of blocks NosLength of planning period
240 Minutes
Work volume (1) Moderate (2) Heavy (3) Above capacity
Minutes
Initial assignment (1) Crane at each block (2) High to low work volume (3) Reduce transfers
-
Preference strategy (1) Minimum transfer time (2) Absolute difference (3) Positive difference (4) Absolute inverse squared distance
-
Algorithm Version (1) Blocks proposing (2) Cranes proposing
-(1)Moderate Workload: 60% of total crane capacity ± 40% of average
workload(2)Heavy Workload: 90% of total crane capacity ± 20% of average
workload(3)Above Capacity Workload: 110% of total crane capacity ± 40% of
average workload
32
Model Effectively Reduces Unfinished WorkloadPercentage incomplete work volume: (Incomplete/Total*100%)Case I - Average number of cranes per block = 1.0
Number of Blocks 10
20
30
Number of Cranes 10
20
30
Work Volume M H AC M H AC M H AC
Minimum Transfer Time 0 0.13 11.34
0 0.10 11.14
0 0.12 11.28
Absolute difference 0 0.16 12.16
0 0.13 12.09
0 0.11 12.16
Positive difference 0 0 13.55
0 0.05 13.08
0 0.04 12.86
Absolute difference squared distance
0 0.13 11.80
0 0.08 11.69
0 0.10 11.81
Mathematical Program 0 0 10.57
0 0 10.57
0 0 10.60
33
Majority of Cases are Optimal/Near-optimalPercentage incomplete work volume: Case II - Average number of cranes per block = 1.5Number of Blocks 10 20 30
Number of Cranes 15 30 45
Work Volume M H AC M H AC M H AC
Minimum Transfer Time 0 0.02 9.79 0 0.20 10.00
0 0.48 10.04
Absolute difference 0 0.07 10.19
0 0.18 10.45
0 0.43 10.48
Positive difference 0 0.14 10.40
0 0.53 10.73
0 1.00 10.71
Absolute difference squared distance
0 0.01 10.06
0 0 10.31
0 0.01 10.38
Mathematical Program 0 0 9.69 0 0 9.77 0 0 9.79
34
Key Findings and Contributions• In ‘medium’ condition all work can be finished • In ‘heavy’ condition the percentage incomplete is 1% or less• In ‘above capacity’ condition the percentage remaining is
within 3% of the lower bound.• ‘Reduce Transfers’ initial assignment works best.• An agent-based approach for solving inter-block scheduling
problem• Provide several preference functions that are effective in
minimizing• Scalable and time efficient, a problem with 30 blocks can be
solved in 3 seconds• Several minutes to solve using Mixed Integer Programs• Provides various strategies for initial assignment of cranes
to blocks
35
Journal Article III
Storage Space Allocation at Marine Container Terminals Using Ant-based
Control
byOmor Sharif and Nathan Huynh
Expert Systems with Applications, 40 (6), 2323-2330 (2013)
36
Assign Newly Arriving Containers to Yard Blocks• Container yard is divided up into yard
blocks• Each yard block has a fixed total capacity• Store export containers brought in by
trucks• Store import containers unloaded from
ships• Allocate yard blocks to newly arriving
containersGates BerthsYard Blocks
Export containers
Import containers
Export containers
Import containers
Land Side Container Yard Water Side
37
Two Competing Objectives of SSAPBalance the workload of blocks– Proper
distribution of arriving containers amongst blocks
Minimize container transportation cost – Minimize the distance traveled by ITs between berth and yard
38
• Reduces relocation of yard cranes• Reduces waiting time of vessels• Reduces average container handling time• Reduces congestion on road network• Improves efficiency of related resources (quay
cranes, yard cranes, storage space, ITs) – Zhang et al., 2003
Workload Balancing is Critical Because…
Workload Imbalance is the variability of number of containers among blocks at some time (we used standard deviation for this measure).
39
We Utilize an Ant-Based Control• ABC Falls under ‘Ant colony optimizations &
algorithms’ • Inspired by collective intelligence of an ant
colony• Can solve complex routing problems in network • For example, used for shortest path, load
balancing• We utilize pheromone laying behavior of ants
Ants have a decision to make
Situation several months later
40
Highlights of Ant Based Control for SSAP• A container terminal is a network of gates, blocks,
berths• Links are bi-directional container transportation
routes• Network also support a traffic of artificial ant
agents• Ants move just like inbound & outbound
containers• Ants lay pheromone as a function of distance &
congestion• Probabilities in pheromone table guide ants &
containers
Fig: Ant choses path according to probability
41
Shortest Path
Workload Balancing
• Ant age Increases with traveled distance• Probability of selection decrease with ant age
• Ants are delayed at congested blocks
• Delay is proportional to degree of congestion
42
Dynamic Interaction between Ants & Containers
43
Simulation
• The snapshot of GUI of our simulation
44
Parameter and Values Used in Model
A demand of 2500 Containers
45
Parameter and Values Used in Model
46
Performance Metric I - Transport Distance
• Converge to shortest path in 500 time-units• Under no congestion converge to lower bound• When congestion occurs distance converge to 2% above
lower bound
47
Performance Metric II - Workload Imbalance
• Standard deviation gradually rises when no congestion• When congestion occurs standard deviation falls• Maintains a distribution to accommodate both objectives
48
Key Findings and Contributions• The approach is effective in balancing workload
in blocks• Reduces transport distances during ship
loading/unloading • Distributed allocation to individual containers• Real-time solution requiring no advanced
container arrival information, more suitable for real-world truck arrival
• Adaptive, scalable.• Synchronous instead of traditional hierarchical
approach
49
Thank You
50
• A class of computational modeling • For simulating the actions and interactions of
autonomous, intelligent agents • To assess the agents’ effects on the system as a
whole.• ABMs are individual-based models• Most agent-based models are often composed of: (1)agents with simple or complex goals(2)distributed decision-making heuristics; (3)learning rules or adaptive processes; (4)an interaction system; and (5)a non-agent environment
What is Multi-agent Systems and Agent-Based Modeling?
51
• ABM is relatively new research field within artificial intelligence.
• ABM is relatively unexplored in the area of terminal operations.
• Suitable for dynamic distributed resource allocation• Traditional approaches use centralized mathematical
programs for optimization• No Polynomial time solution for majority of problems• ABM can outperform in certain domain property• Scalability: Do not require exhaustive search in solution
space, model integration potential• Computational Efficiency: Real time solution, no
advanced computation time is required• Adaptability: No forecast required, no revision for
forecast errors• Fault tolerance: Failure will impact locally, thus more
robust
Why use agent-based modeling in a terminal?
52
• In Recent years integrative modeling for container terminals are being emphasized
• Various processes in a terminal are interconnected
• Improved terminal performance cannot necessarily be achieved by treating the processes separately
• A few studies have attempted such an integrated approach, and they are on the quay side
• More decision tools need to be developed and integrated
• Multiagent systems approach has been proposed as a tool in integrated decisions frameworks in works by Henesey (2006); Franz et al. (2007), Stahlbock and VoB (2008)
• Integrated model using traditional optimization techniques has limited viability due to computational complexity.
Integrative Modeling in CT Research
53
• Drayage is relatively short distance of supply chain
• However, drayage accounts for 25% to 40% of Origin-Destination Expenses
• High cost affect the profitability of an intermodal service
• Reduce the emissions impact on the surrounding communities
• Reducing the idling time of drayage trucks is equivalent to reducing local and regional particulate matter (PM 2.5), nitrogen oxides, and greenhouse gas emissions
• Drayage trucks operate primarily in urban environments, a reduction of these harmful pollutants has a proportionally greater benefit.
Why study drayage?
54
• Terminal operator designates available time windows, truckers choose one of the available time windows
• Experimented in terminals e.g. Los Angeles, Long Beach, Oakland, Port of Vancouver etc. Under consideration at others.
• In theory, the appointment system should reduce truck processing time
• In practice, little time savings for truckers was reported due to lack of structures in appointment system
• Appointment systems are in an early stage of development, with no uniformity between terminals or ports and many implementation issues to be resolved
Appointment System is at early stage of development
55
• The problem addressed in this study is more complex than the original El Farol Bar
• First, the total demand (number of people intending to go to the bar) does not vary with time, truck demands vary throughout the day.
• Second, the bar attendance event occurs at one specific time. Truck dispatching a continuous process that occurs throughout the operational hours of the terminal.
• Third, depots are not homogenous, they are located at different distances; depots closer to the terminal are in a better position to take advantage of the provided real-time information, E(W) will be closer to the actual wait time.
• Fourth, travel time and service time are not applicable to bar problem. Given service time is stochastic, the strategy of estimating truck wait time is more difficult.
Additional difficulty of Decision making by a truck dispatcher VS Individuals in the bar
56
• For each block the initial work volume at the beginning of a planning period is the work volume forecast for that period plus incomplete work volume from the previous period.
• Work volume = Average timeunits required per move X Number of container moves
How to Determine Workload in a Yard Block
57
Proposed Agent-based Model (Contd.)
Pseudo Code of the Program – Part of the Main Loop
58
A Sample Example of Interblock YC Schedule
Initial Assignment Obtained via ‘Reduce Transfer’
(block 0, crane 11); (block 3, crane 19); (block 2, crane 18); (block 7, crane 16); (block 5, crane 14)
Mathematical program gives the same assignments
59
A sample example (Contd.)
60
• Some of the preferences in this example are symmetric
• The crane’s first choice is a block whose first choice is the crane (e.g. Block 0 and Crane 11; Block 7 and Crane 16; Crane 14 and Block 5).
• Notice that Block 3 and Block 0 both wants Crane 11; Block 0 gets it because it is the first item of Crane 11’s list; Block 3 ends up getting 4th choice)
• We can pick different preference strategies; for example, the cranes may use ‘minimum transfer time’, whereas the blocks may use ‘absolute difference’.
Additional Notes on the Sample Example
61
Overview of Storage Space Allocation Problem (SSAP)
62
Block level decision-• Finds the block to store a container• Objectives are ‘workload balancing’ & ‘distance
minimizing’
Stack level decision-• Finds an exact stack in a block• Objective is to ‘minimize future reshuffling’
Two Levels of SSAP
63