opportunistic multipath forwarding in publish/subscribe systems
DESCRIPTION
Opportunistic Multipath Forwarding in Publish/Subscribe Systems. Reza Sherafat Kazemzadeh AND Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto. Agenda. The content-based publish/subscribe model Characteristics and challenges Our approach Overlay neighborhoods - PowerPoint PPT PresentationTRANSCRIPT
Opportunistic Multipath Forwarding in Publish/Subscribe Systems
Reza Sherafat Kazemzadeh AND Hans-Arno Jacobsen
Middleware Systems Research GroupUniversity of Toronto
Opportunistic Multipath Forwarding 2
Agenda
• The content-based publish/subscribe model– Characteristics and challenges
• Our approach– Overlay neighborhoods– Adaptive multipath forwarding
• Experimental evaluation results
Middleware 2012
Opportunistic Multipath Forwarding 3
Content-Based Pub/Sub Model
Pub/Sub
S
SS SS
S
S
PPublish PP
P
SS
Subscribers
PPP
Publishers
Middleware 2012
Opportunistic Multipath Forwarding 4
Content-Based Pub/Sub Model
Pub/Sub
Publish
SS
Subscribers
PPP
Publishers
Middleware 2012
Many to many communication between a large number of publishers/subscribers
Selective delivery based on subscription matching• Unicast (one)• Multicast (some)• Broadcast (all)
Variations in traffic patterns makes it difficult to design an optimal overlay network
Opportunistic Multipath Forwarding 5
Forwarding paths in the overlay are constructed in fixed end-to-end manner (no/little path diversity)
This results in a high number of “pure forwarding” brokers
Current Pub/Sub Overlays
D CE B A PS✗ ✗ ✓✗✓
PP
#msgs delivered#msgs sentSystem Yield = = 1/5 = 20%
Middleware 2012
Opportunistic Multipath Forwarding 6
Summary of Our Goals• Construct a highly connected overlay mesh that provides high
path diversity between publishers and subscribers
• Avoid pure forwarders by allowing brokers to make fine-grained forwarding decisions based on individual publications and their matching sets
• Improve system yield, efficiency, scalability and delivery delay
• Support dynamic adaptive routing based on live traffic patters while avoiding high costs of full overlay reconfiguration
Middleware 2012
Opportunistic Multipath Forwarding 7
• Fixed end-to-end (baseline)Total msgs: 6
• Forwarding strategy 1Total msgs: 5
• Forwarding strategy 2Total msgs: 3
A B C
*
D
*
*
Forwarding Strategies
A B C
*
D
*
*
p * *
*
p * *
*
A B C
*
D
*
*
p * *
*
Middleware 2012
Opportunistic Multipath Forwarding 8
Routing Tables(Δ-neighborhoods
knowledge)
Links Management(best links via a gain
function)
Pub. Forwarding(Path Computations
for strategies)
Our Approach in a Nutshell
A
Δ=1
Δ=2
Δ=3
S
Middleware 2012
Opportunistic Multipath Forwarding 9
A C
* *
*
B DA C
* *
*
B D
Path Computation forForwarding Strategies
Strategy 1
Middleware 2012
A B C
*
D
*
*
p * *
*
A B C
*
D
*
*
p * *
*
Strategy 2
Opportunistic Multipath Forwarding 10
Experimental Evaluations• We have implemented the algorithms and performed large-scale
experimental evaluations with up to 500 brokers
• Datasets– Synthesized based on Zipf distribution– Social networking traces from Facebook
• We measured performance of the system in terms of:– Overlay mesh connectivity– Delivery delay– Maximum system throughput– System yield– Publication propagation path length– Memory and CPU overhead
Middleware 2012
Opportunistic Multipath Forwarding 11
Num
ber o
f Ava
ilabl
e pa
ths
Path Diversity in Overlay Mesh
Network size:120Delta=3
Network size:250Delta=3
Path diversity:20% of brokers w/ 100 paths
Path diversity:10% of brokers w/ 1000 paths
Middleware 2012
Graph is based on a snapshot of the state of links in a running system
Opportunistic Multipath Forwarding 12
Publication Hop CountExperiment setup:• 120 Brokers• Publish rate is 1,800 msgs/sec and number of deliveries: 73,000 (in 5 min)
Middleware 2012
Fixed-end-to-end
Strategy 2
115% throughput enhancement
Opportunistic Multipath Forwarding 13
Conclusions
• Brokers build a highly connected overlay mesh and make fine-grained forwarding decisions for each publication in order to avoid pure forwarding neighbors
• We used the notion of overlay neighborhoods to enable local traffic profiling and avoid high costs of overlay reconfiguration
• Our approach enhances system’s efficiency and yield, and ultimately improves its scalability and performance
Middleware 2012
Opportunistic Multipath Forwarding 14
Thanks
Questions!
Middleware 2012
Opportunistic Multipath Forwarding 15
Client
Broker
Pub/Sub Characteristics
• Many to many communication between a large number of publishers/subscribers
• Selective delivery based on subscription matching– Unicast (one)– multicast (some)– broadcast (all)
• Traffic patterns depends on workload application and may change over time
Middleware 2012
Opportunistic Multipath Forwarding 16
Related Work: Overlay Reconfiguration
Broker overlay is “reconfigured” by addition and removal of links between brokers
Advantages– Forwarding path may be improved bringing some
publishers and subscribers closer together
Disadvantages– Some forwarding paths between publishers and
subscribers may indeed suffer– Resulting overlay still relies on fixed end-to-end paths– Reconfiguration is costly and requires full or partial re-
propagation of subscriptions
[1] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks, FTDCS ‘03[2] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks. In: FTDCS. (2003)
Re-configur
e
Middleware 2012
Opportunistic Multipath Forwarding 17
• Links types– Primary links– Soft links – shortcut links– Candidate links – expeditionary links
• Traffic profiling– In intervals of T sec, brokers count the
number of pubs sent over each links– Gain function
gain(A,B,T) = pub_traffic_during_T * dist(A,B)
• Soft link selection– Profiling input– Link state measured– Broker load information exchanged
Links ranking
A B C D E F G E F …
high → low
Links ManagementOverlay Network
A B
C
DB
Neighbor
loadHigh
ra
nkLink
quality
Primary linkCandidate link Soft link
Middleware 2012
Opportunistic Multipath Forwarding 18
Master v. Working Routing Data Structures
• Overlay map captured by brokers’ Δ-neighborhoods are relatively static Master overlay Map (MOM)
• Brokers link connectivity change dynamically, brokers need an efficient way to compute forwarding paths over the changing set of links Working Overlay Map (WOM)
• MOM is a concise representation of the primary overlay that only contains brokers with a direct link
Master Routing Tables
Working Routing Tables construct
Middleware 2012
Opportunistic Multipath Forwarding 19
Master v. Working Subscription Tables
• Similar to WOM, brokers adapt their subscription tables based on the current set of available links: Working Subscription Table (WST)
Between Brokers
Sets
Beyond Brokers
Sets
Behind Brokers
Sets
Master Subscription
Table
Working Subscription
Table
Working Overlay Map
Middleware 2012
Opportunistic Multipath Forwarding 20
Delivered publications
Strategy Number of pure Pure Forwarders
System Yield
284,000(Dense Workload)
Baseline 195,000 59%
Strategy 1 104,000 73%
Strategy 2 69,000 80%
Delivered publications Strategy Number of pure
Pure ForwardersSystem Yield
73,000(Sparse Workload)
Fixed end-to-end 91,000 44%
Strategy 1 42,000 63%
Strategy 2 29,000 71%
System Yield (measure of efficiency)
Middleware 2012
Opportunistic Multipath Forwarding 21
Publication Hop CountExperiment setup:• 120 Brokers• Sparse and dense workloads• Publish rate of 1,800 msgs/sec Deliveries: 73,000 in 5 min
Middleware 2012
Opportunistic Multipath Forwarding 22
Publication Hop CountExperiment setup:• 120 Brokers• Sparse publication/subscription workload• Publish rate of 1,800 msgs/sec Deliveries: 73,000 in 5 min
Middleware 2012
Opportunistic Multipath Forwarding 23
Publication Hop CountSparse Matching Workload Dense Matching Workload
Multi-path forwarding is more effective in sparse workloads
Middleware 2012
Opportunistic Multipath Forwarding 24
Impact of Broker Fanout on Subscription CoveringExperiment setup:• 500 brokers• Fanout of 5-25
Middleware 2012