centralized coordinator
DESCRIPTION
Centralized Coordinator. Outline. Query type 1, 2, 3 Setup 1, 2, 3 Intra WSs Parallelism vs. No WSs Parallelism Analyze results Conclusions. ∏. GC. YP. Branch. Join. YP. BR. . Project. GC. . ∏. TL. Query type. Query type 1 Query type 2. ∏. GC. YP. BR. . . TL1. ∏. - PowerPoint PPT PresentationTRANSCRIPT
11
Centralized Centralized CoordinatorCoordinator
22
OutlineOutline
Query type 1, 2, 3Query type 1, 2, 3 Setup 1, 2, 3Setup 1, 2, 3 Intra WSs Parallelism vs. Intra WSs Parallelism vs.
No WSs ParallelismNo WSs Parallelism Analyze resultsAnalyze results ConclusionsConclusions
33
Query typeQuery type
Query type 1Query type 1
Query type 2Query type 2
BranchYP Join
Project GC
YP BR
∏ GC
∏ TL
44
Query typeQuery type
Query type 3Query type 3
BR
∏ GC
TL1
UnionSplit
YP
∏ TL2
TL52
55
Setup1Setup1
SQL-Server (YP)
Geocoder Web Service (GC WS)
Tigerline Web Service (TL WS)
Yellow Pages Web Service (YP WS)
Centralized Coordinator Web Service (CC WS)
Broker WS
SQL-Server (GC)
SQL-Server (TL)
Work Load Generator (WLG)
66
Setup2Setup2
Broker WS
Yellow Pages Web Service (YP WS)
Geocoder Web Service (GC WS)
Tigerline Web Service (TL WS)
Centralized Coordinator Web Service (CC WS)
Work Load Generator (WLG)
77
Setup3Setup3
Broker WS
12 Gateways
WLG
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
88
Centralized Centralized CoordinatorCoordinator
Broker WS Show all automotive
Show all automotive tire businesses in
tire businesses in my zip-codemy zip-code WLG
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
99
Centralized Centralized CoordinatorCoordinator
Broker WS
WLGRequest a copy of CC WSRequest a copy of CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
1010
Centralized Centralized CoordinatorCoordinator
http://Gateway12/CCWShttp://Gateway12/CCWS
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
Broker WS
WLG
1111
Centralized Centralized CoordinatorCoordinator
Broker WS
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
Query (zip code, business category, subcategory)
Query (zip code, business category, subcategory)
WLG
1212
Centralized Centralized CoordinatorCoordinator
WLG
Gateway12Gateway12
request a copy of YP WS
request a copy of YP WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Broker WS
Gateway2Gateway2Gateway1Gateway1
1313
Centralized Centralized CoordinatorCoordinator
WLG
Gateway12Gateway12
http://Gateway2/YPWS
http://Gateway2/YPWS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
Broker WS
1414
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
Query (zip code, business
Query (zip code, business
category, subcategory)
category, subcategory)
1515
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
Address, name, etc for
Address, name, etc for
each business
each business
1616
Centralized Centralized CoordinatorCoordinator
WLG
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway2Gateway2Gateway1Gateway1
GC Thread
TL Thread
request a copy of GC WS
request a copy of GC WS
request a copy of TL WS
request a copy of TL WSBroker WS
1717
Centralized Centralized CoordinatorCoordinator
WLG
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway2Gateway2Gateway1Gateway1
GC Thread
TL Thread
http://Gateways2/GCWS
http://Gateways2/GCWS
http://Gateway1/TLWS
http://Gateway1/TLWS
Gateway12Gateway12
Broker WS
1818
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
GC ThreadQuery (address)
Query (address)
TL Thread
Query (address)Query (address)
No WSs Parallelism
1919
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
Gateway12Gateway12
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
GC ThreadResultsResults
TL Thread
ResultsResults
No WSs Parallelism
2020
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
YP WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12
Gateway2Gateway2Gateway1Gateway1
GC Thread
Intra WSs Parallelism
1
2
n
GC WS
TL WS
Gateway11Gateway11
YP WS
GC WS
TL WS
CC WS
Query (address)
Query (address)
Query (address)Query (address)
Query (address)
Query (address)
2121
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
YP WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12
Gateway2Gateway2Gateway1Gateway1
GC Thread
Intra WSs Parallelism
1
2
n
GC WS
TL WS
Gateway11Gateway11
YP WS
GC WS
TL WS
CC WS
Results (la
titude, lo
ngitude)
Results (la
titude, lo
ngitude)
Results (latitude, longitude)
Results (latitude, longitude)
Results (latitude, longitude)
Results (latitude, longitude)
2222
Centralized Centralized CoordinatorCoordinator
Broker WS
WLG
YP WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12
Gateway2Gateway2Gateway1Gateway1
GC Thread
Intra WSs Parallelism
1
2
n
GC WS
TL WS
Gateway11Gateway11
YP WS
GC WS
TL WS
CC WS
2323
Centralized Centralized CoordinatorCoordinator
Broker WS All results from YP, GC, TL
All results from YP, GC, TL
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
WLG
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
2424
Centralized Centralized CoordinatorCoordinator
Broker WS
Show resultsShow results
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
YP WS
GC WS
TL WS
CC WS
Gateway12Gateway12Gateway2Gateway2Gateway1Gateway1
WLG
2525
Query Type by Query Query Type by Query TypeType Setup 1:Setup 1:
query type 1 <= query type 2 <= query type 3query type 1 <= query type 2 <= query type 3 Setup 2:Setup 2:
query type 1 < query type 2 <=> query type 3query type 1 < query type 2 <=> query type 3 Setup 3:Setup 3:
query type 1 <= query type 2 < query type 3query type 1 <= query type 2 < query type 3 Querying with zip code doesn’t utilize Querying with zip code doesn’t utilize
multi-threading in query type 3multi-threading in query type 3
No WSs Parallelism
2626
Setup 1: Setup 1: query type 1 <= query type 2 <= query query type 1 <= query type 2 <= query type 3type 3
QuerytypeQuerytype11
Querytype2Querytype2 QuerytyeQuerytye33
8030380303 203.125203.125 218.75218.75 218.75218.75
3003230032 1265.6251265.625 1281.251281.25 1296.8751296.875
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
2727
Setup 2: Setup 2: query type 1 < query type 2 <=> query query type 1 < query type 2 <=> query type 3type 3
QuerytypeQuerytype11
Querytype2Querytype2 QuerytyeQuerytye33
8030803033
203.125203.125 218.75218.75 218.75218.75
3003300322
1234.3751234.375 1484.3751484.375 1484.3751484.375
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
2828
Setup 3: Setup 3: query type 1 <= query type 2 < query query type 1 <= query type 2 < query type 3type 3
QuerytypeQuerytype11
Querytype2Querytype2 QuerytyeQuerytye33
8030803033
218.75218.75 218.75218.75 265.625265.625
3003300322
1234.3751234.375 1484.3751484.375 1546.8751546.875
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
2929
Setup by SetupSetup by Setup
Query Type 1:Query Type 1: setup 1 => setup 2 <= setup 3setup 1 => setup 2 <= setup 3
Query Type 2:Query Type 2: setup 1 <= setup 2 <=> setup 3setup 1 <= setup 2 <=> setup 3
Query Type 3:Query Type 3: setup 1 <= setup 2 < setup 3setup 1 <= setup 2 < setup 3
setup1 which has only one service setup1 which has only one service in one node gives better ETsin one node gives better ETs
No WSs Parallelism
3030
Query Type 1: Query Type 1: setup 1 => setup 2 <= setup 3setup 1 => setup 2 <= setup 3
Setup1Setup1 Setup2Setup2 Setup3Setup3
8030380303 203.125203.125 203.125203.125 218.75218.75
3003230032 1265.621265.6255
1234.371234.3755
1234.371234.3755
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3131
Query Type 2: Query Type 2: setup 1 <= setup 2 <=> setup setup 1 <= setup 2 <=> setup 33
Setup1Setup1 Setup2Setup2 Setup3Setup3
8030380303 218.75218.75 218.75218.75 218.75218.75
3003230032 1281.251281.25 1484.371484.3755
1484.371484.3755
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3232
Query Type 3: Query Type 3: setup 1 <= setup 2 < setup 3setup 1 <= setup 2 < setup 3
Setup1Setup1 Setup2Setup2 Setup3Setup3
8030380303 218.75218.75 218.75218.75 265.625265.625
3003230032 1296.871296.8755
1484.371484.3755
1546.871546.8755
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3333
Intra WSs Parallelism vs. Intra WSs Parallelism vs. No WSs ParallelismNo WSs Parallelism Only setup3Only setup3 Query Type 1: Query Type 1:
Intra WSs Parallelism (IWP) < No WSs Parallelism Intra WSs Parallelism (IWP) < No WSs Parallelism (NWP)(NWP)
GC ET with NWP < GC ET with IWP.GC ET with NWP < GC ET with IWP. Query Type 2: Query Type 2:
NWP < IWPNWP < IWP Query Type 3:Query Type 3:
NWP < IWPNWP < IWP SQL Server overloading due to multi-SQL Server overloading due to multi-
threading causes high GC & TL ETsthreading causes high GC & TL ETs
3434
Query Type 1: Query Type 1: IWP < NWPIWP < NWP
IWPIWP NWPNWP
8030380303 203.125203.125 218.75218.75
3003230032 1007.81251007.8125 1234.3751234.375
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3535
Query Type 1: Query Type 1: GC ET with NWP < GC ET with GC ET with NWP < GC ET with IWPIWP
IWPIWP NWPNWP
8030380303 218.75218.75 125125
3003230032 2468.752468.75 1078.1251078.125
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3636
Query Type 2: Query Type 2: NWP < IWPNWP < IWP
IWPIWP NWPNWP
8030380303 328.125328.125 218.75218.75
3003230032 1640.6251640.625 1484.3751484.375
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3737
Query Type 3: Query Type 3: NWP < IWPNWP < IWP
IWPIWP NWPNWP
8030380303 312.5312.5 265.625265.625
3003230032 1687.51687.5 1546.8751546.875
Zip code
• Values are execution time (ET)Values are execution time (ET)• ET is medianET is median • ET is in millisecondsET is in milliseconds• # of outputs (80303) = 2# of outputs (80303) = 2• # of outputs (30032) = 16# of outputs (30032) = 16
3838
ConclusionsConclusions
Setup 1 which one node has only Setup 1 which one node has only one service has better Execution one service has better Execution TimeTime
Intra WSs Parallelism causes SQL Intra WSs Parallelism causes SQL Sever overloadingSever overloading
SQL Server overloading increases SQL Server overloading increases CC Execution TimeCC Execution Time
Need better broker which can Need better broker which can avoid SQL Server overloadingavoid SQL Server overloading