proposed diffserv evaluation for emerge sites
DESCRIPTION
Proposed Diffserv Evaluation for Emerge Sites. Volker Sander [email protected] Argonne National Laboratory 1/20/2000. Outline. Basic GARA overview New GARA-features Basic set of experiments Installation of GARA Running the experiments using GARA Timeline. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/1.jpg)
Proposed Diffserv Evaluation for Emerge
SitesVolker Sander
Argonne National Laboratory1/20/2000
![Page 2: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/2.jpg)
2
Outline
Basic GARA overview New GARA-features Basic set of experiments Installation of GARA Running the experiments using GARA Timeline
![Page 3: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/3.jpg)
3
The Basic Diffserv Concept
Aggregate based scheduling implements EF PHB
Classify and Mark
packets
Shape Traffic
Flow based
Classify and Mark
packets
Shape Traffic
![Page 4: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/4.jpg)
4
Proposed Implementation
Client
GARA API
GARA Diffserv
Resource Manager
Server
![Page 5: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/5.jpg)
5
The Multidomain Extension
Server
GARA Diffserv
Resource
Manager
Client
GARA API GARA Diffserv
Resource
Manager
![Page 6: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/6.jpg)
6
GARA Globus Architecture for Reservation and
Allocation Three important contributions:
– Support for Advance Reservations (AG)
– Uniform treatment of underlying resources> Network, disk, CPU etc…
> Currently supported:
Differentiated Services, DPSS, and DSRT
– Layered architecture enables generic co-reservation agents
> User is able to select API best suitable for his need
![Page 7: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/7.jpg)
7
GARA Basics—Reservations There is a generic “reservation”, which has
several properties:– Start Time (“now” or future) and Duration
– Resource type/Underlying resource identifier
– Resource-specific (bandwidth, % CPU…) All reservations are treated uniformly:
– Create/Modify (Given properties)=> Returns Reservation Handle
– Destroy
– Monitor (Callbacks or Polling)
![Page 8: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/8.jpg)
8
A picture of what happens(creating a reservation)
Gatekeeper(Authenticate and Authorize)
CreateUser Program or Agent
Reservation Handle
Resource Manager
Gatekeeper Service
Local Resource Manager API (LRAM)
Resource (Router...)
![Page 9: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/9.jpg)
9
A picture of what happens(claiming a reservation)
User Programbind(handle)
LRAM API
Resource Manager
Setup reservation
ResourceEnforce Reservation
Note: GARA supports third party binding
![Page 10: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/10.jpg)
10
![Page 11: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/11.jpg)
11
Example: Network reservation In advance, make a reservation:
RSL:& (reservation-type=network)
(start-time=now) (duration=3600)
(endpoint-a=128.135.11.4) (endpoint-b=128.135.11.1)
(bandwidth=1000) (protocol=udp)
(reservation-subtype=low-latency)
Handle: Opaque string H1
At run-time:
– Bind(H1,RSL)
RSL:& (which-endpoint=a) (endpoint-a-port=5000)
(endpoint-b-port=5000)
![Page 12: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/12.jpg)
12
The Resource Manager To track advanced reservations, we use a simple “slot
table” We assume exclusive access to resource through the
resource manager We aren’t tied to this particular resource manager
% of available resource allocated
0%
100%
Time
![Page 13: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/13.jpg)
13
End-to-End Network Reservations Algorithm for a single network reservation
– Discover which gatekeepers/resource managers need to be contacted (MDS)
– Make reservation at each one.
User process Workstation
Resource Manager Resource Manager
MDSEnd-to-End Co-Reservation Agent
![Page 14: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/14.jpg)
14
The GARA API
globus_gara_reservation_create() Gatekeeper contact RSL reservation specification Reservation Handle or Error
globus_gara_reservation_modify() Old Reservation Handle RSL reservation specification New Reservation Handle or Error
![Page 15: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/15.jpg)
15
The GARA API (continued) globus_gara_reservation_cancel()
Reservation Handle Error
globus_gara_reservation_status() Reservation Handle Status/Error
globus_gara_reservation_callback_register() Reservation Handle Callback Function/User Parameter Error
![Page 16: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/16.jpg)
16
New GARA features New Reservation-Subtypes
– low-latency> Used to support jitter sensitive applications using the same EF
aggregate behavior as high-throughput TCP applications
> Based on traffic shaping and Priority-Queuing
> Currently being evaluated
– background> Used to support bulk transfers, including deadline support. Takes at
least the amount of premium traffic required to fulfill the deadline; more if not used actively
Monitoring of the network edges– Provides feedback to applications when they send too
fast
![Page 17: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/17.jpg)
17
Network + CPU Reservations
0
20000
40000
60000
80000
100000
0 50 100 150 200 250
Time
Ban
dw
idth
(K
bp
s)
TCP Flow Competitive Traffic
![Page 18: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/18.jpg)
18
Example of bulk transfer (LAN)
0
20000
40000
60000
80000
100000
0 50 100 150 200 250
Time
Ban
dw
idth
(K
bp
s)
backgroundforegroundcompetitive
![Page 19: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/19.jpg)
19
Example of bulk transfer (WAN)
0
10000
20000
30000
40000
50000
0 20 40 60 80 100 120 140 160 180
Time
Ban
dw
idth
(K
bp
s)
backgroundforegroundcompetitive
![Page 20: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/20.jpg)
20
Feedback from Monitoring the Edges
Receiver Bound BW
0
1000
2000
3000
4000
5000
6000
7000
8000
90001
11
21
31
41
51
61
71
81
91
10
1
11
1
12
1
13
1
14
1
15
1
16
1
17
1
18
1
Seconds
KB
/s
Series2
![Page 21: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/21.jpg)
21
Evaluation Tools UDP Traffic Generator
– Modified Version of Andy Adamson’s gen_send and gen_recv
> Evaluate admission control
> Creating competing traffic
– MGEN/Drec> Evaluate delay and Jitter for Premium UDP Flow
Modified Version of ttcp – GARA-enabled (wait for reservation)
– Support for a desired application rate> long term controllong term control
> short term controlshort term control
– Consecutive bandwidth reporting
– Bulk transfer ttcp
![Page 22: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/22.jpg)
22
Basic Experiments Evaluate Admission Control Evaluate premium TCP Flows under Congestion
– Analyze network traces with tcpdump Evaluate UDP
– Delay
– Jitter Evaluate Inter-Domain Issues
Fundamental idea: Get the necessary knowledge to be able to experiment with REAL applications
![Page 23: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/23.jpg)
23
Basic Experiment I Goal: Proof of Admission Control
– Gather packet statistics from gen_recv for different configurations and different traffic profiles:
UDP Rate comparison for different CAR buffer sizes(Desired rate: >40Mbps; Average rate limit 40 Mbps)
Packet size 530 Bytes
36500
37000
37500
38000
38500
39000
39500
40000
40500
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
Time (s)
Ac
tua
l R
ate
(K
bp
s)
100KB
1MB
2MB
Buffer size
![Page 24: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/24.jpg)
24
Basic Experiment I
gen_send
GARA Diffserv
Resource Manager
gen_recv
![Page 25: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/25.jpg)
25
Basic Experiment II Goal: Proof of Premium Quality for TCP flows
– Run modified ttcp with different traffic and reservation profiles; gather receiver bound statistics
– Create congestion using gen_send/recvStable Long Term TCP Traffic
(Attempted Rate: 6000KB/s, Buffer size: 2000000 bytes)
2000
3000
4000
5000
6000
7000
8000
9000
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97Time
Ac
tua
l R
ate
(K
bs
)
Rate
Limit
![Page 26: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/26.jpg)
26
Basic Experiment II Enhancement
– Use tcpdump and tcptrace get a deep overview on the behavior
– Use netlogger_enabled ttcp_gara and gen_recv to publish data into MDS. “Real Time Visualization”
![Page 27: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/27.jpg)
27
Basic Experiment II
GARA Diffserv
Resource Managerttcp -t
ttcp -r
gen_send gen_recv
![Page 28: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/28.jpg)
28
Basic Experiment III Goal: Demonstrate Premium Quality for UDP flows
– Run mgen/drec with different traffic and reservation profiles; gather receiver bound statistics
– Create congestion using gen_send/recv
![Page 29: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/29.jpg)
29
Basic Experiment III
GARA Diffserv
Resource Manager
mgen
drec
gen_send gen_recv
![Page 30: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/30.jpg)
30
Basic Experiment IV Goal: Proof Premium Quality for UDP flows
sharing one EF aggregate– Run mgen/drec with different traffic and
reservation profiles; gather receiver bound statistics
– Run high-throughput ttcp using a correct reservation
– Create congestion using gen_send/recv
![Page 31: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/31.jpg)
31
Basic Experiment IV
GARA Diffserv
Resource Manager
ttcp -t
ttcp -r
mgen drec
gen_recv
gen_send
![Page 32: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/32.jpg)
32
Additional Experiments
Bulk-Transfer Reevaluate inter-domain behavior
– Add Traffic-Shaping between domains Test Inter-Domain Admission control Prepare the Instrumentation Go with real applications
![Page 33: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/33.jpg)
33
Installing GARA Get GARA
– (currently through E-Mail from sander/roy @mcs.anl.gov) Get SSLeay 0.9.0 and install it Get Globus 1.1 configure and compile it
deploy is not necessary for doing tests but recommended
Run grid-cert-request to apply for a Grid X509 Certificate
add “gara-service stderr_log,local_cred - $GARA_DIR/gara/programs/globus_gatekeeper_gara_service -d”
to $GLOBUS_DEPLOY_DIR/etc/globus-services Read gara/docs/* Edit gara/build Run gara/build Customize GARA
– gara/resource_manager/programs/diffserv_manager.conf
– gara/resource_manager/programs/setup_flow.cfg
– (Samples are in the documentation directory)
![Page 34: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/34.jpg)
34
Building GARA $GARA_DIR/build #!/bin/ksh CC=cc CXX=CC CFLAGS="-g -v -mt" GLOBUS_DIR=/soft/pub/packages/globus-1.1.0/development/sparc-sun-solaris2.7_pt hreads_standard_debug SSL_LIBRARY_DIR=/soft/pub/packages/SSLeay-0.9.0/lib
export CC CXX CFLAGS GLOBUS_DIR SSL_LIBRARY_DIR
configure make clean make depend make all # # For portability reasons you might not want to do a make withdsrt.. # in case: just comment out this line. #make withdsrt
Run build
![Page 35: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/35.jpg)
35
Configuring GARA resource_manager/programs/diffserv_manager.confPublicationMethod web mds
LDAPDistinguishedName "dslnet1.mcs.anl.gov:34880:/C=US/O=Globus/O=Argonne Nati
onal Laboratory/OU=MCS/CN=roy”
MDSPublishPassword “password”
WEBPublishFile /home/sander/public_html/diffserv_manager_slots.html
NoOfRouters 2
IPAddressesServed[1] 140.221.48.162,140.221.48.146,140.221.48.164,140.221.48.1
65,140.221.48.166,140.221.48.167,140.221.48.34
IPAddressesServed[2] 140.221.48.98,140.221.48.114,140.221.48.102,140.221.48.10
3,140.221.48.104,140.221.48.105
Parameters: "LDAPDistinguishedName",
"MDSPublishPassword",
"WEBPublishFile",
"FILEPublishFile",
"Quantity",
"ClearSlotTable",
"SlotTableFilename",
"Verbose",
"PublicationMethod",
"DontReallyBindReservations"
![Page 36: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/36.jpg)
36
Configuring GARA resource_manager/programs/setup_flow.conf1_ROUTER 140.221.48.145
1_USERNAME my_userid
1_PASSWORD password
1_INTERFACE1 FastEthernet1/0/1 140.221.48.162
1_INTERFACE2 FastEthernet1/0/1 140.221.48.164
1_INTERFACE3 FastEthernet1/0/1 140.221.48.165
1_INTERFACE4 FastEthernet1/0/1 140.221.48.166
1_INTERFACE5 FastEthernet1/0/1 140.221.48.167
1_INTERFACE6 FastEthernet1/0/0 140.221.48.146
#
#
2_ROUTER baku
2_USERNAME my_userid
2_PASSWORD password
2_INTERFACE1 FastEthernet1/0/1 140.221.48.98
2_INTERFACE2 FastEthernet1/0/1 140.221.48.102
2_INTERFACE3 FastEthernet1/0/1 140.221.48.103
2_INTERFACE4 FastEthernet1/0/1 140.221.48.104
2_INTERFACE5 FastEthernet1/0/1 140.221.48.105
2_INTERFACE6 FastEthernet1/0/0 140.221.48.114
![Page 37: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/37.jpg)
37
Setting up the routers Get new IOS XE beta release
IOS (tm) RSP Software (RSP-PV-M), Experimental Version 12.0(19991129:193607) [rolsen-conn_isp_llq 144]
Available at ftp://ftpeng.cisco.com/rolsen/rsp-pv-mz.120699
Edge-Routers (Update of setup_flow required)class-map match-all EF
match ip dscp 46
class-map match-all priority-qos
match qos-group 99
policy-map emerge
class EF
bandwidth 30000
queue-limit 500
class priority-qos
priority
policy-map set-precedence
– Attach service-policy set-precendence to every input
– Attach service-policy emerge to every output interface
![Page 38: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/38.jpg)
38
Setting up the routers
Interior-Routers (Update of setup_flow required)class-map match-all EF
match ip dscp 46
policy-map emerge
class EF
bandwidth 30000
queue-limit 500
– Attach service-policy emerge to every output interface
![Page 39: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/39.jpg)
39
Updating setup_flow
setup_flow is the script which is configuring the router Possible update of setup_flow (is)
197 expect_after default { close; continue }
198
199 expect "name: " { send "$username\r" }
200 expect "word: " { send "$password\r" }
Might be (without tacacs+)197 expect_after default { close; continue }
198
199 expect ”word: " { send "$password\r” }
200 expect “#” { send “enable\r” }
201 expect "word: " { send "$password\r" }
![Page 40: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/40.jpg)
40
Updating the MDS
End-to-end API searches for end-systems in MDS Publish information for all end-systems into MDS
dslnet1.cldif:
---------------------
dn: hn=dslnet1.mcs.anl.gov, ou=Mathematics and Computer Science Division,
o=Argonne National Laboratory, o=Globus, c=US
…. If the machine is already in MDS this is enough
networkreservationcontact:
dslnet1.mcs.anl.gov:30000:/C=US/O=Globus/O=Argonne National
Laboratory/OU=MCS/CN=roy197 expect_after default { close; continue }
– run grid-info-update Tools will be available soon
![Page 41: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/41.jpg)
41
The GARA Source Tree Subdirectories represent different layers
– end2end
– gara (gara/java GUI)
– lram
– resource_manager
– slot_manager
– common
– logging
– simple_multi
– (dsrt)
We are providing examples for every layer docs directory contains lots of useful
information
![Page 42: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/42.jpg)
42
Doing Further Tests
Select your favorite application Determine port numbers edit (set endpoint systems) and run script:
gara/gara/utilities/demo_make_reservation Create congestion Try demo_modify_reservation for getting
different premium bandwidth Try demo_cancel_reservation Or use the GUI
![Page 43: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/43.jpg)
43
Timeline Establish connectivity (January)
– mostly done!
– Setup accounts for Emerge-User (January!)
– Check connectivity Install and customize GARA (Jan - Feb)
– partly done
– Configure and check with GUI without any application (Just look at differv_manager and router)
Do basic experiments (February-March) Start with multi-domain issues (March-April) Port real applications to the Testbed (Feb-?)
![Page 44: Proposed Diffserv Evaluation for Emerge Sites](https://reader036.vdocuments.site/reader036/viewer/2022081603/56814794550346895db4c85a/html5/thumbnails/44.jpg)
44
Questions...
Now or later… (sander/roy @ mcs.anl.gov)