enhancing assert: making an accurate testbed friendly ehsan nourbakhsh, ryan burchfield, s....
TRANSCRIPT
1
Enhancing ASSERT: Making an Accurate Testbed Friendly
Ehsan Nourbakhsh, Ryan Burchfield, S. Venkatesan, Neeraj Mittal, Ravi Prakash
Department of Computer ScienceUniversity of Texas at Dallas
2
Motivation
• Made observations in [RFiJ] regarding experimentation
• In [ASSERT] we proposed nine main propositions, and built a testbed based on them– Advanced wireleSS Environment Research Testbed
• Our focus changed from only fidelity to also include usability– Proper usage of the testbed should not require
comprehensive knowledge of the design and implementation of the testbed.
3
Propositions
• Accuracy– accurately reflect wireless network behavior
• Controllability – provide enough control to configure topology and environment conditions
• Mobility– emulate mobility of the nodes
• Repeatability – conduct experiments that are reproducible and easily repeatable
• Cost effectiveness – be cost effective in terms of hardware, manpower, space and time requirements to set up, run experiments
on and maintain• Data collection
– provide necessary tools to collect and analyze data• Resource sharing
– be able to share the available resources to conduct multiple experiments without interfering with each other• Multi-nodal capability
– support many types of nodes• Scalability
– have the ability to scale to a large number of nodes
4
Hardware Design
• Site: unit for hardware design• Microprocessor runs Linux• Black-box view of Unit Under Test (UUT)• Control and interaction with the UUT
5
Hardware Design (contd.)
6
Hardware Design (contd.)
• RF board
7
Hardware Design (contd.)
8
Hardware Design (contd.)
9
Software Slices
• software is divided into slices– each slice implements a specific functionality
• Some of the major ones– diagnostics– user interface– experiment control– topology mapper– attenuator– UUT control
10
Software Slices (contd.)
11
Feedback Analysis
• Graduated from “functionality centric” to “user centric”– “it works” vs. “it is easy to work with”
• Ease of running experiments results in expectation of enhanced tools– easily upload their custom images to UUTs– ability to verify proper experiment start and progress – tools for debugging and investigating collected data– possibility of creating customized distribution models.
12
Capability and Flexibility
• Cable Map• Topology Maker• Topology Mapping• Distribution Cleanup• Application Parameters
13
Cable Map
14
Topology Maker
15
Topology Mapping Time
Heuristic Mean (s) Std. Dev. (s) Min (s) Max (s)
None 15.79 13.94 0.60 47.30
1 Desc. 1.45 1.45 0.10 3.30
1 Asc. 68.24 49.63 18.40 183.70
2 Desc. 19.58 28.45 0.60 97.70
1 Desc. + 2 Desc. 0.14 0.15 0.006 0.40
1 Asc. + 2 Desc. 99.17 69.04 25.6 256.00
1 Desc. + 2 Asc. 4.56 0.96 3.80 8.00
1 Asc. + 2 Asc. 0.21 0.31 0.02 1.20
16
Verification
• Log Viewer• Remote Access
17
Log Viewer
18
Task Average Time (s)Create Images 5.3Reservation 4.2Synchronizing Clocks 18Upload images to UUTs 17.1Create Attenuation Model 0.01Restarting UUTs 1.4Enabling Logs 0.9Renew Reservation 2.6Run experiment 1941.5Disable Logging 1.7Abort Reservations 1.2Total (excluding run experiment) 52.9
Run TimeAverage time for each task over three runs for a 30 minute experiment with 24 nodes
19
Data Collection
• Timestamp Adjustments in UUT Logs• Log Download• RSS Log
20
RSS Log Viewer
Unicast by 1013
ACK by 1007
Broadcast by 1007
Broadcast by 1013
Reception by 1007, 1016
21
References
• [RFiJ] Ryan Burchfield and Ehsan Nourbakhsh and Jeff Dix and Kunal Sahu and S. Venkatesan and Ravi Prakash, “RF in the Jungle: Effect of Environment Assumptions on Wireless Experiment Repeatability,” ICC 2009
• [ASSERT] Ehsan Nourbakhsh and Jeff Dix and Paul Johnson and Ryan Burchfield and S. Venkatesan and Neeraj Mittal and Ravi Prakash, “ASSERT: A Wireless Networking Testbed,” TridentCom 2010