ieee international conference on autonomic computing (icac’06)
DESCRIPTION
IEEE International Conference on Autonomic Computing (ICAC’06). - PowerPoint PPT PresentationTRANSCRIPT
nanoHUB.org
1
online simulations and more
Network for Computational Nanotechnology
Autonomic Live Adaptation of Virtual Computational Environments in a Multi-Domain Infrastructure
Paul Ruth, Junghwan Rhee, Dongyan XuDepartment of Computer Science and Center for Education and Research in Information Assurance and Security (CERIAS)
Rick Kennell, Sebastien GoasguenRosen Center for Advanced Computing
Purdue UniversityWest Lafayette, Indiana, USA
IEEE International Conference on Autonomic Computing (ICAC’06)
nanoHUB.org
2
online simulations and more
Network for Computational Nanotechnology
Outline of Talk
• Motivations• Overall architecture• Design and implementation• Real-world deployment in nanoHUB• Related work• Conclusion• Demo
nanoHUB.org
3
online simulations and more
Network for Computational Nanotechnology
Motivations
• Formation of shared distributed cyberinfrastructure (CI) Spanning multiple domains Serving users/user communities with diverse computation needs Exhibiting dynamic resource availability and workload
• Need for virtual distributed environments (VIOLINs), each with Customizability and legacy application compatibility Administrative privileges Isolation, security, and accountability Autonomic adaptation capability - A unique opportunity brought by virtualization (VMs and VNs)
nanoHUB.org
4
online simulations and more
Network for Computational Nanotechnology
Adaptive VIOLINs
Internet
Duke U.
U. Florida
nanoHUB infrastructure@Purdue
Physical cluster
Virtual clusters(VIOLINs)
nanoHUB.org
5
online simulations and more
Network for Computational Nanotechnology
Autonomic VIOLIN Adaptation
• Adaptation triggers: Dynamic availability of infrastructural resources Dynamic resource needs of applications running inside
• Adaptation actions: Resource re-allocation Scale adjustment (adding/deleting virtual machines) Re-location (migrating virtual machines)
• Adaptation goals: Improving application performance Increasing infrastructural resource utilization Maintaining user/application transparency Minimizing infrastructure administrator attention
nanoHUB.org
6
online simulations and more
Network for Computational Nanotechnology
Research Challenges
• Autonomic live adaptation mechanisms VM Resource monitoring and scaling Application profiling and non-intrusive sensing of application needs Live VIOLIN re-location across domains
• Adaptation policies VIOLIN adaptation model
Infrastructure resource availability and topology Application resource needs Application configuration and topology
Optimal VIOLIN adaptation decision-making Goals (cost vs. gains)? When to adapt? How and how much to adapt? Where to migrate?
nanoHUB.org
7
online simulations and more
Network for Computational Nanotechnology
Dom0
Overall Architecture
VIOLIN Switch VIOLIN Switch
Monitoring Daemon
VIOLIN Switch VIOLIN Switch
Monitoring Daemon
VIOLIN Switch VIOLIN Switch
Monitoring Daemon
Monitoring DaemonAdaptationManager
Dom0
Dom0
Dom0
VMs VMs
VMsVMsPhysicalNetwork
Scale Up
CPU Update
Migrate
VIOLIN Switch
VMM
VMMVMM
VMM
nanoHUB.org
8
online simulations and more
Network for Computational Nanotechnology
VIOLIN Adaptation Policies
• Maintain desirable resource utilization level• Reclaim resource if under-utilized over a period• Add resource if over-utilized over a period
Scale up local resource share Migrate to other host(s)
Balance host workload Intra-domain migration first
Minimize migration • Re-adjust resource according to application needs
nanoHUB.org
9
online simulations and more
Network for Computational Nanotechnology
Implementation and Deployment
• Extension to non-adaptive VIOLIN Based on Xen 3.0 (w/ VM Live migration capability) Enabling live VIOLIN migration across domains
IP addresses of VMs Root file systems of VMs
Leveraging Xen libraries for VM resource monitoring (xenstat, xentop) Extending VIOLIN switch for inter-VM bandwidth monitoring
• Deployment in nanoHUB On-line, on-demand simulation service for nanotechnology community
Web interface for regular users “My workspace” interface for advanced users
Local infrastructure: two clusters in two subnets
nanoHUB.org
10
online simulations and more
Network for Computational Nanotechnology
nanoHUB Deployment Overview
Delegated trust
Local Virtual Machines
Migratable
Isolated from Local infrastructure
VIOLIN Virtual Cluster
Virtual Infrastructure over WAN
nanoHUB.org
11
online simulations and more
Network for Computational Nanotechnology
VIOLIN in nanoHUB
Simulationjob
VIOLIN
In the backround:
nanoHUB.org
12
online simulations and more
Network for Computational Nanotechnology
VIOLIN in nanoHUB
Autonomic property:Users focus on simulationsemantics and results, unaware of VIOLIN creation, setup, and adaptation.
nanoHUB.org
13
online simulations and more
Network for Computational Nanotechnology
Impact of Migration on App. Execution
End-to-end execution time of NEMO3D w/ and w/o live VIOLIN migration
nanoHUB.org
14
online simulations and more
Network for Computational Nanotechnology
VIOLIN Adaptation Scenario
Without Adaptation With AdaptationDomain 1 Domain 2
1. Initially VIOLIN 1, 2, 3 are computing, VIOLIN 2 is about to be finished.
Domain 1 Domain 2
2. After VIOLIN 2 is finished, before adaptation VIOLIN 1
VIOLIN 4
VIOLIN 5VIOLIN 3
VIOLIN 2
nanoHUB.org
15
online simulations and more
Network for Computational Nanotechnology
2. After VIOLIN 2 is finished, before adaptation3. After adaptation
VIOLIN Adaptation Scenario
Without Adaptation With AdaptationDomain 1 Domain 2 Domain 1 Domain 2
VIOLIN 1
VIOLIN 4
VIOLIN 5VIOLIN 3
VIOLIN 2
nanoHUB.org
16
online simulations and more
Network for Computational Nanotechnology
VIOLIN Adaptation Scenario
4. After VIOLIN 4, 5 are created
Without Adaptation With AdaptationDomain 1 Domain 2 Domain 1 Domain 2
3. After adaptationVIOLIN 1
VIOLIN 4
VIOLIN 5VIOLIN 3
VIOLIN 2
nanoHUB.org
17
online simulations and more
Network for Computational Nanotechnology
4. After VIOLIN 4, 5 are created5. After VIOLIN 1, 3 are finished
VIOLIN Adaptation Scenario
Without Adaptation With AdaptationDomain 1 Domain 2 Domain 1 Domain 2
VIOLIN 1
VIOLIN 4
VIOLIN 5VIOLIN 3
VIOLIN 2
nanoHUB.org
18
online simulations and more
Network for Computational Nanotechnology
6. ALL VIOLINs are finished5. After VIOLIN 1, 3 are finished
VIOLIN Adaptation Scenario
Without Adaptation With AdaptationDomain 1 Domain 2 Domain 1 Domain 2
VIOLIN 1
VIOLIN 4
VIOLIN 5VIOLIN 3
VIOLIN 2
nanoHUB.org
19
online simulations and more
Network for Computational Nanotechnology
Limitations and Future Work
• Simple, heuristic adaptation policy Application of machine learning and data mining techniques
• Centralized adaptation manager Hierarchical or peer-to-peer adaptation managers
• Imprecise application resource demand inference Multi-dimensional, fine-grain resource demand profiling
• Campus-wide infrastructure Evaluation and deployment in wide-area infrastructure
nanoHUB.org
20
online simulations and more
Network for Computational Nanotechnology
Related Work
• VNET (Northwestern U.)• Cluster-on-Demand (COD) (Duke U.)• Virtual Workspaces on Grid (Argonne National Lab)• SoftUDC (HP Labs)• WOW and IPOP (U. Florida)
nanoHUB.org
21
online simulations and more
Network for Computational Nanotechnology
Conclusions
• Autonomically adaptive virtual infrastructures (VIOLINs) A new opportunity brought by virtualization technologies Decoupled from underlying shared infrastructure Intelligent, first-class entities with user-transparent resource provisioning
• Key benefits Application performance improvement Infrastructure resource utilization Management convenience (at both virtual and physical levels)
“The Cray motto is: adapt the system to the application - not the application to the system.” - Steve Scott, CTO, Cray Inc. on “adaptive supercomputing”, March 2006
nanoHUB.org
22
online simulations and more
Network for Computational Nanotechnology
Thank you.For more information:
Email: [email protected]: http://www.cs.purdue.edu/~dxuGoogle: “Purdue VIOLIN FRIENDS”