pervasive computing self-organizing methods and models for...
TRANSCRIPT
2/17/11
1
Self-organizing Methods and Models for Software Development
Daniel J. Dubois Advisor: Prof. Elisabetta Di Nitto CoAdvisor: Prof. Raffaela Mirandola Tutor: Prof. Gianpaolo Cugola
Milan – February 17th, 2011
Motivation • Context:
– Dynamic and distributed software architectures – Changing environment and requirements
Milan, 17/02/2011 Self-organizing Methods and Models for Software Development 2
• Problem: – Find efficient adaptation strategies to deal with the
complexity of such architectures
Pervasive Computing
Examples
Existing Solutions (1/2) • Top-down adaptation approaches
– Adding one or more adaptation layers on top of the system elements
– Higher layers issue decisions for the lower layers
Milan, 17/02/2011 Self-organizing Methods and Models for Software Development 3
Advantages
• Efficient for small and medium systems
• Deterministic and optimal
Disadvantages
• Scalability issues due to complexity
• Difficult to deal with uncertainty
Existing Solutions (2/2) • Bottom-up adaptation approaches
– Adaptation emerges through simple decisions made by elements at the lower levels
– Inspired by natural phenomena – Also known as Self-organization
Self-organizing Methods and Models for Software Development 4
Advantages
• Simple coordination
• Scalable • Fault tolerant
Disadvantages
• Non-determinism • Less guarantees • Difficult to
engineer
Milan, 17/02/2011
2/17/11
2
Research Hypotheses
Self-organizing Methods and Models for Software Development 5
Assumptions: – Complex distributed software architectures – High number of elements at the same hierarchical level – High uncertainty and dynamism – No elements alone are essential
RH1: Usefulness of Self-organization
• Self-organization is an efficient approach to address selected classes of problems under the aforementioned assumptions
RH2: Engineering Real Self-organizing Systems
• The self-organization design guidelines we propose simplify the adoption of self-organizing approaches in real systems
Milan, 17/02/2011
Approach to Demonstrate Research Hypothesis 1
• Study and analyze selected recurrent problems in distributed software architectures: – Load Balancing problems – Energy reduction problems – Publish/Subscribe middleware optimization
problems
Self-organizing Methods and Models for Software Development 6 Milan, 17/02/2011
self
organizing
algor.
Milan, 17/02/2011 Self-organizing Methods and Models for Software Development 7
Problem 1: Decentralized Heterogeneous Load-balancing algorithm [SASO08, Bionetics08]
Self-Aggregation Load Balancing
7
2
0
4
9
5
2
2
2
0
4
7
5
9
3
5
3
5
3
5
5
Milan, 17/02/2011
100% load
25% load
25% load
75% load
hibernated
75% load
Problem 2: Energy Optimization in a Self-organizing Data Center [SOAR10]
8 Self-organizing Methods and Models for Software Development
2/17/11
3
Problem 3: Traffic Reduction in a Content-based Publish-subscribe System [ICAC09]
Publish: soccer
Subscribe: sport
Publish: soccer
Subscribe: sport
soccer
Broker 2
Broker 1
Broker 3
soccer
socce
r
After Self-organization: 1 saved message
Self-organizing Methods and Models for Software Development 9
Broker 1
Broker 3
Broker 2 soccer
soccer
soccer
soccer
Milan, 17/02/2011
Approach to Demonstrate Research Hypothesis 2
• Study a common software engineering approach for designing self-organizing software: – Identify commonalities in self-organizing algorithms
in terms of principles – Reference architecture and transposition of the
principles into design patterns – Design guidelines and validation approaches
Self-organizing Methods and Models for Software Development 10
self
organizing
algor. software
engineering ?
integration
Milan, 17/02/2011
Self-organization Principles • Derived from the observation of natural phenomena.
• They may be composed together and translated into real deployable algorithms.
• Most important evolutionary principles: – Noise “a strategy of errors” – Emergence “a team is better than the sum of its individuals” – Evolution “natural selection”
• Most important communication principles: – Diffusion “gossiping is not necessarily a bad habit” – Stigmergy “leave a note and I’ll read it”
Self-organizing Methods and Models for Software Development 11 Milan, 17/02/2011
Reference Architecture
Self-organizing Methods and Models for Software Development 12 Milan, 17/02/2011
2/17/11
4
Design Pattern for Diffusion (1/2)
Self-organizing Methods and Models for Software Development 13 Milan, 17/02/2011
Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09]
Identify principles/algorithms: − Analyze the problem/requirements. − Choose the principles/algorithms
that are most suitable for that class of problem.
Self-organizing Methods and Models for Software Development 14 Milan, 17/02/2011
Building Model: – Compose and instantiate principles/
algorithms identified so far. – Prove that the model satisfies its
requirements.
Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09]
Self-organizing Methods and Models for Software Development 15 Milan, 17/02/2011
Example of Model Construction and Validation: The Load Balancing case
• Abstract Model construction: – Graph representation – Transformation rules (may be non-
deterministic) – Expressed using TRIO language (first-order
temporal logic)
• Considered validation alternatives: – Automatic Model Checking on the TRIO model – Monte Carlo Simulations
Self-organizing Methods and Models for Software Development 16 Milan, 17/02/2011
2/17/11
5
Implementation: – Translate the model into a
deployable form. – Use the reference architecture. – Use self-organizing design
patterns based on the principles.
Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09]
Self-organizing Methods and Models for Software Development 17
Advantages
• Code reusability. • Separation between self-organizing aspects
and application logic
Milan, 17/02/2011 Milan, 17/02/2011
Example of Use of Reference Architecture: Publish-subscribe Traffic Optimization Case
Self-organizing Methods and Models for Software Development
Identify and solve problems not captured by the model: – Race conditions. – Component synchronization. – Frequency of iterations. – Amount of communication. – …
Creating Self-org Algorithms for Software Systems [SEAMS09, GIIS09]
Self-organizing Methods and Models for Software Development 19 Milan, 17/02/2011
Conclusions • In this thesis we explore the problem of applying self-
organization techniques to solve problems in distributed software architectures.
• Algorithmic contributions: – Distributed Decentralized Load Balancing in Heterogeneous
Networks. – Overlay Self-organization for Traffic Reduction in Public-
Subscribe Systems. – Energy Optimization in a Self-organizing data center.
• Methodological contributions: – A method for modeling self-organizing software based on the
self-organizing principles. – Examples of application of such method for implementing a
self-organization algorithm in a real software architecture.
Self-organizing Methods and Models for Software Development 20 Milan, 17/02/2011
2/17/11
6
Next Steps • Future challenges:
– Find optimal ways to self-calibrate self-organizing algorithms to reduce the needs of human-assisted preliminary simulations.
– Specialize the principles and the design patterns. – Collect design patterns into a library and make them
available in existing CASE tools and in ready-to-use parameterizable implementations.
– Extend the work with additional real case studies to support the validation of the patterns.
– Investigate more on the validation techniques for self-organization.
Self-organizing Methods and Models for Software Development 21 Milan, 17/02/2011
Publications Related to this Work (1) • International Conferences
1. [Bionetics07] E. Di Nitto, D. J. Dubois, R. Mirandola “Self-aggregation algorithms for autonomic systems”. In Bionetics ‘07, Budapest, Hungary.
2. [Autonomics07] D. Devescovi, E. Di Nitto, D. J. Dubois, R. Mirandola “Self-organization algorithms for autonomic systems in the SelfLet approach”. In Autonomics ‘07, Rome, Italy.
3. [SASO08] E. Di Nitto, D. J. Dubois, R. Mirandola, F. Saffre, R. Tateson “Self-Aggregation techniques for Load Balancing in Distributed Systems”. In SASO ’08 (short paper), Venice, Italy.
4. [Bionetics08] E. Di Nitto, D. J. Dubois, R. Mirandola “Applying self-aggregation to Load Balancing: Experimental results”. In Bionetics ’08, Hyogo, Japan.
5. [ICAC09] E. Di Nitto, D. J. Dubois, R. Mirandola “Overlay self-organization for traffic reduction in multi-broker publish-subscribe systems”. In ICAC ‘09 (short paper), Barcelona, Spain.
6. [ICSS10] D. J. Dubois, C. Nikolaou, M. Voskakis “A Model Transformation for Increasing Value in Service Networks through Intangible Value Exchanges”
7. [Bionetics10] B. A. Caprarescu, N. M. Calcavecchia, E. Di Nitto, D. J. Dubois “SOS Cloud: Self-organizing Services in the Cloud” in Bionetics ‘10 (work in progress paper), Boston, USA.
Self-organizing Methods and Models for Software Development 22 Milan, 17/02/2011
Publications Related to this Work (2) • Workshops and Doctoral Symposium
8. [SEAMS09] E. Di Nitto, D. J. Dubois, R. Mirandola “On Exploiting Decentralized Bio-inspired Self-organization Algorithms to Develop Real Systems”. In SEAMS ‘09 (ICSE Workshop), Vancouver, Canada.
9. [GIIS09] D. J. Dubois “Bio-inspired Self-organization Methods and Models for Software Development”, GIIS Doctoral Symposium, Salerno, Italy.
• Book chapters 10. [SOAR10] D. Barbagallo, E. Di Nitto, D. J. Dubois, R. Mirandola “A Bio-
inspired Algorithm for Energy Optimization in a Self-organizing Data Center”. Self-organizing Architectures, Springer, 2010.
11. [ARTDECO10] N. M. Calcavecchia, E. Di Nitto, D. J. Dubois, C. Ghezzi, V. Mazza, M. Rossi “Complex Autonomic Systems for Networked Enterprises: Mechanisms, Solutions and Design Approaches”. Adaptive infRasTructures for DECentralized Organizations, 2010 (to appear).
Self-organizing Methods and Models for Software Development 23 Milan, 17/02/2011
QUESTIONS
24 Self-organizing Methods and Models for Software Development Milan, 17/02/2011
2/17/11
7
ADDITIONAL SLIDES
Self-organizing Methods and Models for Software Development 25 Milan, 17/02/2011
Design Pattern for Diffusion (1/2)
Self-organizing Methods and Models for Software Development 26 Milan, 17/02/2011
Design Pattern for Diffusion (2/2)
Self-organizing Methods and Models for Software Development 27 Milan, 17/02/2011
28 Self-organizing Methods and Models for Software Development Milan, 17/02/2011
Design Pattern for Stigmergy (1/2)
2/17/11
8
29 Self-organizing Methods and Models for Software Development Milan, 17/02/2011
Design Pattern for Stigmergy (2/2)
30 Self-organizing Methods and Models for Software Development Milan, 17/02/2011
Design Pattern for Evolutionary Principles (1/2)
31 Self-organizing Methods and Models for Software Development Milan, 17/02/2011
Design Pattern for Evolutionary Principles (2/2)