management of complexity in system design of large it solutions
DESCRIPTION
The capability to manage complexity is one of the key competencies of system engineers for large IT-solutions. We call a technical system "complex" (in contrast to "complicated") if it is impossible (due to the networked interaction of its components) to predict the behavior of the whole system, even if you know exactly how each of the system components behave. On the other hand, customers expect increasingly high reliability of IT systems as their business is more and more dependent on the proper operation and interoperation of the IT systems. First we show how a network of interactions increases the complexity of the overall-system. Then we analyze the complexity management strategies of our system engineers and present generalized strategies based on examples of large customer projects. The examples demonstrate that a high maturity in managing complexity enables to provide IT solutions of ultra-high reliability even if they are complex solutions in the above defined sense.TRANSCRIPT
Management of Complexity in System Design for Large IT-Solutions
Dr. Michael HeissGlobal Vice President for Knowledge, Innovation & Technology
Dipl.-Ing. Stefan HuberSenior Architekt
Siemens IT Solutions and Services
Management of Complexity in System Design for Large IT-Solutions
Dr. Michael HeissGlobal Vice President for Knowledge, Innovation & Technology
Dipl.-Ing. Stefan HuberSenior Architekt
Siemens IT Solutions and Services
© Siemens AG Austria 2009. All rights reserved.
Definition
Example: Networking generates complexity
Categories of Complexity
Strategies for Management of Complexity
Agenda
© Siemens AG Austria 2009. All rights reserved.Page 3
A pragmatic definition of complexity
We call a technical system complex
(in contrast to complicated), if it is impossible to predict the behaviour of the whole system,
even if you know exactly how each of the system components
behave and interact.
© Siemens AG Austria 2009. All rights reserved.Page 4
Just a simple example
System 1
0
0,2
0,4
0,6
0,8
1
0 1 2 3 4 5 6 7 8 9
Takt
Au
sgan
swer
t
x(t=0..4) = 0,8
x(t > 4) = 0,1 System 3
0
1
2
3
4
0 1 2 3 4 5 6 7 8 9
Takt
Au
sg
an
sw
ert
System 2
0
0,05
0,1
0,15
0,2
0 1 2 3 4 5 6 7 8 9
Takt
Au
sg
an
sw
ert
Delayone stepDelay
one step
y = x - x²y = x - x²
y = 3,8xy = 3,8x
3 very simple elements
3 simple behaviors
“perfectly predictable”
x y
x y
x y
© Siemens AG Austria 2009. All rights reserved.Page 5
y1
x = y1 y3
x = y3
y2 x = y2
Just a simple example
y = x - x²y = x - x² y = 3,8xy = 3,8x
Complexity is generatedby interaction Delay
one stepDelay
one step
© Siemens AG Austria 2009. All rights reserved.Page 6
Just a simple example … ???
00,10,20,30,40,50,60,70,80,9
1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Takt
Aus
gang
swer
t
y1 volle Genauigkeity1 auf drei Kommastellen
Complexity is generatedby interaction.The result is…
Even the best supercomputer of the world cannot predict more than 300 steps
full precisionlimited precision
Time steps
© Siemens AG Austria 2009. All rights reserved.Page 7
A real-life software example
In a real world software system the number of elements, interactions and dependencies is far bigger than in the simple “toy” examples.Example: Intelligent Networks service platform for telecom systems(more than 100 customers worldwide)
© Siemens AG Austria 2009. All rights reserved.Page 8
Where do we meet complexity?
Complexity of the „problem space“: Real world phenomena to be solved by means of hardware and software
Complexity of the „solution space“:How the problems are solved through hardware and software
© Siemens AG Austria 2009. All rights reserved.Page 9
Where do we meet complexity?
Organizational complexityStructure of organizations and teams
Process complexityHow solutions are developed by the means of processes, methods, tools and technologies
Definition
Example: Networking generates complexity
Categories of Complexity
Strategies for Management of Complexity
Agenda
© Siemens AG Austria 2009. All rights reserved.Page 11
Requirements Engineering – a cycle of detecting and reducing complexity
Detecting complexity(problem space)
Info for the requirements engineer from various sources (requirements documents, interviews with stake holders, discussing prototypes, market studies,...)
The world is more complex than it seems to be at first sight
Reducing complexity(solution space)
Distilling abstractions out of multiple input, finding out which functions are really needed by a customer (and not everything that is stated as requirement)
Make the solution as simple as possible, as complex as needed
© Siemens AG Austria 2009. All rights reserved.Page 12
Usability Engineering – make solutions “user friendly”
Find the right balance in the complexity of the solution and avoid cognitive overload of users
Usage scenario driven requirements engineering (typical tasks are most important)
Prototyping with customer involvement
Standardization of user interfaces (“the Apple way...”)
Staging concepts of functionality (“one click shopping” vs. step by step)
Usability testing with “thinking aloud” technique
Usability testing
Paper prototyping
Source: SIS PSE Support Center Usability
© Siemens AG Austria 2009. All rights reserved.Page 13
Divide and conquer?
Old wisdom of statesmen and commanders
Classical principle for trying to reduce complexity:
Break down a problem into two or more sub-problems of a similar type, until these become simple enough to be solved directly+ focussing - per definition not sufficient for complex systems
The Tower of Hanoi puzzle: A simple algorithm applied recursively
Source: Wikipedia
© Siemens AG Austria 2009. All rights reserved.Page 14
Metrics are useful indicators of complexity
Complexity of requirements: Function Point Analysis
Complexity of code:
McCabe metrics (cyclomatic complexity)
Halstead metrics (static analysis)
Can be a hint showing
Maintainability
Risk of errors
Code worth being reworked or checked e.g. by a senior developer
© Siemens AG Austria 2009. All rights reserved.Page 15
Managing Software Complexity by Patterns
SuboptimalSoftwareDesign
Monolithic orAd-hoc design• Complexity not manageable
• Difficult to understand
• Error-prone
© Siemens AG Austria 2009. All rights reserved.Page 16
Managing Software Complexity by Patterns
SuboptimalSoftwareDesign
Monolithic orAd-hoc design• Complexity not manageable
• Difficult to understand
• Error-prone
© Siemens AG Austria 2009. All rights reserved.Page 17
Managing Software Complexity by Patterns
SoftwareDesign
Patterns
SoftwareDesign
Patterns
Managing complexity by providing higher level abstractions.
Reuse of engineeringknow-how
Patterns can be• named• taught• communicated
© Siemens AG Austria 2009. All rights reserved.Page 18
Managing Software Complexity by Patterns
Business LayerBusiness Layer
Data LayerData Layer
Web / UI* LayerWeb / UI* Layer
SoftwareArchitecture
Patterns
SoftwareArchitecture
Patterns
Patterns at Software architecture level
Design & ArchitecturePatterns Trainingin ourSoftware Architect Curriculum
* UI...User Interface
© Siemens AG Austria 2009. All rights reserved.Page 19Page 19
Architectural Qualities and Tactics
Architectural Qualities (related to non-functional requirements)
E.g. performance, availability, maintainability
Architectural Tactics
Tactics to achieve certain architectural qualities
More than patterns, tactics provide straight-forward approach from non-functional requirements to a design solution
Several architectural tactics support managing complexity
TacticsTactics PatternsPatterns
guides selection of
implements a collection of
QualitiesQualities
are achieved by using
© Siemens AG Austria 2009. All rights reserved.Page 21
Organizational patterns – experience based practices to act successfully in a specific context
Source: Siemens IT Solutions and Services SDE Support Center PM
© Siemens AG Austria 2009. All rights reserved.Page 22
Acting with responsibilities instead of detailed process descriptions
Defining responsibilities as anchor point for software delopment
Performing software design via thinking in responsibilities(e.g. in distributed development)... Provides a basis for independent, local decision making Decouples development process Similarity to management theory (away from Taylorism)
Techniques in software development: CRC Cards (Kent Beck / Ward Cunningham)
Classes – Responsibilities – Collaborations “Responsibility Driven Design”
Design by Contract
Precondition – Invariant - Postcondition
© Siemens AG Austria 2009. All rights reserved.Page 23
Safety nets in development processes
Reviews and inspections of intermediate results (specifications, plans, test cases, etc.)
Quality gates in company internal processes
Impact analyses – identifying the consequences of changes
Test driven development (specify test cases prior to developing code)
Process improvement (root cause analyses, project experience workshops, CMMI-assessments,...)
© Siemens AG Austria 2009. All rights reserved.Page 24
Safety nets in product design
Software Architectural Tactics exist to provide safety nets in the product: “Be prepared for the unforeseeable” (self-healing systems, high system availability, etc.)
Examples:
Process / system monitoring software
Restart routines after error detection(“watchdogs”)
Load balancers
Redundancies, Voting
HeartbeatWorker 1 Worker 2 Worker 3
Load balancer
© Siemens AG Austria 2009. All rights reserved.Page 25
Agile Software Development - the new paradigm for Software Engineering
Time boxing (fixed schedule and effort, functionality to be prioritized by customer – only really important functions are developed)
Iterative development with short iteration cycles (fast changes possible)
Refactoring (continuous improvement of design)
Test driven development
Pair programming
Self-organizing team,Scrum Master in supportive role
Source: Siemens IT Solutiond and Services SDE System Engineering Method SEM
© Siemens AG Austria 2009. All rights reserved.Page 26
Depending on the angle of view the same complexity might be easier to handle
v1
v2
view v1
view v2
?
!
Example from Computational Intelligence:4 different „languages“ to tell the computer what to do
Presenting the computer:
• Examples (neural networks, case based reasoning)
• Rules (fuzzy logic, expert systems)
• Fitness function (optimization algorithms, genetic algorithms)
• Commands (classical program code based on classical system modeling)
„No free lunch“: non of them is better for all problems
Compare to • management theories (all 4)• Test driven development (first: examples)• Definition of non-functional requirements (first 3)
correlation visible
Systems have increased interconnectivity This leads to an increased complexity Customers need high availability
for mission critical IT systems
A high maturity in management of complex systems is required to deliver ultra-highly available and complex IT systems
Conclusion
© Siemens AG Austria 2009. All rights reserved.Page 28
Assoc. Prof. Dr. Michael Heiss Global Vice President for Knowledge, Innovation and TechnologyStefan Huber, Senior ArchitectDr. Benedikt Lutz, Head of Learning Network Martin Arnhof, Student
Siemens AG ÖsterreichSiemens IT Solutions and ServicesGudrunstraße 11
1100 Vienna AustriaPhone +43-5-1707-46560 Fax +43-5-1707-56591 Mobile +43-664-8855 1526 mailto:[email protected]
Thank you for your attention!