Intensive InstrumentationIntensive Instrumentationand Monitoring forand Monitoring for

Software EcosystemsSoftware Ecosystems

Hausi A. MüllerHausi A. MüllerDepartment of Computer ScienceDepartment of Computer Science

University of VictoriaUniversity of Victoria

Dagstuhl Seminar Nº 08031Software Engineering for Self-Adaptive Systems

January 2008

22

Intensive Instrumentation and Intensive Instrumentation and Monitoring for Software EcosystemsMonitoring for Software Ecosystems

Reverse Engineering

andMigration

Technology

Autonomic Computing Technology

Self-*

Adaptive Software

EcosystemsMultiple Control Loops

33

Despite enormous successes in Despite enormous successes in migrating legacy systems to modern platforms,migrating legacy systems to modern platforms,

dealing with legacy software systemsdealing with legacy software systemsdoes not seem to get any easier!!does not seem to get any easier!!

So, what is the problem?So, what is the problem?

44

The Automation ConundrumThe Automation Conundrum

Over the past 50 years, computer systems have had a huge capacity to automateEnormous variety of tasksCost per task greatly reducedIncalculable benefitsUnprecedented success

Key challengesFurther declines in task costs by traditional are

subject to the law of diminishing returnsThe complexity of infrastructure management

threatens to outweigh the benefits of further automation

A. Spector, VP IBM Services and Software Research, 2003A. Spector, VP IBM Services and Software Research, 2003

55

Evolution of Software SystemsEvolution of Software Systems

Legacy systems are migrating tomodern platforms

Integration ofSoftware-Intensive Systems (ISIS)http://www.sei.cmu.edu/isis/

Systems of SystemsEnterprise integrationDynamic systems

• http://www.sei.cmu.edu/programs/ds/

66

Evolution of Software SystemsEvolution of Software Systems

Legacy systems Systems of Systems

Ultra-Large-ScaleUltra-Large-Scale(ULS) Systems(ULS) Systems

Software EcosystemsSoftware Ecosystems

77

Continuous EvolutionContinuous Evolution

Flawed assumption: software systems should support organizational stability and structure be low maintenance strive for high degrees of user acceptance

Continuous evolution: software systems should be under constant development can never be fully specified are subject to constant adjustment and adaptation [Truex99]

Good news for software engineering community (adaptive and reverse

engineering in particular) since this view guarantees research problems for years to come

Bad news most software engineering textbooks will have to be rewritten

Truex et al., Growing Systems in Emergent Organizations, CACM, 1999Truex et al., Growing Systems in Emergent Organizations, CACM, 1999

88

Independent Studies Confirm the Independent Studies Confirm the Notion of Continuous EvolutionNotion of Continuous Evolution

German SE Manifest [Broy06] Challenges for Software Engineering Research

ICSE 2006 Keynote [Boehm06] SE theses and antitheses for every decade from 1950 to 2020 He argues that “the ability of organizations and their products, systems,

and services to compete, adapt, and survive will depend increasingly on software and on the ability to integrate related software-intensive systems into systems of systems.”

SEI ULS [ULS06] Systems of systems are likely to evolve into Ultra-Large-Scale (ULS)

socio-technical ecosystems ULS ecosystems require a radically new perspective with respect to

design and evolution, orchestration and control, as well as monitoring and assessment.

SEI study suggests that traditional top-down engineering approaches are insufficient to tackle the complexity and evolution problems inherent in decentralized, continually evolving software

99

Ultra-Large-Scale (ULS) SystemsUltra-Large-Scale (ULS) Systems

Premise ULS systems will place an unprecedented demand on software

acquisition, production, deployment, management, documentation, usage, and evolution

Needed A new perspective on how to characterize the problem and

realize solutions Breakthrough research in concepts, methods, and tools beyond

current hot topics such as SOA or MDA Proposal

New solutions involving the intersections of traditional software engineering and other disciplines including fields concerned with people—microeconomics, biology, city planning, anthropology

L. Northrop et al., SEI, Ultra-Large-Scale Systems, June 2006L. Northrop et al., SEI, Ultra-Large-Scale Systems, June 2006

1010

Decentralized EcosystemsDecentralized Ecosystems For 40 years we have embraced the traditional centralized

engineering perspective for building software Central control, top-down, tradeoff analysis

Beyond a certain complexity threshold, traditional centralized engineering perspective is no longer adequate nor can it be the primary means by which ultra-complex systems are made real

Firms are engineered—but the structure of the economy is not; yet it adapts

The protocols of the Internet were engineered—but not the Web as a whole; and yet it adapts

Ecosystems exhibit high degrees of complexity, organization and adaptability—but not through top-down engineering

L. Northrop et al., SEI, Ultra-Large-Scale Systems, June 2006L. Northrop et al., SEI, Ultra-Large-Scale Systems, June 2006

1111

Change of PerspectiveChange of Perspective

FromFrom satisfaction of requirements satisfaction of requirements through traditional, top-down through traditional, top-down engineeringengineering

ToTo satisfaction of requirements by satisfaction of requirements by regulation of complex, decentralized regulation of complex, decentralized systemssystems

With adaptive systemsWith adaptive systemsand feedback loops and feedback loops How?How?

The system The system shall do this shall do this … but it may … but it may do this … do this … as long as it as long as it does this …does this …

1212

From Buildings to CitiesFrom Buildings to Cities

A ULS system will operate more like a citybuilt or conceived by many individuals over long

periods of time (Rome)the form of the city is not specified by requirements, but

loosely coordinated and regulated—zoning laws, building codes, economic incentives (change over time)

Every day in every city construction is going on, repairs are taking place, modifications are being made—yet, the cities continue to function

ULS systems will not simply be bigger systems: they will be interdependent webs of software-intensive systems, people, policies, cultures, and economics

SEI, Ultra-Large-Scale Systems, June 2006SEI, Ultra-Large-Scale Systems, June 2006

1313

Characteristics of ULS SystemsCharacteristics of ULS Systems

Decentralized control Inherently conflicting, unknowable, and diverse

requirements Continuous evolution and deployment Heterogeneous, inconsistent, and changing

elements Erosion of the people/system boundary Software and hardware failures are the norm

1414

Challenges in ULS SystemsChallenges in ULS Systems

Design and evolution Rules and regulations Enforcement mechanisms and processes Integration Emergent quality and behaviour

Orchestration and Control Online modification Maintenance of quality and service Creation and execution of policies and rules Adaptation to users and contexts Enabling user controlled orchestration

Monitoring and Assessment Defining indicators Understanding why indicators change Handling change and imperfect information Gauging the human elements

http://www.sei.cmu.edu/uls/

1515

Unprecedented Levels of MonitoringUnprecedented Levels of Monitoring

To be able to observe and possibly orchestrate the continuous evolution of software systems in a complex and changing environment, we need to push the monitoring of evolving systems to unprecedented levels

Instrument software-intensive systems with autonomic elements using reverse engineering technology to enhance their monitoring and assessment capabilities

1616

How to simplify software?How to simplify software?

Historically, in contrast to engineering disciplines, computing science seems to have neglected the control loop

To simplify programs,To simplify programs,inject control loops or inject control loops or autonomic elementsautonomic elementsinto ordinary programsinto ordinary programs

Self-advertising infrastructure Self-advertising infrastructure components [Chris Craddock, CA]components [Chris Craddock, CA]

Knowledge

Plan

ExecuteMonitor

Analyze

Sensors Effectors

Sensors Effectors

1717

Autonomic Program MonitorsAutonomic Program Monitors

Run-time check monitorsMonitor assertions and invariantsMonitor frequency of raised exceptionsContinually measure test coverageData structure load balancingBuffer overflows, intrusionMemory leaksChecking liveness properties

1818

Satisfaction of RequirementsSatisfaction of Requirements

Monitor the satisfaction of requirementsPerform critical regression tests regularly to

observe satisfaction of requirementsPerform V&V operations (transformations)

regularly to ascertain V&V propertiesHow to monitor functional and non-functional

requirements when the environment evolves?

1919

Design programs with theAutonomic Element as a

Core Architectural Component

Teach the notion of a Control LoopTeach the notion of a Control Loopin 1in 1stst Year of Computer Science (and Engineering) Year of Computer Science (and Engineering)

M. Shaw. Beyond Objects: A Software Design Paradigm based on Process M. Shaw. Beyond Objects: A Software Design Paradigm based on Process Control, Control, ACM SIGSOFT Software Engineering NotesACM SIGSOFT Software Engineering Notes, 20(11):27–38, 1995 , 20(11):27–38, 1995

2020

Challenge for our communityChallenge for our community

To instrument software systems with manageability endpoints—sensor and effectors

To monitor software systems and their environments at unprecedented levels

2121

The Most Famous Autonomic SystemThe Most Famous Autonomic System

TemperatureHeart rate

Breathing rateBlood pressure

Blood sugarPupil dilation

TearsDigestion

Immune response

Autonomic Nervous System─ANS Parasympathetic

Day-to-dayDay-to-day internal processes

Sympathetic StressfulStressful situation processes

Dec

ide

Res

ourc

e

Mea

sure

Con

trol

Monitor and RegulateMonitor and Regulate

2222

Intensive Instrumentation and Intensive Instrumentation and Monitoring for Software EcosystemsMonitoring for Software Ecosystems

Reverse Engineering

andMigration

Technology

Autonomic Computing Technology

Self-*

Adaptive Software

EcosystemsMultiple Control Loops