lean & agile systems engineering for systems of systems dr. david f. rico, pmp, csm website:...

Lean & AgileSystems

Engineeringfor Systems of Systems

Dr. David F. Rico, PMP, CSM

Website: http://davidfrico.comLinkedIn: http://www.linkedin.com/in/davidfrico

Facebook: http://www.facebook.com/profile.php?id=1540017424

2

Agenda

Introduction Systems Engineering

Systems Engineering Challenges Lean Systems Engineering

Agile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

3

Author DoD contractor with 25+ years of IT experience B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys. Large gov’t projects in U.S., Far/Mid-East, & Europe

Published six books & numerous journal articles Expertise in metrics, models, & cost engineering Adjunct at George Washington, UMUC, & Argosy Six Sigma, CMMI, ISO 9001, DoDAF & DoD 5000 Agile Program Management & Lean Development

4

Purpose of Briefing Provide an overview of traditional, lean, and

agile systems engineering concepts: Define systems engineering, its purpose, and

identify major approaches to traditional systems development

Identify the strengths and weaknesses of traditional systems engineering for today’s ever changing world

Discuss lean and agile systems engineering as a means of managing ever increasing system complexity

Introduce mechanisms for scaling lean and agile systems engineering for larger systems of systems

Examine iterative testing techniques within agile systems engineering for verification and validation

5

AgendaIntroduction

Systems EngineeringSystems Engineering Challenges

Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

6

What is Systems Engineering? Sys-tem (sĭs-'təm): Interacting, interrelated,

interdependent elements; A complex whole Interdisciplinary approach and means to

enable the realization of successful systems [INCOSE] Interdisciplinary tasks required to transform

customer needs into a system solution [IEEE] Interdisciplinary approach for transforming a set of

customer needs into a product solution [CMMI] Interdisciplinary approach for translating mission needs

into operational systems [DoD 5000] Interdisciplinary processes spanning the conception

of ideas through the retirement of a system [ISO]

7

Purpose of Systems Engineering Manage increasing system complexity (1950s) Optimize [sub]system performance (1960s) Improve system cost and quality (1970s)

Eisner, H. (2002). Essentials of project and systems engineering management. New York, NY: John Wiley & Sons.Blanchard, B. S., & Fabrycky, W. J. (2006). Systems engineering and analysis. Upper Saddle River, NJ: Pearson Prentice-Hall.

8

MIL-STD-1521B Created by U.S. Air Force in 1976 Framework for system and software reviews Standardized milestone reviews and technical audits

U.S. Department of Defense. (1985). Military standard: Technical reviews and audits for systems, equipments, and computer software (MIL-STD-1521B). Washington, DC: Air Force Systems Command (AFSC).

Note: ActualTime Framesof Activities

Must BeTailored to

Each Program

Test

DraftSystem

RequirementsSpecification

SystemRequirementsSpecification

HWCIDevelopmentSpecification

SoftwareTop Level

DesignDocument

SoftwareListing

HWCIProduct

Specification

SoftwareProduct

Specification

IRSDBDD

DraftHWCIProduct

Specification

Sourceand

ObjectCode

SystemRequirements

Review

SystemDesignReview Software

SpecificationReview

PreliminaryDesignReviewCSCI

PreliminaryDesignReviewHWCI

CriticalDesignReviewCSCI

CriticalDesignReviewHWCI

TestReadinessReview

FunctionalConfiguration

AuditHWCI

PhysicalConfiguration

AuditHWCI

FormalQualification

ReviewHWCI


AuditCSCI


AuditCSCI

SystemFunctional

ConfigurationAudit

SystemPhysical

ConfigurationAudit

SystemFormal

QualificationReview

Prepare Testand

EvaluationMaster Plan

PrepareSoftware

TestPlan

PrepareHWCITestPlan

PrepareSoftware

TestDescription

PrepareHWCITest

Procedures

PrepareSoftware

TestProcedures

PerformHWCI

SubsystemTests

PerformSoftware

Tests

PrepareSoftware

TestReports

PerformSystemTests

PrepareSystem

TestReports

ProgramActivity

ConceptExploration

Phase

Demonstrationand Validation

PhaseFull Scale Development Phase

TechnicalReviews

andAudits

Specifica-tions and

otherProducts

SystemFunctional

Identification

AllocatedIdentification

Developmental Configuration(HWCI Only)

Product Identification

SystemFunctionalBaseline

AllocatedBaseline

ProductBaseline

SoftwareRequirementsSpecification

IRDSoftwareDetailedDesign

Document

9

MIL-STD-498 Created by U.S. Navy in 1994 Consolidated multiple U.S. DoD standards Software process and documentation standard

Project Planning and

Oversight

System Requirements

Analysis

SystemDesign

Software Requirements

Analysis

Software Implementation and Unit

Testing

UnitIntegrationand Testing

CSCI Qualification

Testing

CSCI/HWCI Integrationand Testing

System Qualification

Testing

Preparing for Software Use

Preparing for Software TransitionPreliminary Detailed

Software DesignSoftwareActivity

QualityAssurance

Configura-tion

Manage-ment

Baseline Functional Allocated Product(Developmental Configuration)


Audit(FCA)


Audit(PCA)


Audit(FCA)


Audit(PCA)

Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit Audit

J ointReview

FormalQualification

Review(FQR)

FormalQualification

Review(FQR)

SystemRequirements

Review(SRR)

SystemDesignReview(SDR)

SoftwareSpecification

Review(SSR)

PreliminaryDesignReview(PDR)

CriticalDesignReview(CDR)

TestReadiness

Review(TRR)

Program Definition and Risk ReductionConcept Exploration Engineering and Manufacturing DevelopmentAcquisition

Phase

TechnicalReview

· Walkthru· Inspection











· SPS· SVD· SUM· SIOM· SCOM· COM

· SSDD· CPM· FSM

· STD· STR

· STD· STR

· SDD· IDD· DBDD

· SDD· IDD· DBDD

· SRS· IRS

· SSDD· IDD· DBDD

· OCD· SSS· IRS

· SDP· STP· SIP· STrP

SoftwareProduct

U.S. Department of Defense. (1994). Military standard: Software development and documentation (MIL-STD-498). Arlington, VA: Space and Naval Warfare Center (SPAWAR).

10

ISO-15288 Created by ISO/IEC around 2002 Standardization of international practices Meant for complex, computer-based systems

International Organization for Standardization/International Electrotechnical Commission. (2002). Standard for systems engineering: System life cycle processes (ISO/IEC 15288). Geneva, Switzerland: Author.

11

CMMI Created by the SEI in 2002 Merger of SW-CMM, SA-CMM, IPD-CMM, etc. Used for systems engineering process improvement

CMMI Product Team. (2006). CMMI for development version 1.2 (CMU/SEI-2006-TR-008). Pittsburg, PA: Software Engineering Institute.

12

DoD Acquisition Lifecycle Created by the U.S. DoD around 2003 Latest evolution of acquisition best practices Meant for large-scale, multi-billion weapon systems

DAU. (2009). Integrated defense acquisition, technology, and logistics life cycle management framework. Retrieved October 9, 2009, from https://acc.dau.mil/ifc

13

Systems Engineering Benefits Study funded by Australian defense institute Almost 44 programs studied from 2001 to 2004 Systems engineering minimizes schedule overruns

Honour, Eric C. (2009). Demographics in measuring systems engineering return on investment (SE-ROI). Proceedings of the Joint 19th Annual International Symposium of INCOSE/Third Asia-Pacific Conference on Systems Engineering (INCOSE/APCOSE 2009), Singapore.

14

Systems Engineering Studies U.S. Air Force Center for Systems Engineering Case studies of 9 major U.S.Air Force Programs Programs had significant cost and technical issues

Air Force Institute of Technology (AFIT). (2009). Systems engineering case studies. Retrieved October 19, 2009, from http://www.afit.edu/cse/cases.cfm

PROGRAM NAME EST. COST EST. UNIT ACT. COST ACT. UNIT OVERRUN OTHER ISSUES

A-10 Thunderbolt $2,547,600,000 439 $5,511,490,000 360 21,005% Wing Reliability

B-2 Spirit $58,200,000,000 132 $45,300,000,000 20 57,949% Low Observability

C-5A Galaxy $3,413,200,000 120 $4,426,400,000 81 7,585% Maintenance

F-111 Aardvark $2,171,590,000 547 $8,652,000,000 547 298% Structural

RQ-4A Global Hawk $2,904,600,000 51 $3,816,700,000 51 31% Quality

GPS Navstar $814,400,000 28 $8,650,000,000 14 31,764% Sofware Reliability

HST Hubble $651,200,000 1 $1,458,900,000 1 124% Schedule, Mirror

K C-10 Extender $1,055,000,000 20 $1,090,600,000 20 3% Reliability

LGM-118A Peacekeeper $16,600,000,000 21 $16,600,000,000 21 0% Range

$9,817,510,000 151 $10,611,787,778 124 13,196% $85,655,686

15

AgendaIntroductionSystems Engineering

Systems Engineering Challenges Lean Systems Engineering


16

What is a Challenge? Chal-lenge (chăl-'ənj): Contest, competition,

fight, defy, confront, or dispute; To question 21st century systems are more software-intensive and

highly-complex with numerous invisible parts Technology is evolving at an exponential rate of

change which severely limits the planning horizon Global competitiveness has intensified and new

military threats are rapidly emerging all of the time Customers have unpredictable needs and necessitate

decision-making flexibility throughout the program Today’s post-industrial information age knowledge workers

need new systems engineering approaches

17

Information Age U.S. is no longer an industrial-age nation U.S. part of a group of post-industrial countries U.S. consists of information-age knowledge workers

Bell, D. (1999). The coming of post industrial society. New York, NY: Basic Books.

0%

20%

40%

60%

80%

100%

1860 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990

Perc

ent o

f Eco

nom

y

Information

Service

Industry

Agriculture

18

System Complexity is Growing 21st century systems are becoming more complex Number of physical parts are becoming smaller Nano-circuitry and software hide complexity

Moody, J. A., et al. (1997). Metrics and case studies for evaluating engineering designs. Upper Saddle River, NJ: Prentice-Hall.

19

Software Century Number of software-intensive systems is growing Yearly software industry revenue exceeds $3 trillion Poor software quality costing trillions in lost revenues

Dvorak, D. L. (2009). NASA study on flight software complexity. Pasadena, CA: Jet Propulsion Laboratory (JPL).

Software in Military Aircraft

8% 10%20%

35%45%

65%

80%

0%

20%

40%

60%

80%

100%

1960F-4

1964A-7

1970F-111

1975F-15

1982F-16

1990B-2

2000F-22

Perc

ent o

f Fun

ctio

nalit

y

20

Exponential Rate of Change Technology evolving at an ever increasing rate Nano-scale computers will become the norm soon Technological breakthroughs may climax in 25 years

Kurzweil, R. (2005). The singularity is near: When humans transcend biology. New York, NY: Penguin Group.

21

Crossing the Chasm New technology spreads very slowly There are a few innovators and early adopters Years and decades for most to adopt new technology

Moore, G. A. (1991). Crossing the chasm: Marketing and selling technology to mainstream customers. New York, NY: Harper Business.

22

Coping With Big Changes Humans can’t cope with large technological change Changes may be resisted for a long time (years) Big projects plunge organizations into chaos

Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).

23

Global Market Competition Globalization has intensified market competition Domestic competition is no longer the major threat The trade deficit with the Far East is growing bigger

24

Cyber Threats are Growing Cyber threats have increased 10-fold in last decade 70% of cyber incidents perpetrated by U.S. citizens Cyber threats coming from Far East less than 3%

0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 20030

500

1000

1500

2000

2500

3000

3500

4000

4500

Incidents

Vulnerablities

25

Complex Systems are Unstable Large systems experience big downstream changes Project plans designed to cope with small changes Systems engineering not well-suited for changes

Jones, C. (1995). Patterns of software system failure and success. Boston, MA: International Thompson Computer Press.

Basic Assembly 575J CL 400Macro Assembly 400C 225Cobol 74 (Cobol I) 220FORTRAN 210Cobol 85 (Cobol II) 175Pascal 160PL/1 126RPG I 120RPG II/I I I 110Natural 100C++ 80J ava 80dBase III 60Focus 60Clipper 60Oracle 60Sybase 60dBase IV 55Perl 50J avaScript 50VBScript 50Shell Script 50SAS 50APL 50

26

High Project Failure Rates Failed and challenged projects hover around 70% High failure rate due to inability to cope with change Big projects exacerbate challenge and failure potential

Johnson, J., et al. (2009). Chaos summary 2009. Boston, MA: Standish Group International.

27

AgendaIntroductionSystems EngineeringSystems Engineering Challenges

Lean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

28

What is Lean? Lean (lēn): Thin, slim, slender, narrow,

adequate, or just-enough; Without waste A customer-driven systems engineering process that

delivers the maximum amount of business value An economical systems engineering way of planning

and managing the development of complex systems A systems engineering process that is free of excess

waste, capacity, and non-value adding activities Just-enough, just-in-time, and right-sized systems

engineering processes, documentation, and tools A systems engineering approach that is adaptable to

change in customer needs and market conditions

29

Lean Thinking Term coined by John Krafcik of MIT in 1988 Taiichi Ohno of Toyota is credited with its ideas Toyota Production System was adapted from Ford

Womack, J. P., & Jones, D. T. (1996). Lean thinking: Banish waste and create wealth in your corporation. New York, NY: Free Press.Liker, J. K. (2004). The toyota way: 14 management principles from the world’s greatest manufacturer. New York, NY: McGraw Hill.Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley.

Respect for People· Don’t trouble your customer

· Develop people, then build products

· No wasteful work

· Teams and individuals evolve their own practices and improvements

· Build partners with stable relationships, trust, and coaching lean thinking

· Develop teams

Development Practices

· Long-term great engineers· Mentoring manager-engineer-teacher· Cadence· Cross-functional· Team room + visual management· Entrepreneurial chief / product manager· Set-based concurrent development· Create more knowledge

14 Lean Principles

Long-term philosophy, flow, pull, level workload, stop and fix, master norms, visual controls, tested technology, leaders-teachers from within, develop exceptional people, help partners be lean, go see, consensus and action, learning-reflection-kaizen

Continuous Improvement· Go see

· Kaizen

· Spread knowledge

· Small, relentless

· Retrospectives

· 5 whys

· Eyes for waste, variability, overburden, NVA (handoff, WIP, info scatter delay, multitasking, defects, wishful thinking…)

· Perfection challenge

· Work to flow (smaller batch sizes, low cycle time)

Sustainable shortest lead time. Best quality and value (to people and society).Most customer delight, lowest cost, high morale, and safety.

The Goal: Value

Foundation: Management SupportManagement applies and teaches lean thinking, and bases decisions on this long-term philosophy

30

Lean Six Sigma Created in late 1990s by Allied Signal and Maytag Combination of Six Sigma and Lean Thinking Focuses on eliminating waste vs. variation

George, M. L. (2002). Lean six sigma: Combining six sigma quality with lean speed. New York, NY: McGraw-Hill.

31

Lean Development Lean product development emerged in the 1980s Adaptation of Toyota Production System (TPS) “Toyota [New] Product Development System”

Clark, K. B., & Fujimoto, T. (1991). Product development performance: Strategy, organization, and management in the world auto industry. Boston, MA: Harvard Business School Press.

Lean Development

Frequent set-up changes

Lean Manufacturing

Short manufacturing throughput time

Reduced work-in-process inventory between manufacturing steps

Frequent transfer of small batches of parts between manufacturing steps

Reduced inventory requires slack resources and more information flow between steps

Adaptability to changes in volume, product mix, and product design

Broad task assignments for production workers gives higher productivity

Focus on quick problem solving and continuous process improvement

Simultaneous improvement in quality, delivery time, and manufacturing productivity

Frequent product changes

Short development time

Reduced information inventory between development steps

Frequent transfer of preliminary information between development steps

Reduced development time requires slack resources and information flow between stages

Adaptability to changes in product design, schedule, and cost targets

Broad task assignments for engineers (developers) gives higher productivity

Focus on frequent incremental innovation and continuous product and process improvement

Simultaneous improvement in quality, development time, and development productivity

32

Lean Systems Engineering Origin in MIT Lean Aerospace Initiative in 1992 Lean Systems Engineering WG formed in 2006 Lean Enablers for Systems Engineering in 2009

INCOSE. (2009). Lean enablers for systems engineering. Retrieved October 20, 2009, from http://www.incose.org/practice/techactivities/wg/leansewg

� Value

� Map the Value

Stream

� Flow

� Pull

� Perfection

� Respect for People

· Customer involvement

· Streamline development· Deconflict suppliers· Establish metrics

· Communicate effectively· Achieve smooth flow· Ensure program visibility· Use lead

· Pull tasks as-needed

· Appoint a chief· Eliminate waste· Perform continuous improvement

· Create a learning environment· Treat people as valuable assets

· Requirements engineering· Business value analysis

· Program planning· Map value streams· Frontload program

· Program monitoring· Clarify requirements· Frontload architecture· Coordinate activities

· Tailor program

· Continuous improvement· Strive for excellence· Perform lessons learned· Develop communication plan

· Human resources· Respect all people· Strive for technical excellence

33

Lean+ 10X Created by Charles Toups of Boeing in 2008 In-use by P-8A Poseidon and AEW&C System Adaptation of lean thinking for non-manufacturing

Brabant, E. M. (2009). Simple as. Retrieved October 20, 2009, from http://www.boeing.com/news/frontiers/i_ids01.pdf

34

Lean Engineering Benefits MIT has studied dozens of systems for last 15 years They applied criteria to determine if they were lean Numerous programs, past, present, and future

Murman, E., et al. (2002). Lean enterprise value: Insights from MIT's lean aerospace initiative. New York, NY: Palgrave.

35


Systems Engineering ChallengesLean Systems Engineering


36

What is Agility? A-gil-i-ty (ə-'ji-lə-tē) Quickness, lightness, and

ease of movement; To be very nimble The ability to create and respond to change in order

to profit in a turbulent global business environment The ability to quickly reprioritize use of resources

when requirements, technology, and knowledge shift A very fast response to sudden market changes and

emerging threats by intensive customer interaction Use of evolutionary, incremental, and iterative

delivery to converge on an optimal customer solution Maximizing the business value with right-sized, just-

enough, and just-in-time processes and documentation

37

What are Agile Methods? ‘Adaptable’ software development methodologies ‘Human-centric’ method for creating business value ‘Alternative’ to large document-based methodologies

Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org

alsoknown as

CustomerCollaboration

Individuals &Interactions

WorkingSoftware

Respondingto Change

CustomerInteraction

High-Performance Teams

IterativeDevelopment

Adaptabilityor Flexibility

ContractNegotiation

Processes& Tools

ComprehensiveDocumentation

Followinga Plan

Agile Methods‘Values’

alsoknown as

alsoknown as

alsoknown as

valuedmore than

valuedmore than

valuedmore than

valuedmore than

Agile Methods‘Principles’

Traditional Methods‘Values’

38

Crystal Methods Created by Alistair Cockburn in 1991 Has 14 practices, 10 roles, and 25 products Scalable family of techniques for critical systems

Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley.

39

Scrum Created by Jeff Sutherland at Easel in 1993 Has 5 practices, 3 roles, 5 products, rules, etc. Uses EVM to burn down backlog in 30-day iterations

Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.

40

Dynamic Systems Develop. Created by group of British firms in 1993 15 practices, 12 roles, and 23 work products Non-proprietary RAD approach from early 1990s

Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley.

41

Feature Driven Development Created by Jeff De Luca at Nebulon in 1997 Has 8 practices, 14 roles, and 16 work products Uses object-oriented design and code inspections

Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall.

Develop anOverall Model

Build aFeatures List

Plan byFeature

Design byFeature

Build byFeature

Iteration

42

Extreme Programming Created by Kent Beck at Chrysler in 1998 Has 28 practices, 7 roles, and 7 work products Popularized pair programming and test-driven dev.

Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley.

UserStories

ArchitecturalSpike

ReleasePlanning

IterationAcceptance

TestsSmall

Releases

Spike

TestScenarios

SystemMetaphor

CustomerApproval

LatestVersion

ReleasePlan

NextIteration

BugsNew

StoriesRequirements

UncertainEstimates

ConfidentEstimates

XPExtreme Programming

43

Side-Effects of Agile Methods Enable us to cross-the-chasm sooner or earlier Reduce chaos associated with large-scale change Reduce or divide the risk of change into small pieces

Sidky, A. (2008). Becoming agile in an imperfect world. Washington, DC: Agile Project Leadership Network (APLN).

44

Essence of Agile Methods High degree of customer & developer interaction Highly-skilled teams producing frequent iterations Right-sized, just-enough, and just-in-time process

Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley.

Adaptability or Flexibility

High-Performance Teams

Iterative Development

Customer Interaction

45


Systems Engineering ChallengesLean Systems EngineeringAgile Systems Engineering

Agile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

46

What is a Practice? Prac-tice (prăk-'tĭs): Action, tool, technique, or

work instruction; Step-by-step procedure A set of one or more systems engineering techniques

to accomplish a specific action or desired outcome Standard or semi-formal best practices or rules-of-

thumb that are proven to be effective or efficient A suite of manual or automated tools or instruments

that are useful for system design and development An array of optional elements that may be employed on

an as-needed basis, i.e., right tool at the right time Value-adding action that may significantly enhance

productivity, quality, or other key performance metric

47

Release Planning Created by Kent Beck at Chrysler in 1998 Project plan with a 30-60-90-day timing horizon Disciplined and adaptable project management F/W

Beck, K., & Fowler, M. (2004). Planning extreme programming. Upper Saddle River, NJ: Addison-Wesley.

Release Plan Iteration P lan

UserWrite

StoriesUser

Estimate

StoriesUser

Split

StoriesUser

Order

StoriesUser

Analyze

StoriesDev.

Create

TasksDev.

Accept

TasksDev.

Estimate

Tasks

48

Onsite Customers Term coined by Kent Beck in 1999 Customer who sits with developers full-time Fast and efficient way to capture customer needs

Tabaka, J. (2006). Collaboration explained: Facilitation skills for software project leaders. Upper Saddle River, NJ: Addison Wesley.

49

User Stories Term coined by Kent Beck in 1999 Functions or features of value to customers Highly adaptable requirements engineering process

Cohn, M. (2004). User stories applied: For agile software development. Boston, MA: Addison-Wesley.

50

Pair Programming Term coined by Jim Coplien in 1995 Consists of two side-by-side programmers Highly-effective group problem-solving technique

Williams, L., & Kessler, R. (2002). Pair programming illuminated. Boston, MA: Pearson Education.

MOVE PEOPLE AROUND

REFACTOR MERCILESSLY

CONTINUOUS INTEGRATION

CREATE AUNIT TEST

We NeedHelp

ChangePair

SimpleCode

ComplexCode

New UnitTests

NewFunction-

ality

FailedUnitTest

PassedUnitTest

51

Refactoring Term coined by William Opdyke in 1990 Process of frequently rewriting source code Improves readability, maintainability, and quality

Fowler, M. (1999). Refactoring: Improving the design of existing code. Boston, MA. Addison-Wesley.

52

Test-Driven Development Term coined by Kent Beck in 2003 Consists of writing all tests before coding Ensures all source code is verified and validated

Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.

53

Continuous Integration Term coined by Martin Fowler in 1998 Process of automated build/regression testing Evaluates impact of changes against entire system

Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.

BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Compile Source Code

Run Unit Tests

Run Coverage Tests

Static Code Analysis

Build Database

Generate Help Files

Archive and Deploy

54

Agile Documentation Myth that voluminous documentation is needed Myth that agile methods do not use documentation Right-sized, just-in-time, and just enough documents

Rueping, A. (2003). Agile documentation: A pattern guide to producing lightweight documents for software projects. West Sussex, England: John Wiley & Sons.

Contracts

Document Type

Project P lans

Requirements

Architecture

Design

Coding

Tests

User guides

Quality Assurance

Agile Documentation

Performance-based, time-and-materials, level-of-effort

Release plans, iteration plans, story boards, agile repositories

User stories, wire frames, use cases, paper prototypes

Metaphors, spikes, system modeling language, DoDAF

Wire frames, design patterns, unified modeling language

Code patterns, program design language, coding comments

Unit, component, integration, system, and acceptance tests

XML documents, online help, Wikis, FAQs, video and audio clips

Performance, reliability, code structure analysis, and test reports

55

Which Practices Are In-Use Surveys of agile practices are conducted annually Release planning is the most often used practice Continuous integration is also a major practice

Version One. (2008). The state of agile development: Third annual survey. Alpharetta, GA: Author.

86%

75%

72%

65%

62%

60%

59%

59%

52%

49%

44%

35%

34%

31%

31%

0% 20% 40% 60% 80% 100%

Iteration planning

Daily s tandups

Release planning

Continuous integration

Automated builds

Burndown

Retrospectives

Refactoring

Velocity

Test-driven development

Open work area

Digital taskboard

On-s ite customer

Pair programming

Information radiators

56


Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering Practices

Agile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering ValueSummary

57

What is Scalability? Scal-a-ble (skāl-'ə-bəl): To expand, grow,

stretch, raise, or intensify; Increase in size A systems engineering process that applies to projects of

varying size, scope, magnitude, and complexity A product development process that is tailorable to the

type, kind, and class of product under development A process that works well on range of products, from

small to large programs involving systems of systems An approach that enables control of time, cost, scope,

quality, and performance regardless of program type Systems engineering processes designed to maximize

business value under a wide variety of constraints

58

Multi-Level Teams Enables projects to plan for the future and present Decomposes capabilities into implementable pieces Unclogs the drainpipes to let the execution flow freely

Highsmith, J. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education.

59

Multi-Level Backlog Enables multiple levels of abstraction to co-exist Allows customers and developers to communicate Makes optimum use of everyone’s time and resources


n Feature Set· Related user stories that are grouped together

· Also called a Theme, i.e., implemented as entity

· Comprises 6 to 30 days worth of work

n Capability· High-level business or product function

· Also called an Epic, i.e., multiple feature sets

· Comprises 18-90 days worth of work

n User Story· Simple requirement written by customer or user

· A small unit of functionality having business value

· Comprises 2 to 10 days worth of work

60

Multi-Level Planning Enables multiple level enterprise plans to co-exist Allows stakeholders to build viewpoint-specific plans Ensures capabilities are delivered at regular intervals

n Release Plan· Feature set focused, subsystem architecture

· Strategy, objectives, and backlog

· 6 to 12 weeks

n Product Roadmap· Capability focused, enterprise architecture needs

· Vision, objectives, backlog

· 18 to 36 weeks

n Iteration Plan· User story focused, component-level architecture

· Implementation plan, objectives, and backlog

· 2 to 4 weeks


61

Multi-Level Coordination Enables lean and agile methods to scale-up Allows enterprises to create large-scale programs Unleashes optimum productivity and overall control

Schwaber, K. (2004). Agile project management with scrum. Redmond, WA: Microsoft Press.

62

Multi-Level Governance Enables enterprises to achieve functional needs Allows programs to coordinate functional activities Ensures optimal technical performance is achieved

Ambler, S. W. (2009). Scaling agile software development through lean governance. Proceedings of the 2009 ICSE Workshop on Software Development Governance, Vancouver, Canada, 1-2.

Q

I

R

W

T

A

S

C

D

R

R

R

R

R

R

R

R

R

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

T

T

T

T

T

T

T

T

T

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

S

S

S

S

S

S

S

S

S

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

Q

I

R

W

T

A

S

C

D

R T S

63

Multi-Level PMO Enables enterprises to optimize project performance Allows enterprises to control and monitor programs Ensures projects are operating at peak capability

Augustine, S., & Cuellar, R. (2006). The lean-agile PMO: Using lean thinking to accelerate agile project delivery. Agile Project Management Executive Report, 7(10), 1-28.

64

Multi-Level Automation Enables enterprises to be flexible but disciplined Allows enterprises to distribute project work teams Ensures distributed project teams are collaborating

Project Management

Requirements

· DOORS· Requisite Pro· SLATE

Design

· Rhapsody· Telelogic System Architect· Rational System Architect

Coding

· Eclipse· Visual Studio· Sun Studio

Testing

· JUnit· NUnit· Xunit· CPPUnit

· Gtest· Fit· Fitnesse· Selenium

Quality Assurance

· CheckStyle· PMD· EMMA· Jdepend· Cobertura· Gcov

Configuration Mgt

· Subversion (SVN)· Concurrent Versions Sys.· ClearCase

Build Automation

· Ant· NAnt· Maven· Make

Continuous Integ.

· Cruise Control· Hudson· BuildBot

Collaboration

· WebEx· Skype· MeetMe· Wimba

Wiki

· MediaWiki· TracWiki· PhpWiki

Documentation

· NDoc· Javadoc· Doxygen· iText

· Version One· Rally· Scrum Works· VSTS

· Agile Team· Agile Enterprise· Scope Manager· Story Studio

· XP Plan It· Iterate· XP Tracker· Agilo

· XP CGI· XP Web· Xplanner· Ice Scrum

· Project Cards· Target Process· Xtreme Planner· Team System

· Community· Enterprise· Mingle· Hansoft

65


Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering Scaling

Agile Systems Engineering TestingAgile Systems Engineering ValueSummary

66

What is Integration? In-te-gra-tion (ĭn-'tĭ-grā-shən): To add, group,

mix, or assemble; Act of combining A critical verification and validation step in the

systems engineering process for a complex new system A process of testing and evaluating units,

components, subsystems, systems, and systems of systems A key best practice that enables suppliers to deliver

operational systems to customers early and often A automated systems development process that lowers

the risks of developing large-scale complex systems A lean and efficient process that maximizes business

value by eliminating waste from traditional testing

67

What is Agile Testing? Traditional testing is a late, manual process Agile testing is an early and automated process The goal of agile testing is to deliver early and often

Traditional Testing

· Combining source files

· Combining software and environment

· Combining software and data

· Combining software and tests

· Combining developers

Agile Testing

· Code is frequently checked in

· Code is automatically retrieved

· Compilation is done automatically

· Tests are done automatically

· Code reports are generated

· Developers get instant feedback

· Code is automatically deployed or packaged for delivery

Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.

68

Agile Testing Process Developers check-in changes as they occur Server detects all changes and initiates testing Server compiles, tests, analyzes, builds, and deploys


BuildIntegration

Server

VersionControlServer

BuildScripts

UsesWatches

BuildStatus

ProvidesDeveloper A

Developer B

Developer C

CommitsChanges

CommitsChanges

CommitsChanges

Compile Source Code

Run Unit Tests

Run Coverage Tests

Static Code Analysis

Build Database

Generate Help Files

Archive and Deploy

69

Agile Testing Technologies There are literally hundreds of agile testing tools There are tools for building, testing, and deploying Integration tools monitor repositories and initiate tests

Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.

70

Agile Testing Statistics Fewer builds leave in higher bug counts A high number of builds eliminates the defects Goal is to have as many, early builds as possible

Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.

71

Scaling Agile Testing Agile testing slows down with very large systems Slow testing slows integration and increases bugs Agile testing can speed back up with proper attention

Kokko, H. (2009). Increase productivity with large scale CI: Reduce feedback cycle from weeks to 100 minutes. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.

Wide Impact Tuning

· Fast builds – less changes – more green

· Parallelization of test runs

· ClearCase to subversion

· Pre-installing as much as possible

· Removal of randomness

· Compilation in memory

· Installation starting parallel with system build

Focused Impact Tuning

· More memory and CPUs

· Parallelize builds

· Replace 3rd party test libraries

· Reduce/remove timeouts in tests

· Select different tests

· Refactor code & components

· Tune the network & software

· Tune the database

72

Agile Testing Costs Most agile testing tools are “free” open source A build server is no more than a commodity PC Low overhead for new and subsequent setup time

Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.

þ

þ

þ

þ

þ

þ

þ

Free, Open Source Software

$500 for a dedicated build machine

4 hours configuration time for new user

2 hours for an experienced user

20 minutes to set up a new project

It becomes more valuable with use

Less than half the cost of traditional testing

73

Agile Testing Benefits Reduces the cost-of-change by 10 times Frequent builds dramatically lower defect levels Enables early “what-if” tests as well as late changes

Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.

þ 36% reduction in defect ratewhen integration/regression testing at each code check-in

þ 90% reduction in bugs reaching QAMajor municipal gas utility

þ

þ

þ

þ

þ

95% cut in cost of bugsLarge retail web site

90% cut in defect remediation costGlobal supplier of healthcare equipment

Faster time-to-marketMore features and higher quality

Agility in the marketplaceAdded new functionality 2 weeks before ship

Confidence in the process“Oozing Confidence”

74

Agile Testing is NextGen Manual testing is CMMI Capability Level 0 or 1 Agile testing is a CMMI Capability Level 5 practice It is planned, defined, measured, and it’s optimizing

Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Proceedings of the Sixth Japan Symposium on Software Testing (JASST 2008), Tokyo, Japan.

75

Agile Testing Side-Effects Eliminates big-bang integration in the 11th hour Creates a repeatable and reliable testing process Evaluates system-wide changes throughout project


76


Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering Testing

Agile Systems Engineering ValueSummary

77

What is Business Value? Val-ue (văl-'yōō): An amount, quantity, rate,

magnitude, or desirability; Economic worth An economic estimation of the tangible worth of the

organizational assets such as buildings and equipment An appraisal of intangible assets such as knowledge,

experience, skills, patents, processes, and methods A technique for evaluating the costs and benefits of

investments in a business, operations, or personnel The economic impact of deploying a new product

development approach such as systems engineering The total life cycle costs of institutionalizing lean

and agile systems engineering techniques in an enterprise

78

Agile (138 pt.) and traditional methods (99 pt.) Agile methods fare better in all benefits categories Agile methods 359% better than traditional methods

Agile Methods Traditional MethodsLow Median HighCategory

ROI 240% 2,633% 8,852%

Satisfaction 70% 70% 70%

Quality 10% 70% 1,000%

Productivity 14% 122% 712%

Schedule 11% 71% 700%

Cost 10% 26% 70%

Low Median HighCategory

ROI 200% 470% 2,770%

Satisfaction -4% 14% 55%

Quality 7% 50% 132%

Productivity 9% 62% 255%

Schedule 2% 37% 90%

Cost 3% 20% 87%

Rico, D. F. (2008). What is the ROI of agile vs. traditional methods? TickIT International, 10(4), 9-18.

Studies of Agile Methods

79

Productivity of Agile Methods PP productivity 32X more than trad. methods Scrum productivity 5X more than trad. methods Agile methods productivity 20X more than traditional

Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing.

33.4044

29.28

21.2374

16.1575

5.4436

1.0619

0

8

16

24

32

40

PP TDD Agile XP Scrum CMMI®

Software Method

Pro

du

cti

vit

y (

LO

C/H

ou

r)

80

Quality of Agile Methods XP quality 13X better than trad. methods Scrum quality 3X better than trad. methods Agile methods quality 5X better than traditional

0.7466

1.79722.155 2.355

3.945

9.667

0

2.2

4.4

6.6

8.8

11

XP Agile TDD PP Scrum CMMI®

Software Method

Qu

ality

(D

efe

cts

/KL

OC

)


81

Costs of Agile Methods XP costs 8X less than traditional methods Scrum costs 2X less than traditional methods Agile methods cost 5X less than traditional methods

$136,551

$226,807 $249,653 $265,436

$578,202

$1,108,233

$0

$325,000

$650,000

$975,000

$1,300,000


Software Method

Co

sts


82

Benefits of Agile Methods XP benefits 1.5X more than traditional methods Scrum benefits 1.3X more than traditional methods Agile methods benefits 1.4X more than trad. methods

$4,372,282$4,282,026 $4,259,180 $4,243,397

$3,930,631

$3,023,064

$2,750,000

$3,237,500

$3,725,000

$4,212,500

$4,700,000


Software Method

Be

ne

fits


83

ROI of Agile Methods XP ROI 18X more than traditional methods Scrum ROI 3.4X more than traditional methods Agile methods ROI 10X more than trad. methods

3,102%

1,788%1,606%

1,499%

580%

173%

0%

925%

1,850%

2,775%

3,700%


Software Method

Re

turn

on

In

ve

stm

en

t


84

NPV of Agile Methods XP NPV 2.4X more than traditional methods Scrum NPV 1.9X more than traditional methods Agile methods NPV 2.3X more than trad. methods

$3,649,388$3,480,979 $3,438,351 $3,408,902

$2,825,313

$1,509,424

$1,000,000

$1,787,500

$2,575,000

$3,362,500

$4,150,000


Software Method

Ne

t P

rese

nt

Va

lue


85

Real Options of Agile Methods XP ROA 1.6X more than traditional methods Scrum ROA 1.4X more than traditional methods Agile methods ROA 1.6X more than trad. methods

$4,265,936$4,105,884 $4,066,678 $4,040,377

$3,608,772

$2,633,052

$2,200,000

$2,825,000

$3,450,000

$4,075,000

$4,700,000


Software Method

Re

al O

pti

on

s


86


Systems Engineering ChallengesLean Systems EngineeringAgile Systems EngineeringAgile Systems Engineering PracticesAgile Systems Engineering ScalingAgile Systems Engineering TestingAgile Systems Engineering Value

Summary

87

Summary Agility is the evolution of management thought Confluence of traditional and non-traditional ideas Improve performance by over an order-of-magnitude

Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing.

þ

þ

þ

þ

þ

þ

þ

Systems engineering approaches

Agile methods and practices are ...

New product development approaches

Expertly designed to be fast and efficient

Intentionally lean and free of waste (muda)

Systematic highly-disciplined approaches

Capable of producing high quality systems

Right-sized, just-enough, and just-in-time tools

þ Intended to maximize business value for customers

88

New Book on Agile Methods Guide to Agile Methods for business leaders Communicates business value of Agile Methods Rosetta stone to Agile Methods for traditional folks

http://davidfrico.com/agile-book.htm (Description) http://www.amazon.com/dp/1604270314 (Amazon)

Table of Contents 1. Introduction to Agile Methods 2. Values of Agile Methods 3. History of Agile Methods 4. Antecedents of Agile Methods 5. Types of Agile Methods 6. Practices of Agile Methods 7. Agile Project Management 8. Agile Software Engineering 9. Agile Support Processes10. Agile Tools and Technologies11. Comparison of Agile Methods12. Agile Metrics and Models13. Surveys of Agile Methods14. Costs-Benefits of Agile Methods15. ROI Metrics of Agile Methods16. Measures of Agile Methods17. Costs of Agile Methods18. Benefits of Agile Methods19. ROI of Agile Methods20. NPV of Agile Methods21. Real Options of Agile Methods22. Business Value of Agile Methods23. Agile vs. Traditional Methods24. Future of Agile Methods

lean & agile systems engineering for systems of systems dr. david f. rico, pmp, csm website:...

Documents