© olivier de weck, oct 2008 page 1 strategic engineering olivier l. de weck, ph.d. [email protected]...

© Olivier de Weck, Oct 2008

Strategic EngineeringStrategic Engineering

Olivier L. de Weck, [email protected]

Associate Professor of Aeronautics and Astronautics and Engineering Systems

October 7, 2008

Designing Systems for an Uncertain FutureDesigning Systems for an Uncertain Future

Version 2

Change Propagation Analysis in Complex SystemsChange Propagation Analysis in Complex Systems


SystemArchitecture

Integrated Modelingand Simulation

MultidisciplinaryDesign Optimization

Strategic Engineering – “the big picture”

“optimal” design x* at t=to

technology

regulations markets

concept

performance, cost, risk

Design forChangeability

uncertainty

changes

at t=to+t requirements change and x* is no longer optimal

flexibilityreal options

TemporalDimension

Design forCommonality

more than one variant of the system is needed: x1

*, x2, … xn

variety

standardization

SpatialDimension

commonalityplatforms

http://strategic.mit.edu

© Olivier de Weck, August 2008

F/A-18 Center Barrel Section

Y488Y470.5

Y453Wing

Attachment

74A324001


F/A-18 Complex System Change

F/A-18 System Level Drawing

OriginalChange

FuselageStiffened

Manufacturing Processes Changed

Flight ControlSoftware Changed

Gross Takeoff Weight

Increased

Center of Gravity Shifted


Change Propagation Analysis

in Complex Systems

Giffin M., de Weck O., Bounova G., Keller R., Eckert C., Clarkson J., “Change Propagation Analysis in Complex Technical Systems”, DETC2007-34652, ASME 2007 Design Engineering Technical Conferences, DETC2007-34871, Las Vegas, NV, September 4-7, 2007

In Press: ASME Journal of Mechanical Design

Sponsor: Raytheon Integrated Defense Systems

ProblemAddressed

Understanding change propagation patterns in large technicalprojects involving hardware, software and human operators

ScientificContribution

Developed procedure for data-mining of a large change request database (9 years, 41,500 changes) and analyzing change patterns (“motifs”) as well as classification of system components with a Change Propagation Index (CPI)

Outcome, Impact Applied to a large USAF Radar System project at Raytheon. Identified areas that are likely candidates for flexibility infusion


System Description

Complex Sensor System Complex sensor system,

complex hardware, software, human operators

Derivative of earlier system 9 Year development

46 Areas (“Subsystems”) Hardware Software Program Documentation

System Map (graph) Interconnections between

areas


Data Set

Change Request Database technical, managerial, procedural track parent, child, siblings by

areas with unique ID number chronologically numbered IDs

Data Mining Procedure Export from DBMS to text file Written into MySQL database

with Perl scripts Equivalent to a MS Word

document with 120,000 pages Sorting, Filtering, Anonymizing Write simplified change request

format (see right side)

ID Number 12345

Date Created Date Last Updated

06-MAR-Y5 10-JAN-Y6

Area Affected 19

Change Magnitude 3

Parent ID 8648

Children ID(s) 15678, 16789

Sibling ID(s) 9728

Submitter eng231

Assignees eng008 eng231 eng018

Associated Individuals Admin_001 Engineer_271

Stage Originated, Defect Reason

[blank], [blank]

Severity [blank]

Completed? 1

Typical Change Request


Change Networks

Apply Graph Theory to extract networks of connected changes

parent-child changes sibling changes

Most changes are only loosely connected

2-10 related changes

Some large networks emerged

Question: do these networks emerge from a single initial change?

(rank) (connected changes)

1 2579

2 424

3 170

4 87

5 64


Network plot of largest change network in the dataset, with 2579 associated change requests.

Change Propagation Network


2302423922

23729

23821

23831

23925

23942

23945

23992

24659 25053

24781

24926

24927

25463

24980

25476

25481

25515

8000

12156

13320

22850

22946

26117

27169

27592

27952

281622860128696

29226

29227

2935329731

29744

29826 30126

27627

28878

28166

28567

2765628528 28428

28009

30148

28067

28186

2852928821

28531

27027

27585

28007

28122 28153

28187 28213

28695

2878828790

28846 2939929538

29547

26331

26333

27023

29711

30548

30143

30344

30465

3046630501

30503

30614

30771

31235

31471

31966

31967

31972

31973

32289

32645

Mapping Changes to affected subsystem areas

Change Propagation Network

System Network Map


Change Propagation Index (CPI)

Classify each area Absorber, Carrier, Multiplier

( ) ( )

( )ij ij

ij

c parent c siblingp

Ctot j

1

( ) ( )N

out ji totj

C i p C i

Area 1 2 3 4 5 6

1 0.4843 0.0011 0.0136 0.0057 0.0125 0.0023

2 0.0061 0.0000 0.0000 0.0030 0.0000 0.0000

3 0.0173 0.0000 0.1053 0.0050 0.0012 0.0000

4 0.0224 0.0000 0.0112 0.0449 0.0000 0.0000

5 0.0137 0.0000 0.0000 0.0000 0.1262 0.0000

6 0.0417 0.0000 0.0000 0.0000 0.0000 0.0833

DSM Change Propagation Frequency

receiving area

instigating area

A change in Area 1 caused changes in Area 6 with afrequency of 4.17%.

1

( ) ( )N

in ij totj

C i p C j

( ) ( )( )

( ) ( )

out in

out in

C i C iCPI i

C i C i

change propagation probability

totalcompletedchangesin Area j

-1 <= CPI <= +1


System Area Classification

Areas found to be strong multipliers 16: hardware performance evaluation 25: hardware functional evaluation 5: core data processing logic 32: system evaluation tools 19: common software services 3: graphical user interface (GUI)

Areas found to be perfect reflectors 27, 41: look like perfect absorbers but actually zero changes implemented despite numerous changes proposed = perfect reflectors

CPI Spectrum


Change Requests Written per Month

0

300

600

900

1200

1500

1 5 9 13

17

21

25

29

33

37

41

45

49

53

57

61

65

69 73 77

81

85

89

93

Month

Nu

mb

er

Wri

tte

nChange Request Generation

[Eckert, Clarkson 2004]

Discovered new changepattern: “inverted ripple”

component design

subsystem design

systemintegrationand test

bug fixes

major milestonesor managementchanges


Insights Inverse relationship between change magnitude and frequency of occurrence

Large changes are infrequent, small ones are ubiquitous

Many change requests are never implemented Some are rejected, others are ignored (~ 50%)

Changes may form complex networks over time. Most are small (<10 changes), a few large ones exist (beware of these !) Change networks form through coalescence and not necessarily through multi-

step causal change propagation

Changes can propagate between areas that are not direct neighbors in the system DSM (not shown here, but we found this is so)

Subsystems can be classified as: Multipliers CPI > ~0.3 Carriers -0.1<CPI<1.0 Absorbers CPI<-0.3

Reflectors of Change CRI>CAI Acceptors of Change CAI>CRI

Analysis of change database revealed that Real world change processes more complex than expected Industry data tends to be “noisy” Potential for deriving change impact and likelihood for future projects


Future Work Change Prediction:

How good are our predictions regarding actual versus planned effort? How can change propagation patterns observed on past projects be leveraged for

future design decisions (e.g. modularity, flexibility)

Data Processing: Standardize methods for recording and processing data, tracing large change

networks in greater depth- attempt to reconstruct logic

Staffing and Organization: Analyze effects of staffing on changes and components Patterns based on which personnel/organization work on the changes?

Contractual: Can change propagation be used to write better prime and sub-contracts?

Statistical: Are there critical numbers for change propagation? Limits on the number of

propagation steps? .

CMI-Sponsored Workshop on Engineering ChangeMIT Endicott House, October 30-31, 2008~ 12 firms from various industries (aerospace, auto, printing, construction)

Cambridge-MIT-Institute (CMI)Engineering Change Twin WorkshopsEngineering Change Twin Workshops

Trinity Hall College, UKUniversity of Cambridge

April 7-8, 2008

MIT Endicott House, USAOctober 29-31, 2008

Reasons for Change

Problems discovered during production and operations in the field such as retrofits, recalls ….(melioration)

Customization of product variants for different customers and market segments (globalization)

Infusion of new technologies during product refreshes or major “block” upgrades (innovation)

Cost reduction Initiatives, response to new features introduced by other firms (competition)

New government regulations (e.g. fuel economy standards, no lead in electronics …(compliance)

Others ….?

Workshop Goals

Obtain multi-faceted industry perspective on state-of-the art in engineering change practice

Present academic perspective and recent research advances to industry

Establish a research agenda for the next 5 years Put in place basis for Special Issue of RED* Stimulate interest in follow-up collaboration Establish user community for advanced engineering change methods

and tools

* Research in Engineering Design (RED) Journal

Invited Companies

UK Rolls Royce (A/C

Engines)* Perkins (Diesel)* Volvo (Trucks, Engines)* BAE Systems (Defense)* Bosch (Auto Supplier)* BMW (Cars)* BP (Oil & Gas)* MAN Roland (Printing

Systems) Arup (Construction)

US Xerox (Printing Systems) Ford, GM (Cars and Trucks) Agusta Westland (Helicopters) Boeing (Aircraft) General Mills (Food) Fluor (Construction) Mack (Highway Trucks) Gerber (Textile Machines) NASA (Spacecraft) Raytheon (Defense Systems) Ventana Systems (S/W) Aberdeen Group United Technologies Corp.

*attended April 2008


Strategic Engineering

Strategic Engineering is the process of designing systems and products in a way that deliberately accounts for customization and future uncertainties such that their lifecycle value is maximized.

© olivier de weck, oct 2008 page 1 strategic engineering olivier l. de weck, ph.d. [email protected]...

Documents

change propagation analysis

areas olivier

large change request

change patterns motifs

associated change requests

complex systems olivier

single initial change

system level drawing