seari overview and dynamic multi-attribute tradespace...

SEAri Overviewand Dynamic Multi-Attribute Tradespace

Exploration

Dr. Donna Rhodes

Dr. Adam Ross

MIT-Portugal Program

Transportation Systems Workshop

Friday, September 5, 2008

seari.mit.edu © 2008 Massachusetts Institute of Technology 2

Systems Engineering Advancement Research Initiative (SEAri)

Current Sponsors: US Air Force Office of Scientific Research, Singapore DSO, US Air Force SG/HSI, MIT Portugal Program, Lean Advancement Initiative, US Government Agency

3 Cambridge Center

NE20 – 388/343/352

Mission

Advance the theories, methods, and effective practice

of systems engineering applied to complex socio-

technical systems through collaborative research


Evolving Systems Engineering

SYSTEMS ENGINEERING (Traditional) Systems engineering is the process of selecting and synthesizing

the application of the appropriate scientific and technical knowledge in order to translate system requirements into system design. (Chase)

SYSTEMS ENGINEERING (Advanced)Systems engineering is a branch of engineering that

concentrates on design and application of the whole as distinct from the parts… looking at the problem in its entirety, taking into account all the facets and variables and relating the social to the technical aspects. (Ramo)

SEAri sits here


Advancing SE

Multiple heterogeneous stakeholder groups with divergent cost goals and measures of success

Single or homogenous stakeholder group with stable cost/funding profile and similar measures of success

Cost

Intense concept phase analysis followed by continuous anticipation, aided by ongoing experimentation

Concept phase activity to determine system needs

Anticipation of Needs

SoS component systems separately acquired, and continue to be managed and operated as independent systems

Centralized acquisition and management of the system

Acquisition and Management

Enhanced emphasis on “ilities” such as Flexibility, Adaptability, Composeability

Reliability, Maintainability, Availability are typical ilities

System “ilities”

Component systems can operate independently of SoS in a useful manner Protocols and standards essential to enable interoperable systems

Defines and implements specific interface requirements to integrate components in system

System Interoperability

Dynamic adaptation of architecture as needs change

System architecture established early in lifecycle; remains relatively stable

System Architecture

Evolving new system of systems capability by leveraging synergies of legacy systems

Development of single system to meet stakeholder requirements and defined performance

Purpose

Advanced

Systems Engineering

Traditional

Systems Engineering


Underlying Research Structure

Prescriptive methods seek to advance state of the practice based on

sound principles and theories, as grounded in real limitations and

constraints

• Normative research: identify principles and theories -- “should be”

• Descriptive research: observe practice and identify limits/constraints


Structured with four interacting “clusters” that undertake research in a portfolio of five topics:

1. Socio-Technical Decision Making

2. Designing for Value Robustness

3. Systems Engineering Economics

4. Systems Engineering in the Enterprise

5. Systems Engineering Strategic Guidance

SEAri Structure and Portfolio


Sponsor Engagement Models

1. Classical “basic research” sponsors

– Targeted topic toward broad scientific goals

2. Innovation grant sponsors

– Higher risk/higher payoff research

3. Contract research sponsors

– Toward solving sponsor problem

4. Consortium sponsors

– Pooled funds for shared research benefits

5. “Deep engagement” partnerships

– Symbiotic relationship


Research Portfolio (1)

SOCIO-TECHNICAL DECISION MAKING

This research area seeks to develop multi-disciplinary representations, analysis

methods, and techniques for improving decision making for socio-

technical systems. Examples include:

– Studies of decision processes and effectiveness of techniques – Constructs for representing socio-technical systems to perform impact

analysis– Decision strategies for system of systems– Visualization of complex trade spaces and saliency of information

– Understanding and mitigating cognitive biases in decision processes

While organizational theorists have well developed theories of how organizations function and make decisions, this understanding needs to be integrated into the design phase in a quantifiable way….then it will be

the case that a priori the effect of the enterprise organization on the engineering system will be predicted rather than being a surprise

Hastings, MIT ESD Symposium, 2004



DESIGNING for VALUE ROBUSTNESS

This research area seeks to develop methods for concept exploration,

architecting and design using a dynamic perspective for the purpose of

realizing systems, products, and services that deliver sustained value to

stakeholders in a changing world. Examples include:

– Methods for dynamic multi-attribute tradespace exploration

– Architecting principles and strategies for designing survivable systems

– Quantification of the changeability of a system design

– Techniques for the consideration of unarticulated and latent stakeholder value

– Taxonomy for enabling stakeholder dialogue on “ilities”

Value robustness is the ability of a system to continue to deliver stakeholder value

in the face of changing contexts and needs. Architecting value robust systems

requires new methods for exploring the concept tradespace, as well as for decision

making. Also needed are architecting principles and strategies, an approach for the

quantification of changeability, and an improved ability for architects and analysts

to classify value for purposes of dialogue and implementation

Ross and Rhodes, 2008



SYSTEMS ENGINEERING ECONOMICS

This research area aims at developing a new paradigm that encompasses an

economics view of systems engineering to achieve measurable and

predictable outcomes while delivering value to stakeholders. Examples

include:

– Measurement of productivity and quantifying SE ROI – Advanced methods for reuse, cost modeling, and risk modeling – Application of real options in systems and enterprises – Leading indicators for systems engineering effectiveness

In a 2004 Air Force/MIT workshop, Dr. Marvin Sambur, (then) Assistant Secretary

of the AF for Acquisition, noted that the average program is 36% overrun

according to recent studies – disrupting the overall portfolio of programs


Research Portfolio (4)SYSTEMS ENGINEERING in the ENTERPRISE

This research area involves empirical studies and case based research for the

purpose of understanding how to achieve more effective systems

engineering practice in context of the nature of the system being

developed, external context, and the characteristics of the associated

enterprise. Examples include:

– Collaborative, distributed systems engineering practices

– Social contexts of enterprise systems engineering

– Development of engineering systems thinking in the workforce

– Alignment of enterprise culture and processes

– Socio-technical systems studies and models as teaching cases

The understanding of the organizational and technical interactions in our

systems, emphatically including the human beings who are a part of them,

is the present-day frontier of both engineering education and practice.

Dr. Michael D. Griffin, Administrator, NASA, 2007 Boeing Lecture, Purdue University



SYSTEMS ENGINEERING STRATEGIC GUIDANCE

This research area involves synthesis of theory with empirical and case based research for the purpose of developing prescriptive strategic guidance to inform the development of policies and procedures for systems engineering in practice. Examples include:

– Systems Engineering research guidelines

– Participation in focus groups and pilot-phase reviews

– Position papers on proposed policies

– Recommendations for integrating SE research into curriculum

– Identification of SE research gaps and opportunities

The full impact of systems engineering research can only be achieved through synthesis of research outcomes


ANCHOR PROGRAM:

MPP Year One Engineering Systems Architecting and Design: A Creative Group Decision-Making Effort

MPP Year Two and ThreeApplication of Dynamic Multi-Attribute Tradespace Exploration to the Architecting and Design of a Transportation Engineering System

• Dr. Donna H. Rhodes, Principal Investigator

• Dr. Adam M. Ross, Lead Research Scientist


MPP Year One, Anchor Program

Engineering Systems Architecting and Design: A Creative Group Decision-Making Effort

Education Materials

• Seven module set course syllabi based on research

• Draft set of module presentations files

• Preliminary set of relevant readings for each module

MIT CONTACTSDr. Donna Rhodes and Dr. Adam Ross MIT Systems Engineering Advancement Research

Initiative [email protected]

For more information on the research, visit http://seari.mit.edu


MPP Year Two and Three, Anchor Program Dynamic Tradespace Exploration Method Applied to

Design of Transportation Systems

Designing today’s complex systems involves sophisticated decision analysis under conditions of high uncertainty.

Too often decision are made for the present situation rather than for a a world that keeps changing.

New MIT methods help designers explore tradespaces of possible designs, and select designs that are most responsive to change.

Previously applied to aerospace systems, this project is studying how this method can be applied to transportation systems

MIT CONTACTSDr. Donna Rhodes and Dr. Adam Ross MIT Systems Engineering Advancement Research

Initiative [email protected]

For more information on the research, visit http://seari.mit.edu

Dynamic Multi-Attribute Tradespace Exploration (MATE)

A Short Introduction


How can we make good decisions?

Value is primarily determined at the beginning of a program

After Fabrycky and Blanchard 1991


Tradespace Exploration Paradigm: Avoiding Point Designs

Cost

Utility

Tradespace exploration enables big picture understanding

Differing types of trades

1. Local point solution trades

2. Multiple points with trades

3. Frontier solution set

Designi = {X1, X2, X3,…,Xj}

4. Full tradespace exploration

5. Dynamic tradespace relations (NEW)

(t)

(t)


Models and simulations determine attribute “performance” of many

designs (1000s to 10000s or more)

Tradespace Exploration Coupled with Value-driven Design

Assessment of cost and utility of large space of possible system designs

ATTRIBUTES: Design decision metrics– Data Lifespan (yrs)– Equatorial Time (hrs/day)– Latency (hrs)– Latitude Diversity (deg)– Sample Altitude (km)

Orbital Parameters– Apogee Altitude (km)– Perigee Altitude (km)– Orbit Inclination (deg)

Spacecraft Parameters– Antenna Gain – Communication Architecture– Propulsion Type– Power Type– Total Delta V

DESIGN VARIABLES: Design trade parameters

Cost, Utility

Many system designs can be compared through tradespace exploration:

1. Elicit “Value” with attributes and utility

2. Generate “Concepts” using design variables and cost model insights

3. Develop models/sims to assess designs in terms of cost and utility


What’s in a Name?

• Multi-Attribute– “Multi”: reflects the multi-criteria nature of the design-value problem space

– “Attribute”: reflects the focus on decision maker-defined metrics for success, i.e., value-focused

• Tradespace Exploration– “Tradespace”: reflects the broad consideration of design alternatives and the structured tensions underlying differences between and among options

– “Exploration”: reflects discovery and communication of patterns in the design space, with no “best”destination, as opposed to optimization


Developing a Static Tradespace

• Determine Key Stakeholders

• Scope and Bound the Mission

• Elicit Attributes–Determine Utilities

• Define Design Vector Elements–Includes Fixing Constants Vector

• Develop model(s) to link Design and Attributes–Includes Cost Modeling

• Generate the Tradespace

• Tradespace Exploration

MissionConcept

Attributes

Calculate Utility

Develop System Model

Estimate Cost

Architecture Tradespace

Define Design Vector

Decision Makers


Example: DRSSMulti-Concept Disaster Responsive Surveillance

satellites

swarms

aircraft

DESIGN VARIABLES (5+7)

• Aircraft DV– Configuration Flag 1-13 (3x jet, 2x small prop, 2x med UAV, small UAV, existing

Cessna, Orion, Predator, ScanEagle, Global Hawk)

– Gross Weight Flag low, medium, high

– Number of Assets 1-6

– Payload Type visible, infrared

– Aperture Size 0.01, 0.02, 0.04, 0.07, 0.08 m

• Spacecraft DV– Deployment Strategy 1-4 (on-orbit, launch-ready, pre-fab parts, classic design)

– Altitude 120-1100 km

– Inclination 0,23,90,sun-synch

– Number of assets 1-5

– Payload Type visible, infrared

– Excess Delta-V 600-1200 m/s

– Ops Lifetime 5-10 yrs

ATTRIBUTES (10): Firefighter/Owner– Acquisition Cost ($M), Price/day ($K/day), Cost/day ($K/day), Responsiveness (hrs),

Time to IOC (days), Max % of AOI (%), Time to Max Coverage (min), Time between AOI (min), Imaging Capability (NIIRS level), Data Latency (min)

Number of Designs Explored: 2340+8640Number of Designs Explored: 2340+8640

Data Latency

Decide to “build”

Time to IOC

Request “service”

Responsiveness

AOI_1 in “view”

AOI_1 “target”

Time between AOI

AOI_2 “target”

Max AOI Covered

Time to Max Coverage

End of “service”

“Service” durationCost/dayAcquisition Cost

Time of first “need” AOI(s) Type of image(s)Mission Variables (disaster-specific)

Imaging Capability

Price/day

Data Latency

Decide to “build”

Time to IOC

Request “service”

Responsiveness

AOI_1 in “view”

AOI_1 “target”

Time between AOI

AOI_2 “target”

Max AOI Covered

Time to Max Coverage

End of “service”

“Service” durationCost/dayAcquisition Cost

Time of first “need” AOI(s) Type of image(s)Mission Variables (disaster-specific)

Imaging Capability

Price/day

Witch Creek FireWitch Creek Fire

October 2007October 2007

Image * from http://www.boeing.com/companyoffices/gallery/images/scaneagle/dvd-1390-1.html

*


Characteristics of Tradespace Exploration

• Model-based high-level assessment of system capability

• Ideally, many alternatives assessed

• Can compare heterogeneous, new and old concepts on common basis

• Avoids optimized point solutions that will not support evolution in

environment or user needs

• Provides a basis to explore technical and policy uncertainties

• Provides a way to assess the value of potential capabilities

• Can serve as a boundary object for communicating desires and

technical feasibility

A process for understanding complex solutions to complex problems

Enhances knowledge for “upfront” decision-making and planning


0.00

500.00

1000.00

1500.00

2000.00

2500.00

3000.00

3500.00

4000.00

0.00 0.20 0.40 0.60 0.80 1.00

Utility (dimensionless)

Low Biprop

Medium Biprop

High Biprop

Extreme Biprop

Low Cryo

Medium Cryo

High Cryo

Extreme Cryo

Low Electric

Medium Electric

High Electric

Extreme Electric

Low Nuclear

Medium Nuclear

High Nuclear

Extreme Nuclear

Typical Benefits:Understanding Limiting Physical or Mission constraints

Hits “wall” of either physics (can’t change!) or utility (can)

SPACETUG• General

purpose orbit

transfer

vehicles

• Different

propulsion

systems and

grappling/

observation

capabilities

• Lines show

increasing fuel

mass fraction

Cost (M

$)


Assessing Changing Requirements

0

500

1000

1500

2000

2500

3000

3500

4000

0.0 0.2 0.4 0.6 0.8 1.0


Biprop

Cryo

Electric

Nuclear

0

500

1000

1500

2000

2500

3000

3500

4000

0.0 0.2 0.4 0.6 0.8 1.0


Biprop

Cryo

Electric

Nuclear

Space Tug example: added requirement for rapid response

drastically lowers utility of electric propulsion designs

User needs change, soUtilities recalculated

Cost (M

$)

Cost (M

$)


0

500

1000

1500

2000

2500

3000

3500

4000

0.0 0.2 0.4 0.6 0.8 1.0


Biprop

Cryo

Electric

Nuclear

Comparing Point Designs

Designs from traditional process

Tradespacehelps compare“apples and oranges”concepts

Providesa context for understandingalternatives

Cost (M

$)


Understanding Uncertainties

• Often learn a lot by simple examination

• Better: Explicitly look at sensitivity of models to uncertainties– Clouds are possible locations of a single design

• Uncertainties can be market, policy, or technical

• Mitigate with portfolio, real options methods

0

100

200

300

400

500

0 0.2 0.4 0.6 0.8 1


B Architectures:Changes (in anything)may cause large added cost

A Architectures:Changes (in anything) have less drastic affect; more value may be available for modest added cost

Cost (M

$)


Tradespace Analysis: Selecting “best” designs

Cost

Utility 1

A

B

C

D

E

Cost

Utility 2

A

B

C

D

E

If the “best” design changes over time, how does one select the “best” design?

Time

New “best” designNew “best” designClassic “best” designClassic “best” design


Tradespace Networks: Changing designs over time

Cost

Utility 1

A

B

C

D

E

Cost

Utility 2

A

B

C

D

E

“Best” designs in new contexts may differ; Changeability may provide opportunity

Time

Classic “best” designClassic “best” design New “best” designNew “best” design

Generate tradespace networks

Tradespace designs = nodes

Applied transition rules = arcs

1

2

3

4

Cost

1 2

1

2

3

4

Cost

1 2

∆∆∆∆Cost


Dynamic MATE Summary

A Layered Approach

Perform Static MATE

Attributes Designs Proposed Rules

Define Epochs

Potential Contexts ∆ DV, X, R

Construct Eras

Dynamic StrategiesEpoch Series

TjEpoch j

U

0

TjEpoch j

U

0

Epoch jU

0

C

U

C

U

U

0

Epoch i

TiU

0

Epoch i

TiU

0

U

0

U

0

Method provides insights into strategies for achieving value robustness


Achieving Value Robustness

Utility

Cost

State 1 State 2

U

Cost

DV2≠DV1

DV2=DV1

Utility

0

Epoch 1 Epoch 2

S1,b S1,e S2,b S2,e

T1 T2

Active Passive

Research suggests two strategies for “Value Robustness”

1. Passive• Choose “clever” designs that remain high value

• Quantifiable: Pareto Trace number

2. Active• Choose changeable designs that can deliver high value when needed

• Quantifiable: Filtered Outdegree

Value robust designs can deliver value in spite of inevitable context change

Time

New Context Drivers• External Constraints• Design Technologies• Value Expectations


Increased knowledge (including understanding of uncertainties) allows better decisions

Changing the Picture

Classic decision impacts New paradigm decision impacts


Applying to new domains

• MATE developed through Aerospace domain cases

• MPP project will apply to Transportation case(s) – Anticipated contributions:

• Uncover domain biases (Aerospace v. Transport)

• Opportunity for new insights in transportation planning and analysis

• TBD…

An Application of Dynamic MATE to the Architecting and

Design of a Transportation Engineering System

Student: Julia Nickel, ESD SM expected ’09


Example: Airport Express for Chicago

Motivation:• Kennedy Expressway congested• CTA Blue Line slow, unreliable, inconvenient

Planned: • Rail connection• Operated by private concessionaire• Maintenance out-contracted to CTA

MATE Study:Conducting value elicitation interviews with Stakeholder representatives as first step of applying MATE

Question: Does “Airport Express” make sense, and if so, what design?


Example: Airport Express for Chicago

Rank City of Chicago CTA Private Operator

Attributes Measure, Range (min, max utility)

Attributes Measure, Range

Attributes Measure, Range

1 Estimated tax base change

Increase in equalized assessed property value downtown (4%-10%)

Up front investment

Mn $(100, 0)

Return on investment, pre-tax

%(12, 35)

2 Generation of Employment

# jobs created(20,000 -100,000)

Impact on current operations- overall capacity

% of capacity needed for airport express(25%- 0%)

Competition (Competing CTA services, construction of roads)

Scale 1-5(3, 5)

3 Availability of outside project funding

Local share requirement(50%-0%)

- probability of recurring delays to existing trains

%(5%- 0%)

Autonomy to make changes (e.g. fares)

Parties to consult (3, 1)

Stakeholder attributes- project specific?

Attributes expressed in interviews with representatives raise questions:- Is an airport express the best option to fulfill these wishes?- Is the need for an airport express the true driver of this project?

Begs the question: is this a “solution” looking for a problem? (Classical dilemma)


Preliminary insights: Identified salient “new” domain issues

• Iterative planning process – determined by availability of funding, granting of permits, and other events

• Less emphasis on a “mission” that drives a project– focus on goals (“benefits”) and feasibility (“costs”, both financial and political)

• More classes of stakeholders– Losers (-> compensation), stakeholders without decision making power

• Large numbers of stakeholders (passengers, public)– need for aggregate measures (elasticities, political representation) – indirect feedback (e.g. purchase decisions)

• Multiple cost types to be considered– social, environmental, monetary (including externalities)

• Strong role of inheritance– infrastructure constraints: physical and expectations

• Market structure of players– natural monopolies and regulation

Nickel, J., Ross, A.M., and Rhodes, D.H., “Cross-domain Comparison of Design Factors in System Design and Analysis of Space and Transportation Systems,” 6th Conference on Systems Engineering Research, Los Angeles, CA, April 2008.

It is expected that grappling with these issues using MATE will result in reciprocating insights for case applications across domains


Introduction to Website

seari.mit.edu

PurposeWeb portal for sharing

research within SEAri, MIT, and outside world

Navigation

Home

About

People

Research

Related Courses

Documents

Events

Sponsors

Community

Contact

Internal