welcome [seari.mit.edu]seari.mit.edu/documents/summit/2008/01-searisummit... · view of...

WELCOME to the

SEAri Research Summit

October 21, 2008

http://seari.mit.edu © 2008 Massachusetts Institute of Technology 2

Agenda

9:00 Welcome and Introductions Dr. Donna Rhodes, SEAri Director

9:30 SEAri – Overview of the SEAri Research Program Dr. Donna Rhodes, Research Director

10:00 Research Profile: Socio-Technical Decision Making and Designing for Value Robustness

Dr. Adam Ross, Research Scientist

10:45 Break

11:00 Research Report: Designing Systems for Survivability Matt Richards, Doctoral Research Assistant

11:30 Research Report: Real Options in Enterprise Architecture Tsoline Mikaelian, Doctoral Research Assistant

noon LUNCH

1:00 Research Profile – Systems Engineering in the Enterprise Dr. Donna Rhodes, Principal Research Scientist

1:30 Research Report: Leveraging Organizational Culture, Standard Process, and Team Norms to Enable Collaborative Systems Thinking

Caroline Lamb, Doctoral Research Assistant

2:00 Stretch Break

2:10 Research Profile: Systems Engineering Economics Dr. Ricardo Valerdi, Research Associate

2:50 Research Poster Session with Refreshments SEAri Research Assistants

4:15 Participant Feedback and Recommendations for Research SEAri Leadership

4:55 Closing Remarks Dr. Donna Rhodes

5:00 Adjourn


OVERVIEW of RESEARCH PROGRAM

October 21, 2008

Dr. Donna H. Rhodes

Massachusetts Institute of Technology

[email protected]


Topics

• About SEAri

• Motivations

• Collaboration

• Research Portfolio

• Transfer to Practice


About SEAri

and

MIT Engineering Systems Division


Systems Engineering

Field of Practice

Systems Engineering

Considers both the business and the technical needs of all customers with the goal of providing a quality product that meets the user needs.

Engineering

Management SystemsEngineering


Engineering

Management

Social S

cience

EngineeringSystems

Engineering Systems:

Field of Scholarship

ENGINEERING SYSTEMS

A field of study taking an integrative holistic view of large-scale, complex,

technologically-enabled systems with significant enterprise level

interactions and socio-technical interfaces.


MIT Engineering Systems Division

Academic Unit for SEAri

Engineering systems is a field of study taking an integrative holistic

view of large-scale, complex technologically enabled systems

with significant enterprise level interactions

and socio-technical interfaces

MIT’s Engineering Systems Division (ESD) is a cross-cutting

academic unit -- engineering, management, and social sciences.

It Broadens engineering practice to include context of challenges

as well as consequences of technological advancement

50 faculty 300 masters students 60 PhD students


ENGINEERING

SYSTEMS

PoliticalEconomy

Economics,Statistics

Systems Theory

OrganizationalTheory

Operations Research/Systems Analysis

System ArchitectureSystems EngineeringProduct Development

EngineeringManagement

Technology & Policy

Engineering Systems

as a Field of Scholarship


Engineering Systems Requires

Four Perspectives

1. A very broad interdisciplinary perspective, embracing technology, policy, management science, and social science.

2. An intensified incorporation of system properties (such as sustainability, safety and flexibility) in the design process.

3. Enterprise perspective, acknowledging interconnectedness of product system with enterprise system that develops and sustains it.

4. A complex synthesis of stakeholder perspectives, of which there may be conflicting and competing needs which must be resolved toserve the highest order system (system-of-system) need.


Systems Engineering Advancement

Research Initiative (SEAri)

Major Sponsors: US Air Force Office of

Scientific Research, MIT Portugal Program,

Singapore Defense Sciences Office, US Air

Force HIS Office, Lean Advancement Initiative,

selected US Government Agencies

(many other engagement partners)

3 Cambridge Center

NE20 – 388/343

Mission

Advance the theories, methods, and effective practice

of systems engineering applied to complex socio-

technical systems through collaborative research


Motivation


MOTIVATIONChanging Face of Systems Engineering

TRADITIONAL SE

Transformation of customer requirements to design

Requirements clearly specified, frozen early

Minimizing changes

Design to meet well specified set of requirements

Performance objectives specified at project start

Focus on reliability, maintainability, and availability

ADVANCED SE

Effective transformation of stakeholder needs to fielded (and sustainable) solution

Focus on product families and systems-of-systems

Complex interdependencies of system and enterprise

Importance of systems architecting

Designing for dynamic relevance

Emphasis on expanded set of “ilities” and designing in robustness, flexibility, adaptability in concept phase


MOTIVATION

Paradigm Shift

Classic paradigm New paradigm


MOTIVATION

Notional ROI

PRACTICE COMPETENCY CULTURE

Basic training

and processes

Strategy to

establish SE as

competency

Transform culture –

mindset and approaches

Effectiveness


Collaboration


Research Predisposed

Toward Application

For most engineering students, the goal of a career in industry

motives their pursuit of advanced study and this will

increasingly be the case on the future. Because of this,

engineering students’ outlook on research is predisposed

toward application in engineering practice

National Academy of Engineering, 2005

Survey of

SEANET

doctoral

students

shows only

25% plan

academic

careers


Engineering education and research

must be a collaborative endeavor

of government, industry, and academia

Complex engineering research can not take place solely in a laboratory within university walls but rather real world enterprises must be our “learning laboratories”

Expanded view of who an “educator” is --faculty, researchers, practitioners, policy makers, peers

Additionally, we need more cross cutting experiences for educators and practitioners alike

Engineering education and research can not be just a cooperation; must be a true collaboration

SE Research

requires real

world laboratory


Sponsor Engagement Models

Classical “basic research” sponsors – Targeted topic toward broad scientific goals

Innovation grant sponsors – Higher risk/higher payoff research

Contract research sponsors – Toward solving sponsor problem

Consortium sponsors – Pooled funds for shared research benefits

“Deep engagement” partnerships – Symbiotic relationship


Research Structure and

Portfolio


Underlying Research Structure

Prescriptive methods seek to advance state of the practice based on

sound principles and theories, as grounded in real limitations and

constraints

• Normative research: identify principles and theories -- “should be”

• Descriptive research: observe practice and identify limits/constraints


Research Projects Map to Structure

Theory Development with ModelingMethodology Guidance Document

Codified Successful Practices based on

Empirical Studies

Method Validated in Real World Cases


Portfolio

RESEARCH PORTFOLIO

1. Socio-Technical Decision Making

2. Designing for Value Robustness

3. Systems Engineering in the Enterprise

4. Systems Engineering Economics

5. Systems Engineering Strategic Guidance


Research Portfolio (1)

SOCIO-TECHNICAL DECISION MAKING

This area of research is concerned with the context of socio-technical systems. Based on a multi-disciplinary approach, decision making techniques are developed through the exploration of:

– Studies of decision processes and effectiveness of techniques

– Constructs for representing socio-technical systems

– Decision strategies for system of systems

– Visualization of complex trade spaces

– Understanding/mitigating cognitive biases in decision processes

While organizational theorists have well developed theories of how

organizations function and make decisions, this understanding needs to

be integrated into the design phase in a quantifiable way….then it will be

the case that apriori the effect of the enterprise organization on the

engineering system will be predicted rather than being a surprise

Hastings, MIT ESD Symposium, 2004


Survivability Tradespace

McManus, H., Richards, M., Ross, A., and Hastings, D., “A Framework for Incorporating

“ilities” in Tradespace Studies,” AIAA Space 2007, Long Beach, CA, September 2007.



DESIGNING for VALUE ROBUSTNESS

This area of research seeks to develop methods for concept

exploration, architecting and design using a dynamic perspective

for the purpose of realizing systems, products, and services that

deliver sustained value to stakeholders in a changing world.

– Methods for dynamic multi-attribute trade space exploration

– Architecting principles for designing survivable systems

– Quantification of the changeability of a system design

– Techniques for consideration of unarticulated and latent value

Value robustness is the ability of a system to continue to deliver stakeholder value

in the face of changing contexts and needs. Architecting value robust systems

requires new methods for exploring the concept tradespace, as well as for decision

making. Also needed are architecting principles and strategies, an approach for the

quantification of changeability, and an improved ability for architects and analysts

to classify value for purposes of dialogue and implementation

Ross and Rhodes, 2008


Linking Tradespaces over Time

Utilit

y (d

imensio

nle

ss)

Time

0 1 2 3 4

1

0

System (Within Epoch) Change Timeline

U

0

Epoch 1 Epoch 2 Epoch 3 Epoch n

…S1,b S1,e S2,b S2,e S3,b S3,e Sn,b Sn,e

T1 T2 T3 Tn

Time

U

0

Epoch 1 Epoch 2 Epoch 3 Epoch n

…S1,b S1,e S2,b S2,e S3,b S3,e Sn,b Sn,e

T1 T2 T3 Tn

Time

System EraPareto Tracing across

Epochs

Rk

ODk

≈Nk

Rk

ODk

≈Nk

Changeability Quantified

as Filtered Outdegree


Research Portfolio (3)SYSTEMS ENGINEERING in the ENTERPRISE

This research area involves empirical studies and case based researchfor the purpose of understanding how to achieve more effective systems engineering practice in context of the nature of the system being developed, external context, and the characteristics of the associated enterprise.

– Engineering systems thinking in individuals and teams

– Collaborative, distributed systems engineering practices

– Social contexts of enterprise systems engineering

– Alignment of enterprise culture and processes

– Socio-technical systems studies and models

The understanding of the organizational and technical interactions in our

systems, emphatically including the human beings who are a part of them,

is the present-day frontier of both engineering education and practice.

Dr. Michael D. Griffin, Administrator, NASA, 2007 Boeing Lecture, Purdue University


MIT/MITRE Social Contexts of ESE

(2006 – 2008)

Collaborative Distributed SE (Utter 2007)

Empirical Studies of Systems

Thinking (Davidz 2006)

Collaboration

Situation and

Management

AA BB

Knowledge, Data

and Decision

Management

SE Processes

and Practices

Collaboration

Tool Use

CDSE Social

and Cultural

Environment

CDSE Benefits

and Motivation

Description RecommendationLesson

LearnedIssue or Barrier Success Factor Irrelevant

OR

Tool Training Network Reliability Tool VersionsTool AccessLearning

CurvesClassified Data

Interview Heading Topics

Company

Transcripts

Subtopic

Interviewee Experience

Data Analysis

Example

SE in the Enterprise

Empirical Research



SYSTEMS ENGINEERING ECONOMICS

This research area aims at developing a new paradigm that encompasses an economics view of systems engineering to achieve measurable and predictable outcomes while delivering value to stakeholders.

– Measurement of productivity and quantifying SE ROI

– Advanced methods for reuse, cost modeling, and risk modeling

– Application of real options in systems and enterprises

– Leading indicators for systems engineering effectiveness

In a 2004 Air Force/MIT workshop, Dr. Marvin Sambur, (then) Assistant Secretary

of the AF for Acquisition, noted that the average program is 36% overrun

according to recent studies – disrupting the overall portfolio of programs


Models, Measures, and Leading

Indicators for Project SuccessThrough Better Execution of Systems Engineering

Cost and schedule modelingProject Risk Assessment

Leading Indicators for Performance Systems Engineering ROI

Person Months Risk

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

55%

60%

65%

70%

75%

80%

85%

90%

95%

100%

0 25000 50000 75000 100000

Person Months

Risk (= Prob. That Actual Person Months

Will Exceed Indicated, X-Axis, Figure)

Person Months Confidence (Cumulative Probability)

0%

5%10%

15%20%25%30%35%40%

45%50%55%60%65%70%

75%80%

85%90%95%

100%

0 25000 50000 75000 100000

Person Months

Cumulative Probability of Person

Months



SYSTEMS ENGINEERING STRATEGIC GUIDANCE

This research area involves synthesis of theory with empirical and case based research for the purpose of developing prescriptive strategic guidance to inform the development of policies and procedures for systems engineering in practice.

– Systems Engineering research guidelines

– Participation in focus groups and pilot-phase reviews

– Position papers on proposed policies

– Recommendations for integrating SE research into curriculum

– Identification of SE research gaps and opportunities

The full impact of systems engineering research can only

be achieved through synthesis of research outcomes


Guide to SE

Leading IndicatorsJune 2007

Guide to SE Leading Indicators(December 2005)

BETA

AF/DOD

SE Revitalization Policies

AF/LAI Workshop on Systems Engineering

June 2004

SE LI Working Group

With PSM

+

Pilot Programs

(several companies)

Masters Thesis

(1 case study)

Validation Survey

(>100 responses/ one

corporation)

SE LI Working Group

With SEAri and PSM

+

+

V. 1.0

Knowledge

Exchange

Event

Tutorial on SE Leading

Indicators

(many companies)

(1) January 2007

(2) November 2007

Practical Software

& Systems

Measurement

Workshops

(1) July 2005

(2) July 2007

(3) July 2008

Applications

IBM® Rational Method Composer – RUP

Measurement Plug-in

Systems

Engineering

Leading

Indicators

Project


Transfer to Practice


Technology Transfer

Publications

• Research Bulletin

• Conference Papers

• Journal Papers (preprint)

• Working Papers (SEAri, ESD, others)

• Conference and Event Briefings

• Theses (with Exec Summaries)


Publications Highlight

Implications for Practice

Dynamic Multi-Attribute Tradespace Exploration Method

Implications for Systems Engineering Practice1. Ability to explore many design options and prevent too early

focus on single ‘point design’

2. Enables quantitative assessment of factors such as variability

in technical performance and cost, and impacts in markets

3. Suitable to multiple domains and demonstrated to improve

decision making

Vision: designers will have an

enhanced ability to consider concept

alternative in a rigorous way, not only

for present situation but also in

considering futures where needs and

contexts have shifted


Technology Transfer

Involvement of Students

• Case Studies

• Student Internships

• Student Research Presentations Onsite

• Research Sponsorship Engagement

• Students Hired into Organizations

• SEAri Alumni Network


Technology Transfer

Professional Education

• SEAri Lectures at MIT

• SEAri Lectures for Sponsors

• SEAri Tutorials (conference, etc.)

• MIT Professional Institute

• SDM Program Certificate Program

• Sponsored Research Deliverables


Technology Transfer

Strategic Initiatives

• Policy Recommendations

• Innovation Research Initiatives

• Review Teams for Industry Standards and Guides

• Advisory Boards of Selected Organizations

• Leadership of INCOSE Doctoral Network

• Leadership Roles in Professional Societies,

Industry Working Groups, and Study Boards


Access to

Interim Research Results

SEARI Research Bulletin Published

at End of Each Semester

2008


October 21

MIT Faculty Club


SEAri Website

http://seari.mit.edu

Portal to information:

• Publications

• Briefings

• Research Bulletins

• Upcoming Courses

• Other Information


Indicators of Research

Dissemination and Relevance• 28,000 website visitors in two years

• 22 conference publications last year (30+ year ahead)

– 2008 IEEE Systems Best Paper

– Both Best Paper Awards at INCOSE 2008

• Awarded highly competitive innovation research project from a USgovernment agency

• Multi-national, cross-domain, multi-sector research program

• First MIT Professional Institute Class (13 organizations)

• More than two dozen organizations visited

• First SEAri Summit Held

We seek your input on ways to make research

relevant and available


Agenda

9:00 Welcome and Introductions Dr. Donna Rhodes, SEAri Director

9:30 SEAri – Overview of the SEAri Research Program Dr. Donna Rhodes, Research Director

10:00 Research Profile: Socio-Technical Decision Making and Designing for Value Robustness

Dr. Adam Ross, Research Scientist

10:45 Break

11:00 Research Report: Designing Systems for Survivability Matt Richards, Doctoral Research Assistant

11:30 Research Report: Real Options in Enterprise Architecture Tsoline Mikaelian, Doctoral Research Assistant

noon LUNCH

1:00 Research Profile – Systems Engineering in the Enterprise Dr. Donna Rhodes, Principal Research Scientist

1:30 Research Report: Leveraging Organizational Culture, Standard Process, and Team Norms to Enable Collaborative Systems Thinking

Caroline Lamb, Doctoral Research Assistant

2:00 Stretch Break

2:10 Research Profile: Systems Engineering Economics Dr. Ricardo Valerdi, Research Associate

2:50 Research Poster Session with Refreshments SEAri Research Assistants

4:15 Participant Feedback and Recommendations for Research SEAri Leadership

4:55 Closing Remarks Dr. Donna Rhodes

5:00 Adjourn

welcome [seari.mit.edu]seari.mit.edu/documents/summit/2008/01-searisummit... · view of...

Documents