systems engineering – a perspective dinesh verma, ph.d. professor and associate dean stevens...
TRANSCRIPT
Systems Engineering – Systems Engineering – A PerspectiveA Perspective
Dinesh Verma, Ph.D.Professor and Associate DeanStevens Institute of Technology
Simple Definition…Simple Definition…
Systems Engineering is “problem solving and solution delivery.” A key prerequisite to good “problem solving” is good “problem definition.” Now this has other prerequisites!
Some best practices: Translating customer needs (business and technical) into
acceptance criteria - 5 to 7 critical customer requirements agreed to in measurable/testable form.
Identifying requirements and then managing them (and tracing them) through the subsequent development, integration, testing phases.
Translating the requirements into an “architecture” that becomes a “linkage” between what the customers want and what the developers will build and test.
Developing a test architecture, test plans and procedures that are traceable to the requirements for maximum focus and efficiency
Sounds very simple! A lot of organizations have developed processes that attempt to capture the above intent. But very few are able to execute it…
System Design Life Cycle Models:System Design Life Cycle Models:An Automotive Example (VOLVO Car Corporation)An Automotive Example (VOLVO Car Corporation)
3
System Design Life Cycle System Design Life Cycle Models:Models:A Telecom Example (NOKIA Networks)A Telecom Example (NOKIA Networks)
E-1 E0 E1 E2 E3 E4 E5Define Plan and
specify
Design and
implement
Implement and
integrate
Verify Ramp-up
Program
initiated
Program proposal
ready
Program plan
ready
Ready for integration
Ready for verificatio
n
Ready for Ramp-up
Capability for
Volume Deliveries
E0.5
Program main contents frozen for program planning purposes (optional)
Requirementsspecs done
Real HW done and HW in maintenance mode. HW and SW main verification starts. SW is module tested and proof on product functionality exist (=SW implementation ready).
Traditional Pilot deliveries start. HW and SW have been tested together and released as a product
"Proof of concept" *HW implemented.Real HW and basic/ lowlevel SW integrated andcore functionality works. Idea of performance exists.First SW build made. Proof of product architecture.
Commitment of features, resources and milestone dates.Specification done
Volume deliveries can start
Program allowed to start using resources
E1.5
Product Design frozen(optional)
Program established
E 3.5
Trial deliveries can start (Optional)Functional tests done and HW fullfillslegal type approval requirements
E 5.5
Programcompleted(optional)
* Core functionality can be I.e. control plane,signal goes through (typically not call yet). Exact contentsof core functionality is need to be defined in E1
Optional Milestones can be moved.I.e. E1 and E1.5 dates can be the same.
4
System Design Life Cycle Models:System Design Life Cycle Models:A Workstation Example (SUN A Workstation Example (SUN Microsystems)Microsystems)
5
The IBM AMS Systems Engineering Process The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (I)defines deliverables and a series of Reviews (I)
Need / OpportunityIdentification
Detailed DesignComponent Architecture
ConceptualSystem
Specification
CustomerCustomerBaselineBaseline
SystemSystemBaselineBaseline
Architecture/ComponentArchitecture/ComponentBaselineBaseline
DesignDesignBaselineBaseline
SystemsSystemsRequirementsRequirementsReview (SRR)Review (SRR)
PreliminaryPreliminaryDesign Design Review (PDR)Review (PDR)
CriticalCriticalDesignDesignReview CDR)Review CDR)
Customer Provided Systems Engineering Provided Component Developer Provided
Business Business RequirementsRequirementsReview (BRR)Review (BRR)
BusinessBusinessRequire.Require.Specs.Specs.
SystemsSystemsReq’mentReq’mentSpecsSpecs
RTVMRTVMSystemSystemLevelLevelArchitect.Architect.
ComponentComponentLevelLevelArchitectureArchitecture
TestTestArchitectureArchitecture
ComponentComponentDesignDesign
Component Component Test PlanTest Plan
ComponentComponentReq’ment Req’ment SpecsSpecs
ComponentComponentRTVMRTVM
6
The IBM AMS Systems Engineering Process The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (II)defines deliverables and a series of Reviews (II)
TestTestBaselineBaseline
ProductionProductionBaselineBaseline
DesignDesignBaselineBaseline
TestTestReadinessReadinessReview (TRR)Review (TRR)
Production Production Readiness Readiness Review (PRR)Review (PRR)
Customer Provided Systems Engineering Provided Component Developer Provided
CDRCDR
New ProductionSystem
Test and ProductionSystem Update
Development
System System Test Test DataData
Test Test Traceability Traceability Matrix.Matrix.
Move toMove toProd.Prod.PlanPlan
Data Data MigrationMigrationPlanPlan
Detail Design
Comp.Comp.DesignDesign
Comp. Comp. Test PlanTest Plan
System Test Provided Service Delivery / Managed Ops Provided
DeploymentDeploymentPlanPlan
System System Test Plan /Test Plan /Test CasesTest Cases
System System TestTestStrategyStrategy
ReleaseReleaseContentContent
7
SE in the System Life CycleSE in the System Life Cycle“The Wall Chart”“The Wall Chart”
8
Key Concepts…Key Concepts…
The notion of Systems Thinking and Contextual Setting The notion of Baselines
Facilitates requirements definition and management, change management, and scope control
The notion of Functional (Dynamic) And Non-Functional (Static) Requirements A holistic view that addresses new or modified Business Processes,
while also addressing concerns pertaining to Performance, Availability and Reliability, Scalability and Security
The notion of Baseline Reviews and Objective Metrics An in-process review to validate completeness of the baseline and
identify risks and defects The notion of the System Life Cycle (Define – Design – Build – Test
– Deploy – Operate – Upgrade/Retire) Systems Engineering should have “ownership” and accountability
through the various phases of the life cycle.
““I try to be very clear in separating I try to be very clear in separating the “problem space” from the the “problem space” from the “solution space”.” Chief Architect, “solution space”.” Chief Architect, NOKIA Technology PlatformsNOKIA Technology Platforms
“I try to design at three levels: my level, “I try to design at three levels: my level, the level above me, and the level below the level above me, and the level below me.” Chief Architect, L3 Communicationsme.” Chief Architect, L3 Communications
““I architect and design in two iterations. In the I architect and design in two iterations. In the first, my focus is on feasibility and completeness, first, my focus is on feasibility and completeness, while in the second, my focus is on “goodness” while in the second, my focus is on “goodness” and other refinements and embellishments such and other refinements and embellishments such as scalability, modularity, and such.” Chief as scalability, modularity, and such.” Chief Architect, IBM Global ServicesArchitect, IBM Global Services
10
Systems Engineering – ExpectationSystems Engineering – Expectation
Successful implementation of proven,
disciplined systems engineering
processes results in a total system
solution that is:
Robust to changing technical, production, and
operating conditions;
Adaptive to the needs of the users; and
Balanced among the multiple requirements,
design considerations, design constraints, and
program budgets.
11
Systems Engineering – ExpectationSystems Engineering – Expectation
Successful implementation of proven,
disciplined systems engineering
processes results in a total system
solution that is:
Robust to changing technical, production,
and operating conditions;
Adaptive to the needs of the users; and
Balanced among the multiple requirements,
design considerations, design constraints, and
program budgets.
12
DoD Systems Engineering DoD Systems Engineering Shortfalls*Shortfalls*
Root cause of failures on acquisition programs include: Inadequate understanding of requirements
Lack of systems engineering discipline, authority, and resources
Lack of technical planning and oversight
Stovepipe developments with late integration
Lack of subject matter expertise at the integration level
Availability of systems integration facilities
Incomplete, obsolete, or inflexible architectures
Low visibility of software risk
Technology maturity overestimated
* DoD-directed Studies/Reviews
Major contributors to poor program performance
13
Customer Related Input:• Isolation from real “user”
• Customer requirements and (even) identity not clear
• Customer doesn’t know what they want
• Scope creep; Undocumented system scope and functionality
• User/buyer too distant
• Don’t understand the customer value system
Management Related Input:• Executive management doesn’t buy in
• Lack of teamwork
• Program Managers not empowered
• Program manager and capture managers are different
• Unstable funding stream; Lack of upper management support
Organizational/Cultural Input (Some Perceptions):• SEA only adds to the Project Cost
• SEA often seen as an “outside” team or “project reviewer” role
Some Inhibitors to Good Systems Engineering:Based on a survey of IT architects and project managers
14
We would like you to We would like you to build us a lawn build us a lawn mower please!mower please!
Theory versus Theory versus (Virtual)(Virtual) Reality… Reality…Primary Reasons for Dysfunctional Behavior – My Primary Reasons for Dysfunctional Behavior – My OpinionOpinion
Confusion between “What you NEED” versus “What you WANT” Also called Gold-Plating
It is the moral duty of a systems engineer to articulate the resulting cost and schedule delta
Confusion with regard to the SYSTEM BOUNDARY This is more difficult for legacy systems with undocumented and
implied interfaces; and even more so for “network-centric systems” and “SoS”
Confusion (?) with regard to fidelity between the technical project scope and its allocated budget and schedule
The result is cynicism and complacency, along with other negative behavioral patterns
Lack of Leadership
Leadership at the Individual Level…Leadership at the Individual Level…
Leadership and Focus
Honesty
Openness – Lexicon (Key Issue)
Domain Knowledge – This is the key ingredient missing in most
systems engineers today, resulting in significant impact to the
credibility of the discipline
Ability for Systems Thinking
A very strong mental model, in the face of:
A diverse lexicon
Need for rough estimates, and to identify outliers
Changing standards (are they really changing?)
New fads (6-sigma, IPPD, Concurrent Engineering, Simultaneous
Engineering)
Changing life cycle models (incremental, spiral, V, etc.)16
Deploying Systems Engineering within a Deploying Systems Engineering within a Commercial Global Leader: Commercial Global Leader: Some Some ResultsResults
Systems Engineering Has Been Applied to Systems Engineering Has Been Applied to Both Internal and Commercial Accounts Both Internal and Commercial Accounts
20012001200020002nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr 2nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr
1st project uses SE principlesSE organization introduced
SE Reviews / Scorecards introduced Directive to use SE on projects >$1M
‘Fundamentals of SE’ course introduced1st commercial account uses SE
Directive to use SE on projects > $500KFormal SE dept created
20032003200220022nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr 2nd qtr2nd qtr1st qtr1st qtr 4th qtr4th qtr3rd qtr3rd qtr
13 projects using SE
SEI introduces CMMI 1.1‘SE Design’ Class introduced
SE deliverables templates provided
SE team grows to 14
17 completed and over 50 active projects using SEOver 230 trained in SE Fundamentals
SE team grows to 3012 completed and 13 active projects using SE
SE process integration - AMS MS
SE process updated for CMMI
SE process integration - GS Method
Systems Engineering Process defines Systems Engineering Process defines deliverables and a series of Reviews (Part I)deliverables and a series of Reviews (Part I)
Need / OpportunityIdentification
Detailed Design
Component Architecture
ConceptualSystem
Specification
CustomerCustomerBaselineBaseline
SystemSystemBaselineBaseline
Architecture/ComponentArchitecture/ComponentBaselineBaseline
DesignDesignBaselineBaseline
SystemsSystemsRequirementsRequirementsReview (SRR)Review (SRR)
PreliminaryPreliminaryDesign Design Review (PDR)Review (PDR)
CriticalCriticalDesignDesignReview CDR)Review CDR)
Customer Provided Systems Engineering Provided Component Developer Provided
Business Business RequirementsRequirementsReview (BRR)Review (BRR)
BusinessBusinessRequire.Require.Specs.Specs.
SystemsSystemsReq’mentReq’mentSpecsSpecs
RTVMRTVMSystemSystemLevelLevelArchitect.Architect.
ComponentComponentLevelLevelArchitectureArchitecture
TestTestArchitectureArchitecture
ComponentComponentDesignDesign
Component Component Test PlanTest Plan
ComponentComponentReq’ment Req’ment SpecsSpecs
ComponentComponentRTVMRTVM
Systems Engineering Process defines Systems Engineering Process defines deliverables and a series of Reviews (Part II)deliverables and a series of Reviews (Part II)
TestTestBaselineBaseline
ProductionProductionBaselineBaseline
DesignDesignBaselineBaseline
TestTestReadinessReadinessReview (TRR)Review (TRR)
Production Production Readiness Readiness Review (PRR)Review (PRR)
Customer Provided Systems Engineering Provided Component Developer Provided
CDRCDR
New ProductionSystem
Test and ProductionSystem Update
Development
System System Test Test DataData
Test Test Traceability Traceability Matrix.Matrix.
Move toMove toProd.Prod.PlanPlan
Data Data MigrationMigrationPlanPlan
Detail Design
Comp.Comp.DesignDesign
Comp. Comp. Test PlanTest Plan
System Test Provided Service Delivery / Managed Ops Provided
DeploymentDeploymentPlanPlan
System System Test Plan /Test Plan /Test CasesTest Cases
System System TestTestStrategyStrategy
ReleaseReleaseContentContent
A “productized” process for efficient implementation
Globally consistent templates and processes,
Uniform and consistent metrics and lexicon (part of the SE culture)
Focus must be on the “necessary” and critical subset of the overall methodology and theory (Flexibility and Adaptability)
Tailoring for time-to-market considerations
Tailoring for schedule and resource considerations
Risk tolerance must be explicitly considered in the tailoring process
Implementation must be organizationally supported and nurtured
Linkage to strategic organizational goals is key
A well managed competency development program and a “community of practice”
Successful implementation of System Successful implementation of System Engineering needs…Engineering needs…
ISM delivered 5% under budget ISM delivered 5% under budget and with higher quality in production and with higher quality in production
The charts here are based on data collected from a recent study analyzing project defects by type and phase. Here ISM defects by phase is compared to 46 similarly sized projects not utilizing SE.
Total defect counts for non-SE projects exhibited 53.4% of total project defects during the Test Phase of the project. On ISM defects were detected earlier in the project life-cycle. In fact 56% of ISM detects were detected in Plan Phase.
The chart on the left illustrates the cost implications of early defect detection as found with ISM 2.0.
In effect ISM 2.0 expended 2.4 times less than what would have normally been required for the non-SE oriented projects compared to in the study.
IGA Metrics show 8% cost avoidance when IGA Metrics show 8% cost avoidance when comparing SE&A projects to non-SE&A comparing SE&A projects to non-SE&A projectsprojects
Cumulative Costs to Repair Requirement and Design Defects
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
$1,600,000
Req Design Build Test Install
Phase Detected
Accu
mu
late
d C
osts
SE&A non SE&A
= 15% of $10.7MM Baseline
= 7% of $10.7MM Baseline
= 8% Cost Avoidance
Discipline Centric Systems Discipline Centric Systems Engineering Programs:Engineering Programs: These are programs where These are programs where the major is designated the major is designated only as Systems only as Systems Engineering Engineering
Domain Centric Systems Domain Centric Systems Engineering Programs:Engineering Programs: These These are programs where the major are programs where the major is designated as X and is designated as X and Systems Engineering; or Systems Engineering; or Systems and X Engineering.Systems and X Engineering.
In this case, the most common In this case, the most common instances of “X” Engineering instances of “X” Engineering are:are:
Industrial EngineeringIndustrial Engineering
Manufacturing EngineeringManufacturing Engineering
Electrical EngineeringElectrical Engineering
Management EngineeringManagement Engineering
Computer EngineeringComputer Engineering
Academic Perspective: Discipline and Academic Perspective: Discipline and Domain Centric Systems Engineering Domain Centric Systems Engineering ProgramsPrograms
The primary source of this data is: Fabrycky, W.J., “Systems Engineering Education in the United States”, Proceedings, Conference on Systems Integration (CSI), Stevens Institute of Technology, New Jersey, March 2003.
Air Force Institute of Technology Air Force Institute of Technology
California State UniversityCalifornia State University
Colorado School of MinesColorado School of Mines
Cornell UniversityCornell University
George Mason UniversityGeorge Mason University
George Washington UniversityGeorge Washington University
Iowa State UniversityIowa State University
Johns Hopkins UniversityJohns Hopkins University
National Technological UniversityNational Technological University
Naval Postgraduate SchoolNaval Postgraduate School
Oakland UniversityOakland University
Polytechnic University - FarmingdalePolytechnic University - Farmingdale
Portland State UniversityPortland State University
Purdue UniversityPurdue University
Rochester Institute of TechnologyRochester Institute of Technology
Southern Methodist UniversitySouthern Methodist University
Stevens Institute of TechnologyStevens Institute of Technology
University of Alabama - HuntsvilleUniversity of Alabama - Huntsville
University of ArizonaUniversity of Arizona
University of IdahoUniversity of Idaho
University of Illinois at Urbana-ChampaignUniversity of Illinois at Urbana-Champaign
University of Maryland University of Maryland
University of MassachusettsUniversity of Massachusetts
University of MinnesotaUniversity of Minnesota
University of Missouri - RollaUniversity of Missouri - Rolla
University of PennsylvaniaUniversity of Pennsylvania
University of Rhode IslandUniversity of Rhode Island
University of Southern CaliforniaUniversity of Southern California
University of VirginiaUniversity of Virginia
VPI and State UniversityVPI and State University
Academic Perspective: Universities with Academic Perspective: Universities with Discipline Centric Systems Engineering Discipline Centric Systems Engineering Programs - 30Programs - 30
The list in the primary reference contains 35 records. Four of these referred to Universities with only an Undergraduate Program in Systems Engineering, and one to a University with only a Doctoral Program in Systems Engineering.
Auburn University
Boston University
California State University - Fullerton
Case Western Reserve University
Georgia Tech
Massachusetts Institute of Technology
New Jersey Institute of Technology
North Carolina A and T University
Northeastern University
Ohio State University
Ohio University
Polytechnic University
Purdue University
Rensselear Polytechnic University
Rutgers, The State University
San Jose State University
Stanford University
Texas Tech University
University of Alabama - Huntsville
University of Arizona
University of Central Florida
University of Connecticut
University of Florida
University of Houston
University of Illinois
University of Memphis
University of Michigan Ann Arbor
University of Michigan-Dearborn
University of Minnesota
University of Pittsburgh
University of Rhode Island
University of South Florida
University of Southern California
University of Southern Colorado
University of St. Thomas
Virginia Tech
Wichita State University
Youngstown State University
Academic Perspective: Universities with Domain Academic Perspective: Universities with Domain Centric Systems Engineering Programs - 38Centric Systems Engineering Programs - 38
3030++3838
~15~15
Available…Available… Relevant…Relevant…
Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining
Definition of Relevant:Definition of Relevant:1.1.Relevance of the curriculum – orientation Relevance of the curriculum – orientation
to DoD projects and programsto DoD projects and programs2.2.Portability and flexibility of the delivery Portability and flexibility of the delivery
format – distributed organizationsformat – distributed organizations3.3.Hybrid – credit and continuing educationHybrid – credit and continuing education
Basis:Basis:The Boeing SE Education Program (50 > 15 > 6 > 2)The Boeing SE Education Program (50 > 15 > 6 > 2)A DoD Component SE Education Program (80 > 25 > A DoD Component SE Education Program (80 > 25 >
11 > 2)11 > 2)
??3030++3838
~15~15
Available…Available… Relevant…Relevant… Critical Mass…Critical Mass…
Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining
Definition of Critical Definition of Critical Mass:Mass:
1.1.Number of Tenured Number of Tenured or Tenure Track or Tenure Track FacultyFaculty
2.2.Number of Faculty Number of Faculty with DoD/Aerospace with DoD/Aerospace Relevant Relevant Project/Program Project/Program ExperienceExperience
3.3.Bench StrengthBench Strength
……Why?Why?
??3030++3838
~15~15
Available…Available… Relevant…Relevant… Critical Mass…Critical Mass…
Industry/Government Perspective: Industry/Government Perspective: Systems Engineering Education and Systems Engineering Education and TrainingTraining
Definition of Critical Definition of Critical Mass:Mass:
1.1.Number of Tenured Number of Tenured or Tenure Track or Tenure Track FacultyFaculty
2.2.Number of Faculty Number of Faculty with DoD/Aerospace with DoD/Aerospace Relevant Relevant Project/Program Project/Program ExperienceExperience
3.3.Bench StrengthBench Strength
It is my opinion that we do not have a critical mass in graduate SE education in the It is my opinion that we do not have a critical mass in graduate SE education in the US…US…
However, we do have a very mature base and recognition within industry and However, we do have a very mature base and recognition within industry and government to facilitate the building of this critical mass…government to facilitate the building of this critical mass…
A Fundamental DilemmaA Fundamental Dilemma
We’ve forgotten (and are continuing to forget) much of the systems engineering we once knew.
Reversing this trend has become an imperative for both Government and commercial industry.
and yet…
What we once knew may be inadequate for solving the systems engineering problems we now face.
30
What do I mean…What do I mean…
Security in the Port of NY & NJ Ten threat scenarios were defined Each is low-likelihood, high-
consequence What is the eleventh?
The Paradox of Ownership and The Paradox of Ownership and Boundaries…Boundaries…
The Driver and the Driven – The The Driver and the Driven – The System and the Organization…System and the Organization…
Top Down…Top Down…
