© 2014 spec innovations and teaching science and technology, inc. all rights reserved model-based...

© 2014 SPEC Innovations and Teaching Science and Technology, Inc. All Rights Reserved

Model-Based Reviews

The Next Step in Model-Based Systems Engineering

Presented by Steve Dam, SPEC Innovations and Jerry Sellers, Teaching Science and Technology, Inc. (TSTI)

April 2014

1


Presentation Outline

• What is Model-Based Systems Engineering (MBSE)?

• What is a Model-Based Review (MBR)?

• MBR Example• Discussion

2


What is Model-Based Systems Engineering (MBSE)?

• Computer-based modeling and simulation capabilities have been around since the 1960s

• Application of modeling and simulation to systems engineering has been around almost as long (e.g., SREM in 1969)

• In 2007, INCOSE began the MBSE initiative to promote the movement from a document centric SE environment to a model centric one

3

“The formalized application of modeling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later life cycle phases. (INCOSE 2007).” From SEBoK


MBSE Vision (2007)• Presented at

INCOSE International Symposium in San Diego by Sandy Friedenthal, Regina Griego, and Mark Sampson

• Goal: Make MBSE the norm by 2020

4


MBSE Must Support the Overall Program Lifecycle

• Every lifecycle process has milestone or “gate” reviews

• Traditionally, ever increasingly large amounts of documents must be produced to support these reviews

5

IOCBA

System Development& Demonstration

Production & Deployment

Pre-Systems Acquisition

Systems Acquisition

Operations& Support

C

Sustainment

Concept Refinement

TechnologyDevelopment

System Integration

System Demonstration

LRIP Full-Rate Prod & Deployment

DesignReadinessReview

FRPDecisionReview

Sustainment

Disposal

FOC

ConceptDecision


MBSE Implications for Reviews• To support the traditional process, MBSE tools

have focused on providing this documentation as an output from the datacaptured

• Benefit: as the data changes, new versions of the report can quickly be produced

• Benefit: Data overlap between documents is handled automatically

• But do these documents reflect all the information captured in the tool?

6


MBSE Implications for Reviews

• The simple answer to the previous question is no, but many may consider it “good enough”

• The problem with “good enough” is that errors in the model that propagate to the resulting sub-systems/components, still may not be discovered until too late

• Also, we spend a large amount of time and money preparing for, conducting, and post review

• Reviews are the equivalent of aproduct inspection

7

“You can not inspect quality into the product;

it is already there.” W. Edwards Deming


Model-Based Reviews (MBR)• What would a MBR entail?

– Ideally this would be accessing the model in it’s native environment (a tool)

– Be able to “walk through” the model to understand how it works, what assumptions are made, what decisions were made and why, what risks these assumptions result

– Have a place to provide comments for adjudication

• Hence, this is both a process and supporting tool(s)

8


Example of a MBR

• For a Stevens Institute of Technology graduate course, Dr. Jerry Sellers had the students conduct MBRs for the project– Using Inspiration Mars as the system of

interest

• They conducted two reviews: a preliminary Mission Concept Review (MCR) and final MCR

• Used NASA input/exit criteria to set up reviewer guide

9


Background - SYS 638• SYS638: Human Spaceflight Design

– Graduate course at Stevens Institute of Technology

• The 6 students were given the top-level design concept for Inspiration Mars as our System of Interest– Proposed by Dennis Tito to send 2 humans on a 501 day free-return voyage around

Mars and back and asked to analyze the technical merits of the proposed mission. – They were asked to reverse systems engineer the proposed mission and apply

human spaceflight design and analysis techniques to validate the proposed architecture.

• To support the challenge of conducting a design course 100% online with students spread across the country, we leveraged a new cloud-based model-based systems engineering tool, Innoslate. – Gave us the additional opportunity to research how to perform a model-based design

review. – This topic is of keen interest to NASA and other organizations who are increasingly

interested moving toward MBSE but who are uncertain as to how to transition from traditional paper-based reviews to model-based ones.

10


Inspiration Mars Foundation• The iMars Foundation is on a mission to...

• inspire "the next generation of explorers to pursue their destiny through STEM education."

• "...push the envelope of human experience, while reaching out to our youth to expand their views of their own futures in space exploration."

• "...provide a platform for unprecedented science, engineering and education opportunities, using state of the art technologies derived from NASA and the [ISS]."

• The iMars Foundation consists of:

•Dennis Tito - Foundation Chairman and former ISS visiting astronaut

•NASA Ames Research Center

•Paragon Space Development Corp

•Applied Defense Solutions

•SpaceX

www.inspirationmars.org

11


iMars Mission• "In 2018, the planets will literally align, offering a unique orbit opportunity to travel to Mars and back

to Earth in only 501 days. Inspiration Mars is committed to sending a two-person American crew – a

man and a woman – on an historic journey to fly within 100 miles around the Red Planet and return

to Earth safely."

"The mission’s target launch date is Jan. 5,

2018. This exceptionally quick, free-return orbit

opportunity occurs twice every 15 years. After

2018, the next opportunity won’t occur again

until 2031... It will be financed primarily through

philanthropic donations, with some potential

support from government sources."www.inspirationmars.org

12


Model-based Review ConceptsApproach Description Pros Cons

Reviewer Asset-Action

Reviewers are entered into the model as “assets” and each asset is assigned an “action” within the model. Each action describes the steps the reviewer needs to take to complete the review.

• Review is contained completely within the model• Model is directly critiqued

• Model becomes very complex depending on the amount of reviewers• Reviewers require good working

knowledge of using the model

Artifact

Model elements are exported into document files that are provided to the reviewers.

• Requires no working knowledge of the model• Uses a more familiar

document-centric review approach

• Review is held outside the model• Model is not directly critiqued• Loses the essence of a “model-

based review”

ModelReviewDiagram

A diagram is generated within the model that can be followed by reviewers. Each object in the diagram contains the items to be reviewed. The diagram can be either sequential or follow an entrance/exit criteria.

• Review is contained completely within the model• Model is directly critiqued

• Model becomes more complex • Reviewers require some working

knowledge of using the model

Hyperlink

Reviewers are given a set of hyperlinks that directs them to model elements.

• No additional items need to be created in the model• Model is directly critiqued• Allows the option for the

reviewer to independently review the model depending on their comfort level

• Reviewers require little working knowledge of using the model• Hyperlinks must be well managed

13


Model-based Review Process

14


Step Process Details

1Reviewers Receive Project Invite • All reviewers were sent an e-mail invitation along with general instructions for the review and Innoslate

account setup

2Reviewers Setup Innoslate Account • All reviewers setup an account with Innoslate

O It’s a free subscription• Model owner assigned all reviewers read-only status

3Reviewers Participate-In/Review Presentation Primer

• Team conducted a Webex presentation primer • Reviewers had opportunity to ask questions• Reviewers that could not attend viewed the recording

4

Reviewers Examine MCR Document • MCR document contained the entrance & exit criteria along with all the relevant hyperlinks to the model and a self-assessment of how well the team performed against the criteria• Team created a shared spreadsheet MCR document using Google Drive (AKA Google Docs) and it was discussed during presentation primer• Reviewers examined MCR document

5Reviewers Examine Model • Reviewers used the hyperlinks in the MCR document to navigate the model

• Reviewers had the freedom to explore the model deeper once directed through the hyperlink

6Reviewers Create Comment/Action/Approval

• Reviewers left Comments/Actions/Approvals directly in an entity in the model• These were tracked using the ‘export comment report’ feature in Innoslate

7 Reviewers Complete Online Survey • Reviewers completed an online survey through www.surveymonkey.com• Most questions used Likert scaling with the ability to leave direct comments

8Team Updates Model Based On Feedback

• Team reviewed all feedback (model and survey comments) and allocated actions among team members to update the model as required

9 Team Peer Review • Team met in a Webex environment to discuss actions, updates and lessons learned

10Team Documents Lessons-Learned • Team documented all lessons-learned in MBR paper

11Team Baselines Model • Team baselined model using the ‘export project’ feature in Innoslate

• Baseline model become more of an artifact than a means of directly evaluating future updatesO Innoslate is working on baseline feature

Model-based Review Process

15

http://www.surveymonkey.com/


Review Navigation Process

Road Map Spreadsheet with hyperlinks to model

Road Map Spreadsheet Expanded view Read instructions and click link Instructions

Link takes to artifact in model

Review artifact and leave comments

16


Artifact Example - WBS

17


Artifact Example - ConOps

18


Artifact Example - Mission Context

19


Artifact Example - Capabilities AllocationAction(System Capability)

Description included by (Function)

generated by (characteristic)

C.0 Capabilities These capabilities span the range of those needed for the iMars mission. F.0 Perform iMars Mission

C.1 Mission Duration Mission Duration includes total mission duration, medium and short durations for the iMars mission.

F.0 Perform iMars Mission

C.1.1 Short Mission Durations Short durations include single artifacts for the iMars mission (i.e. staging, impulses, etc.).

C.1.2 Medium Mission Durations Medium durations include all mission segments greater than impulsive (short) durations.

C.1.3 Total Mission Duration Total mission duration is the entire iMars mission duration. CH.1.1 Mission Duration

C.2 Propulsion Provide thrust, delta-V, course corrections, breaking burns. Includes propellant loading storage and pressurizations functions.


C.2.1 Fueling and Conditioning Includes propellant loading storage and pressurizations capabilities for the iMars mission.

F.4 Perform Pad & Launch Operations

C.2.2 Major Maneuver Delta V Provide thrust and major maneuver delta-V and breaking burns for the iMars mission.

F.7 Perform TMI OperationsF.9 Perform Mars Fly-By Operations

C.2.3 Course Correction Maneuvers Provide course correction maneuvers during transit operations for the iMars mission.

F.8 Perform Trans Mars Coast OperationsF.10 Perform Trans Earth Coast OperationsF.11 Perform Earth Arrival Operations

CH.2.3 TMI Delta V

C.2.4 Gross Lift-Off and Payload Insertion Mass

Captures the Gross Lift-off Mass (GLOM) and Payload Insertion Mass for each system supporting the iMars mission.

F.5 Perform Ascent Operations CH.2.1 PayloadCH.2.2 Launch Trajectory

C.2.5 In-Flight Propellant Storage Provides for in-flight propellant storage capabilities - boil-off and zero boil-off capabilities for the iMars mission.

F.7 Perform TMI OperationsF.8 Perform Trans Mars Coast OperationsF.9 Perform Mars Fly-By OperationsF.10 Perform Trans Earth Coast OperationsF.11 Perform Earth Arrival Operations

C.3 Structural Dynamics Methods or mechanisms for separation and jettisons, providing structural integrity, and withstanding loads exerted during all activities and events for the iMars mission.


20


Artifact Example - Risk Matrix/TableRisk Description

RS.1.0 Insufficient Funding Given the very high cost of building and flying a crewed space mission to Mars and the nation's difficult economic challenges, there is a possibility that insufficient funds will be raised to design, build and fly the current mission. This could result in reduced mission scope, lower vehicle performance, and potentially program cancellation.

RS.1.6 Inability to Use Data Rights Given the lack of legal precedence in obtaining data rights for private human space missions, there is a possibility that the iMars program will be unable to legally utilize program (system, scientific, and crewmember) data for public entertainment and education resulting in reduced public engagement and support. This could lead to insufficient public, private, government, and international support and limit the impact of the iMars program need to inspire future trips to Mars.

RS.2.4 Government Collaboration Given the inconsistency of US government support for private space exploration, there is a possibility that the iMars program may not be able to reach agreements with NASA and other government agencies interested in space utilization, which could result in the ineffective use of government resources and knowledge of space flight development and operations. This could lead to schedule delays in accessing needed government facilities, resources, and knowledge sources for designing, building, integrating, testing, and operating the launch vehicle element, in-space element, ground and mission operations element, technology development, and communications network.

RS.3.0 Lack of Communication Coverage Throughout the Mission

Given the long distance between space element and control center, there is a possibility that communication will get interfered/lost resulting in delay or loss of data to be transferred between that two.This performance impact could result in delay in space operations or in worst case loss of crew.

RS.3.1 Unavailability of ECLSS Architecture for Integration by Launch Date

Given the lack of proven ECLSS technologies for long-duration space missions, there is a possibility that a mature ECLSS architecture will not be available for integration into the design of the in-space element in time to support the In-Space Element PDR resulting in schedule delays. This schedule impact could result in missing the launch window in January 2018.

RS.3.1 Long Lead Procurement & Integration of Launch Vehicle & In-space Element

Given that tight launch window, there is a possibility of missing it due to long lead procurement and integration of available heavy launch vehicle. This schedule impact could result in delay in mission operations.

RS.3.5 Inadequate Performance of In-Space TPS During Earth Re-entry

Given the high re-entry velocity of the in-space element upon re-entering the Earth's atmosphere during the return leg of the mission, there is a possibility that the in-space element thermal protection system is not capable of withstanding the extreme heat loads which will adversely affect the structural integrity of the in-space element. This risk could lead to break up of the in-space element and crew death during re-entry. An analysis was performed to show that PICA was the right material, but the thickness required can only be calculated with an analysis beyond the scope of a pre-phase A study. There is a risk that the thickness required is greater than 10 inches which is the current manufacturing limit of PICA TPS shields.

RS.3.5 Crew Safety During Earth Re-entry

Given the high re-entry velocity experienced during Earth re-entry, there is a possibility that the in-space element re-entry and landing systems may not adequately dissipate/absorb the energy of the in-space element resulting in higher than desired velocity upon water/land landing. This could result in crew injury or death upon landing.

RS.4.5 MMOD Threat to Crew Survival Given the lack of data on MMOD distributions encountered along the way to Mars, there is a possibility that the in-space element may be struck by MMOD with higher than expected momentum resulting in breaches of and damage to the in-space element structure and internal systems. This could lead to permanent damage to critical systems (power, thermal, ECLSS, data, GN&C, etc.) and potentially risk to crew health and survival.

RS.5.2 Uncertainty of Crew Psychological Effects

Given the extremely confined environment in which the crew must live during the entire 1.4-year mission, there is a possibility that the confined working and living environment could create unhealthy conditions for the crew. This could lead to the crew's increased susceptibility to illness and psychological stress, which may ultimately affect crew operational performance and mission success.

RS.5.2 Effects From Extended Exposure to Radiation Environment

Given the uncertainty in the knowledge of the space radiation environment between Earth and Mars, there is a possibility that the in-space element may not provide adequate radiation shielding leading to insufficient shielding from space radiation. This can result in overexposure of the crew to high levels of radiation and impact crew health and safety.

21


Commenting Tool

23


Review Board and Feedback Collection• Survey Questions

– 1.Have you participated in a traditional, document-based design review before (e.g., SRR, PDR, CDR, etc.)? (Response: Yes/No)

– 2.If you answered "yes" to question 2, please give a brief description of the review process (e.g., power point charts, formal documents, RIDs, etc.). (Response: Reviewer Text Input)

– 3.Have you ever participated in a model-based design review such as this one? (Response: Yes/No)

– 4.If you answered "yes" to question 4, please rate your experience participating in the review. (Response: 1- 5 Rating)

– 5.For this iMars MCR, how well did the kickoff presentation address the following? (Response: 1- 5 Rating)

– 6.Any particular thoughts you'd like to share with us regarding question 6?– 7.How well did the SYS 638 design team achieve the MCR success criteria? (Response: 1-

5 Rating) – 8.So how did this model-based review (MBR) compare to traditional document-based

reviews (DBR)? (Response: 1- 5 Rating) – 9.We want to make model-based systems engineering an effective means of doing design

and reviews. What can we do to make future model-based systems engineering reviews more effective and easier to become involved? (Response: Reviewer Text Input)

24


Lessons Learned• Tool Access

– Tool access is a pre-condition for a model-based review– Use of a cloud-based tool GREATLY simplifies the problem of universal access (platform independent, no

software to install). – Trying to do our research using a platform-based tool would have been infeasible given the diverse reviewer

group.

• Tool Knowledge– Most people are generally uncomfortable navigating a new tool (such as Innoslate) for the first time while

accomplishing the review process. » Some reviewers wanting to see outputs from the model that they didn’t have to navigate through (even though the majority

of the essential review elements were one page hyperlinked diagrams from the MCR walkthrough spreadsheet). » If Model-Based systems engineering reviews are going to thrive, standard tools, standard interfaces, standard products

(e.g. DoDAF) may help. » Absent the above: Reviewers will need some training to be comfortable navigating the tool to allow them to find and

interpret information.

• Context– Tools tend to excel at capturing detail, but can miss the big picture. – Reviewers (especially those from outside the project) need background and context to interpret review data

and products– Providing an overview briefing/video greatly aided reviewers in learning this context.

• Design Data– An integrated design spreadsheet was used to capture detailed mission analysis data. This tool was also

available for online review. However, again, this provided detail at the expense of context. – A more clear and concise way to present the overall effect of each trade against the baseline architecture

would have been a very useful addition to the spreadsheet approach. 25


Lessons Learned (cont’d)• “Review View”

– MBSE tools ofter many views of the system (requirements, assets, functions, etc.). – A “design review” view would be a helpful addition. – Could follow standard review entrance and exit criteria - tailorable by users - that would allow

model artifacts to be linked to it

• “Humanizing” the Review Process– A completely remote, model-based review where engineers and reviewers never meet makes

the project more abstract.– The real-time presentation given to the reviewers prior to the allowing them access to the

model for the review process was generally regarded as a good thing. Some reviewers that were not in attendance and could watch a playback video.

– This helped to “humanize” the review and provided a remote introduction between engineers and reviewers.

• Real-time Interaction– Some type of real-time forum is helpful to allow the reviewers to ask the design team

questions about the model and review products directly in real or near-real time, in addition to simply leaving comments. This system would allow other reviewers to view the question as well, along with the answer, would prevent the same questions from being asked by multiple reviewers.

– The iMars project team utilized an email distribution system through Google to allow the reviewers to email one email address that would alert the entire design team of their question. It was thought that this would be enough to satisfy any concerns or questions that the reviewers had. Apparently this was not the case per the survey results.

26


Observations and Conclusions• Observations

– Scalability » Given the novelty of powerful MBSE tools, there is little collective experience on conducting model-centric design

reviews for large projects» Additional research to characterize MBSE tool features and model-centric design review processes that enable

successfully scaling the approach documented in this study to much larger programs would be helpful

– Organizational Adoption» While NASA/ESA and other organizations do use MBSE tools and methods for some projects, the norm is the

traditional document-centric approach. • For example, NASA’s Jet Propulsion Lab (JPL) uses MBSE methods for some of its planetary exploration

missions. Other NASA centers use MBSE methods either in limited capacity or not at all for larger programs. » Identify a strategy for a particular agency to adopt MBSE methods and tools as the norm for future programs

accounting for initial capital and human resource investment, integration with other design tools and processes, and overcoming cultural barriers to adoption.

– Organizational Inertia» We like our documents. Some organizations exist to produce a specific document. Model-based Reviews (MBRs)

can threaten sacred cows.» Moves to MBRs requires organizations to change their current paradigms for reviews

• Conclusions– Model-based Reviews can work!– Model-based Reviews save a HUGE duplication of effort copy/pasting artifacts into Power Point slides– Many issues identified with the MBR are also present in traditional, document-based reviews (e.g. how

to navigate through the data)– As organizations migrate to MBSE, the move to MBR becomes a logical imperative.

» Failure to do so GREATLY reduces the project-level savings of investing in MBSE in the first place27

© 2014 spec innovations and teaching science and technology, inc. all rights reserved model-based...

Documents

spec innovations

teaching science

modelbased design review

modelbased ones

modelbased review mbr

rights reserved mbse

rights reserved example

model centric