wheatcraft hill.lou terry

Developing Requirements for Constellation’s Next Generation Space Suit

Presented at PM Challenge 2010Presenter's:Terry Hill, NASA/JSC Lou Wheatcraft, Compliance Automation

February 2010

Used with Permission

Overview Part 1: The Basics

Risk and Requirements Requirement Validation

Part 2: Application Case Study CxP Suit Requirement Development Process Results CxP Suit Continuing Requirement Management

Process What we could have done better

National Aeronautics and Space Administration 2

WHAT’S COMING NEXTRisk and Requirements


NASA OIG “NASA must be vigilant in its process of establishing and

validating project requirements.” “Program risks increase when NASA awards contracts

before developing a sound business case and clearly defining requirements;” Placing “the project at risk of significant cost overruns, schedule

delays, and performance shortfalls.”

“Effective risk management, safety, and mission assurance controls are key to supporting robust and reliable operations in the context of very challenging launch and mission schedules.”


NASA’s Most Serious Management and Performance

Challenges – Nov 2008

GAO “The start of product development represents the point at which

program managers make a commitment to provide a product that will perform as required and be delivered on time and within estimated costs.”

“Programs are more likely to succeed if program managers are able to achieve a match between user needs, which eventually become requirements, and resources (technology, design and production knowledge, money, and time) at the start of product development.”

“Conversely, if they do not match requirements with resources, cost overruns and schedule delays are likely to occur, reducing the organization’s buying power in other areas.”


GAO-03-598

Effect Of Requirements Definition Investment On Program Costs


20

80

60

40

160

140

120

100

200

180

5 10 15 20 25 30

OMV

GRO 78

GALLTDRSS

CEN HST

EUVE/EP

GOES I-M

ACT

MARSMAG

SEASATUARS

DESMM

ERB 77

LAND 76

IRAS

TETH

STS LAND 78

GRO 82ERB 80

VOYAGER

ULYSSESPION/VEN

IUE

ISEE

COBE

HEA

Target Total CostRequirements Definition and Preliminary Design

Targ

et C

ost

Actu

al –

Targ

et C

ost

Why are you running so fast when you don’t know where you are going?

German proverb


Cost to fix requirement defects

Requirements Phase

Design Phase

CodingPhase

Development Testing

Acceptance Testing

Operations

40

30

70

10

3

15

1000

40

61

1,000-

40-

10-

60-

50-

30-

20-

0-

Importance of Best Requirement Practices on Project Success “The companies using best requirements practices will estimate a

project at $3 million and better than half the time will spend $3 million on that project. Including all failures, scope creep, and mistakes across the entire portfolio of projects, this group will spend, on average, $3.63 million per project.”

“The companies using poor requirements practices will estimate a project at $3 million and will be on budget less than 20% of the time. 50% of time, the overrun on the project both in time and budget will be massive. Across the entire portfolio of successes and failures, this company with poor requirements practices will (on average) pay $5.87 million per project.”

2008 Study by Keith Ellis, IAG Consulting of 100 companies with projects in excess of $250,000

Setting Yourself Up for Failure Project success is “improbable” for 68% of the companies Ellis

studied

“Projects might succeed – but not by design. Based on the competencies present, these companies are statistically unlikely to have a successful project.”

While these companies indicated they recognized that requirements are important to project success, they still failed to take effective actions to insure a good set of requirements.

By doing so, they tripled their chances of project failure


What Needs to be Done “Organizations understand conceptually that

requirements are important, but do not internalize this understanding and change their behavior as a result.

The most successful of companies do not view requirements as a document which either existed or didn’t at the beginning of a project, they view it as a process of requirements discovery.

Only companies that focus on both the process and the deliverables are consistently successful at changing project success rates.”


No Surprises


People who write bad requirementsshould not be surprisedwhen they get bad products…

but they always are.

Ivy Hooks

http://www.sxc.hu/browse.phtml?f=download&id=1084632�

Words of Wisdom


“Putting forth the same effort, or using the same approach, then expecting different results is...insanity”

- Charles Bolden, NASA Administrator, July 2009

A Winning Product Delivers what’s needed Within budget Within schedule With desired quality


Risk: Anything that can prevent you from delivering a winning product!

WHAT’S COMING NEXTRequirement Validation


Requirement Validation vs. Verification Requirement validation

confirms the completeness and correctness of the requirements

Starts with first requirement and continues through life cycle

Verification confirms that the designed and built product meets the requirements

Done after design and build


Verification makes sure you built it right.

Requirement validation makes sure you are

building the right thing.

Requirement Validation Helps ensure our requirements are:

Needed, verifiable, achievable Clear, concise Correct, consistent, and complete

Two types Continuous Discrete


Continuous Validation Process Requirement Writer:

Write requirement and attributes Check requirements against standards Submit to gatekeeper for review in “chunks”

Gatekeeper (person or inspection points) One or more experts Reviews requirements against standards Accepts defect-free requirements for input to

database or document Return requirements to author if defects found


Who Does Requirement Validation?


Writers

Developers

Managers

Reviewers

Everyone is accountable

Continuous Validation Process Continuous validation holds everyone

responsible Requires standards and checklists Requires training Management has to enforce discipline and

accountability Benefits

Stops the creation of BIG bad documents Most effective way to realize process improvement Reduces time for milestone reviews Prevents lost time due to rework


Discrete Validation Process Key milestone that requires

time and resources Formal process

Complete document

Involves a wide range of stakeholders

Requires standards and feedback mechanisms

Requires training

Management has to ensure responsiveness

SRR results in a requirement baseline

Effective IF The right people are involved

The products are ready for review

The participants know what to do

Management ensures compliance


Scope

System Requirement Review (SRR)


SRR

PDRCDR

TRR

MCRDesign

Verification

Operations

Upgrade/Maintain

Requirements

SDR

Manufactureor Code

MCR: Mission Concept Review

SRR: System Requirements Review

SDR: System Definition Review

PDR: Preliminary Design Review

CDR: Critical Design Review

TRR: Test Readiness Review

Baseline requirements

Assess feasibility

Set expectations

WHAT’S COMING NEXT

CxP Suit Element Requirement Development Process


Background: Case Study Shuttle / ISS Extra Mobility Unit (EMU)

Shuttle was designed to be a shirt sleeve environment not requiring space suits of any kind.

EMU developed after Shuttle was designed to address a late in design possible failure scenario with the Shuttle –risk that payload bad doors would not close and would have to do it manually. Suit dimensions were driven by Shuttle hatch sizes.

Many, many requirements and later functionality of the suit were not addressed or defined early in the process.

The Shuttle EMU was later augmented and recertified to work at and around the ISS to perform station assembly and repair.


Shuttle/ISS EMU Case Study –Design and Development Cycles


Pre-declared Cert.

Initial Cert.

Re-Certification and Ops Con & Requirements

evolved

Background: EMU Case Study Lessons Learned Any little design change will take you into some

level of re-certification. Always larger and more expensive and takes longer

than ever planned. The better the ConOps is thought through in the

beginning allows the writing of better and more comprehensive requirements eliminating many re-certification cycles later and thus saving Life Cycle dollars.


Background: CxP Suit Element Requirement Development Challenge The Task

The team was asked to develop a CxP Initial Capability (and Lunar requirements when common hardware would be used) requirement set for the Space Suit Element in time to support the CxP Suit Element SRR and for release with the RFP for a prime contractor in mid-fall.

Schedule Very Short Schedule – 3 Months from initiation of

requirements generation to Major Project Milestone review and 5 months to baseline set of functional requirements.


Background: CxP Suit Element Requirement Development Challenge The Philosophy

Learn from past projects mistakes in how and when requirement are written

Clean sheet approach to developing a space suit and writing the requirements

Exercise the text book methodology of Systems Engineering and Requirement generation in a NASA project

Produce quality requirements that are verifiable in a cost effective manner that address the functionality defined in the CxP EVA System Operations Concept and EVA Systems Architecture documents.


Background: CxP Suit Requirement Development Schedule Planned Schedule for 2007:

May 31-Jun 5 – Requirement Training and Kick-off Jun 1-22 – Suit Element Requirements Generation Activities June 25-29 – Suit Element SRR Doc(s) review July 2-6 – Suit Element SRR Doc(s) update & SRR final prep. July 10 – Suit & Vehicle Interface Elements SRR Kick-off July 10-20 – Suit/VI Element SRR Doc review & RID submittal July 22-Aug. 7 – Suit/VI Element SRR Panels and Boards. Aug. 9-Oct 20 – Close SRR Actions and update ERD Oct. 23 – Suit ERD for Baselining at EVA PCB. Oct. 29 – Suit ERD rev. A draft submitted to EVA CM for pre-blackout CSSS

Tech. Library drop for Prime contract RFP release


* Final outcome, Element SRR slipped one week to ensure quality of products where ready for review. Review revealed products were ready and of the appropriate fidelity by EVA project management.

Background: CxP Suit Requirement DevelopmentApproach: Co-located the team off-site in a conference

room facility to enable a concentrated effort – this reduces the risk of day-to-day distractions which is a risk to product quality and schedule.

Provided task training in requirement development and writing processes – this reduces the risk to quality.

Reduce risk of requirement development by contracting out the training and using consultation which provided standard training as provided to CxP, and an independent “fresh set of eyes” on how requirements could be interpreted and implemented – this reduces the risk to quality.

Used CRADLE tool to develop requirements prior to baseline. Post baseline, the requirements were managed out of CRADLE, but draft revisions were handled outside of CRADLE until change approval.

Ground rules Requirements meet the criteria for good

requirements in the Checklist for Good Requirements,

Rationale for each requirement, no copying parent requirement with a noun change,

All requirements verifiable, clear, concise and if it can be interpreted in more than one way, it is not ready for acceptance.



Putting Requirement Risk in the Proper Perspective Not to put too much pressure on you….

The Requirements Document is probably the single most influential piece of paper that we have control over in the entire Constellation Program.

This is our chance to make sure that we are asking for what we really want. Let’s get it right.

This is a big, fat, hairy deal. If we don’t get this right, folks 20 years from now will be shaking their heads and saying, “What were those yahoos thinking?”

I’ll be around and don’t want to go to that meeting.

CxP EVA Suit PGS Team Requirement Kickoff Meeting 5/2007


Suit Requirement Development Process

Subsystem Teams Responsible for the drafting & modifying requirements Four independent teams

Suit Element-level (integrated across subsystems) Subsystems

Pressure Garment, Crew Survival, Power-Comm & Avionics, Life Support, (Ground Support Equipment team was after the initial requirement generation effort)

Each team consisted of: Functional Lead NASA Subject Matter Experts (SMEs) “Core” Scrub team

Teams given training during kick off meeting Briefed in the Goal, Schedule and Process Checklist for Good Requirements



Scrub Team “Core” Scrub Team:

SE&I representative Compliance Automation representative – independent “goodness

assessment”, and guidance in requirement development. Tech Writer CRADLE Operator

Review requirements from the Subsystem Teams Edit & “clean” Requirements based on Checklist for Good Requirements If they could “fix” requirement & verify the intent was understood, would do

so & pass onto the CRADLE Board If intent not understood and the requirement needed more work,

representatives from appropriate Requirement Team were called in for clarification.

Kept list of common defects and briefed Requirement Teams daily


CRADLE Board Had three functions:

Determined if requirements where technically traceable and appropriate

Determined if requirements where technically appropriate and the correct mission phases applied

And determine any outstanding issues were resolved as the CRADLE Board was used as a dry-run for final approval.

Base Team: SE&I representative Subsystem representative or subject matter expert

Secondary Team: Project Management Subsystem Leads

Focus Meetings a.k.a. Offline Resolves outstanding issues Membership as required


Approval Board Function: Team Consensus/Final Approval for requirement to be

included in ERD Base Team:

SE&I representative Project Management Subsystem Leads CRADLE operation Subject matter expert

If discussions proceeded for longer than 15 minutes, the requirement and discussion were sent to a Focus Meeting for resolution. The idea was that if a requirement was not clear enough to convey the intent and importance in 15 minutes of discussion, then it was not written correctly.

Focus Meetings a.k.a. Offline Resolves outstanding issues Membership as required

WHAT’S COMING NEXTResults


Potential bidders for the development of the Suit Element stated that the ERD was:

The JSC Engineering Directorate Crew and Thermal Systems Division Chief was also very impressed with the quality of the Suit ERD, saying:

Results in Project Reviews For the Suit ERD SRR, a ratio of 0.38 Review Item Descriptions

(RIDs) were received per requirement. In comparison, the parent document had a 2.94 RID/requirement ratio at

its SRR.


“… the most comprehensive and of the highest quality they ever remember seeing.”

“I can't say enough about how amazed I am by this set of requirements documents. As far as I know, no other Cx project has allocated and decomposed anywhere near to this level of depth. You are the first. I have also never seen anything like these from previous programs.”

WHAT’S COMING NEXT

CxP Suit Continuing Requirement Management Process


Post-baseline Requirement Development and Validation As a result of the extremely compressed requirement development

schedule, there remained several areas that needed more work and issues that needed to be resolved prior to preliminary design taking place.

These open areas included: resolving TBDs/TBRs in the requirement set, finishing the verification requirements, defining the internal interfaces and maturing applicable interface requirements and adding Ground Support Equipment (GSE) requirements to the ERD.

Therefore there was requirement development and maturation process needed to continue this process.



Requirement Review Process Post-ERD Baseline

Requirement Review Process Post-ERD Baseline: SEIWG The SEIWG (Systems Engineering and Integration Working Group)

Meets once a week to discuss new and existing requirements and/or requirements issues, such as TBDs/TBRs, action items, allocations, etc.,

Serves as the SE&I pre-coordination for the Suit Control Board.

SEIWG Procedure Step 1:

All potential changes must be submitted via a change request form.

Step 2: Scheduled SEIWG meeting to present/discuss change and any final modification

of the draft FROM/TO language.


Suit Control Board Functions:

Detailed review of proposed FROM/TO language to go into the ERD. A minimum of a 2 business day review by board members

prior to the Suit Control Board meeting.

Functions as the official forum of stakeholder review, concurrence or rejection

Functions as the final approval gate for base-lined requirements.

Note: Since Baselineing of ERD the SCB closed approximately 180 TBD/Rs prior to levying the document on the prime contractor.

WHAT’S COMING NEXTWrap up


What we could have done better Challenging schedule resulted in a lot of open work post baselining

of ERD. Large number of TBDs/TBRs

Interfaces identification and definition incomplete Some of the external interfaces just didn’t exist due to sister projects were

evolving in parallel and at times without the same rigor demonstrated in the Suit Element effort.

Traceability incomplete

Verification requirements

GSE requirements

Difficult to keep requirements at the right level Some requirements may not have been needed

Some requirements reflected or assumed a design where NASA was clear there was only one desirable functional solution.



Parting Thoughts Address Requirement Risk at the beginning of

your project Develop a formal requirement development

process that includes continuous requirement validation

Include continuous requirement validation into your requirement management process

Train your team Enforce the process Allocate the time and resources needed to do

the job right – the first time!!

Presenter Biographies



Terry HillNASA/JSC Constellation Space Suit System Engineering Project Manager

Terry Hill is NASA’s Johnson Space Center’s Engineering Project Manager and deputy CxP EVA SuitLead for the CxP Suit Element, responsible for the development of the functional, performance, andquality requirements and preliminary design of NASA’s next generation space suit system.

While at NASA ,Terry has worked on projects and programsspanning verification of ISS navigation software, Shuttle DesignTest Objectives (DTO) and back room mission support, X-38 CrewReturn Vehicle navigation algorithm development, Space LaunchInitiative technology development, Orbital Space Plane projectoffice ISS-Prime integration, STS-107 Return to Flight Tile Repaircapability development, to Constellation Program Space SuitSystem leadership.

In leading the CxP Suit Element engineering team, Terry hasfacilitated the development of system requirements for space suitdevelopment and a clean-sheet design approach that has widelyrecognized within and outside of NASA.

Terry has a BS in Aerospace Engineering and a MS in Guidance, Navigation & Control Theory with aminor in Orbital Mechanics and Mathematics from the University of Texas at Austin.He began his career at NASA while working on his masters thesis project in developing banks of simplified Kalman filters integrated into an artificial neural network to obtain an optimal state solution for precision landing on Mars.


Lou WheatcraftCompliance Automation – Senior Consultant Trainer

Lou Wheatcraft is a senior consultant/instructor for Compliance Automation, who has over 40 years experience in the aerospace industry, including 22 years in the United States Air Force. Lou has taught over 120 seminars in requirement development and management for NASA’s APPEL Program and industry over the past nine years. He has worked with, and provided intact team training and consultation to multiple NASA project teams at many of the NASA Centers.

Lou has had articles published in the International Council of Systems Engineering (INCOSE) INSIGHT magazine and in DoD’s magazine, CrossTalk. Lou has made presentations at NASA’s PM Challenge, INCOSE’s International Symposium, and at the local Project Management Institute (PMI) Chapter Meetings.

Lou has a BS degree in Electrical Engineering, an MA degree in Computer Information Systems, an MS degree in Environmental Management, and has completed the course work for an MS degree in Studies of the Future.

Lou is a member of INCOSE, co-chair of the INCOSE Requirements Working Group, a member of PMI, the Software Engineering Institute, the World Futures Society, and the National Honor Society of Pi Alpha Alpha. Lou is the recipient of NASA’s Silver Snoopy Award and Public Service Medal and was nominated for the Rotary Stellar Award for his significant contributions to the Nation’s Space Program.

wheatcraft hill.lou terry

Technology