© grant thornton llp. all rights reserved. independent verification and validation ensuring project...

© Grant Thornton LLP. All rights reserved.

Independent Verification and Validation

Ensuring Project Success

NASACT

August 24, 2015


Speakers

Cynthia Dannels, Deputy State Treasurer, SC State Treasurers Office

Tony Hernandez, Principal, Grant Thornton, LLP

2


Agenda

• What is Independent Verification and Validation?

• The business case for independent oversight

• Scope of independent oversight

• Elements of effective oversight

• Case Study - South Carolina State Treasurers Office

• Question and answer

3


What is Independent Verification & Validation?

Definition

Independent Verification and Validation (IV&V) is a process of employing rigorous methodologies for evaluating the correctness and quality of a project throughout the project lifecycle. Done correctly, IV&V is conducted by a third party organization not involved in the development of the product.

4


What is Independent Verification & Validation?

Key Considerations

• Verification – "Did I build the thing right?"

• Validation – "Did what we build work as expected?

• Accreditation – "Should it be used?"

• Realization – "Are we getting the benefits we expected?"

• There is also an underlying implicit principle, and its key

question: Credibility – Should it be trusted?

5


The Business Case for Independent Oversight

Business Drivers

• Public sector technology projects are getting larger and more complex, project successes and failures are increasingly more public, and overall projects costs are increasing. This is especially true of large, complex IT acquisitions.

• While the adoption of formal project management methodologies and techniques is becoming more common, many public sector technology projects continue to fail or deliver solutions that are late and over budget.

6



Critical Success (and Failure) Factors

The reasons why technology projects fail are many and varied, but some root causes appear consistently in troubled projects:

– Lack of a well-defined business case;

– Poorly defined or constantly changing requirements;

– A hurried or badly designed procurement that leads to the wrong vendor being chosen or to a badly written contract;

– The inability to make timely, fact-based decisions; and

– Inadequate skills and experience of government and vendor staff.

7



Critical Success (and Failure) Factors (cont'd)

It is well accepted in the software development industry that the cost of fixing a defect increases exponentially as a project moves through the development lifecycle.

However, this principle applies equally well to stages of the project that occur long before software development starts. In particular:

• Errors or omissions in the project business case can lead to the wrong solution being procured, so all the project budget may be wasted;

• Errors in the procurement can cause a project to be effectively doomed from the moment the contract is signed;

• Weaknesses in project governance and decision-making can seal a project's fate regardless of the quality of project team efforts.

8



How Independent Oversight Can Help

When the above types of issues impact projects, it is often because there was a particular element missing from the project governance structure. This element is a party with:

– The experience and foresight to anticipate the problems that could occur in the future;

– The ability and authority to communicate these problems and their consequences to executive management;

– The expertise to recommend a practical course of action to avoid the future problems; and

– The trust of executive management in the objectivity and independence of the recommendations.

An independent oversight consultant can fulfill this role, and can provide project leadership with practical, unbiased advice that can help projects avoid major problems and improve the likelihood of project success.

9



How Oversight Should Pay for Itself

Independent oversight is analogous to preventative care in the medical industry – a relatively small investment up front can reduce the likelihood of significant expenses later on.

The level of investment in oversight should be a function of the size, complexity and criticality to the mission of the project. An industry average for oversight/Independent Validation & Verification (IV&V) services ranges from 5 - 10% of total project development and implementation cost. Similarly, the cost of IV&V is a function of the size, risk and complexity of a project.

10


The Scope of Independent Oversight

Full Project Lifecycle

Issues and risks that impact project success can occur very early in the project lifecycle, and go well beyond the traditional software development activities that are the focus of standards such as the IEEE 1012 standard for IV&V.

For this reason, the scope of independent oversight services should include the full project lifecycle, from initial project concept through sustainment in production operations.

11



Oversight Categories

• Governance – To assess the level of executive stakeholder awareness, buy-in and support, and the willingness and ability to make timely, fact-based decisions.

• Project Management – To determine if the project is planned and managed in a fashion that effectively defines and controls scope, schedule, budget and resources.

• Risk and Issue Management – To determine whether risks and issues are accurately and comprehensively identified and managed. Also to determine whether the culture of the organization supports proactive and transparent communication of risks and issues.

• Quality Management – To assess the activities involved in quality planning, implementation of the quality plan, and the quality assurance functions within the project.

12



Oversight Categories (cont'd)

• Organizational Change and Training – To determine whether internal and external stakeholders are being prepared for – and will be ready for – the impending changes.

• Requirements Management – To assess the requirements management plan and validate that requirements management activities are implemented in accordance with the requirements management plan.

• Procurement and Contract Management – To determine whether procurement activities enable the best possible vendor selection and whether resulting contracts are managed in a fashion that reinforces quality and accountability.

• Solution Development and Implementation –To validate that solutions are being designed, developed and implemented in accordance with approved plans and designs.

• Maintenance and Operations – To determine whether the project is conducting maintenance and operations activities according to plan and service agreements.

13


Elements of Effective Oversight

Success Factors and Approach

Oversight success factor High value oversight approach

The experience and foresight to anticipate the problems that could occur in the future.

• Be predictive, not reactive.• Leverage experience to foresee the future consequences

of current decisions and actions.• Findings must be factual and evidence-based.

The ability and authority to communicate these problems and their consequences to executive management.

• Direct line of communication to executive management.• Translate problems and recommendations into language

that is meaningful and relevant to decision-makers.

The expertise to recommend a practical course of action to avoid the future problems.

• Identify pragmatic, actionable solutions to problems.• Be part of the solution, not just an identifier of problems.

The trust of executive management in the objectivity and independence of the recommendations.

• Independence in fact and in appearance.• Integrity and courage of conviction to state the facts, even

(especially) if they are unpopular or uncomfortable.

14



Communications Planning

• Communication tools

– Initial review reports

– Status reports

– Briefings

– Dashboards

– Assessments

– Final review reports

– Close-out reports

15



Risk Identification and Management

• Risk quantification and reporting

– Risk identification

– Risk analysis and assessment

– Risk rating and ranking

– Risk monitoring and control

– Risk mitigation, escalation and resolution

– Risk reporting

… Enables pro-active decision-making!

16


Elements of effective oversight

Key considerations

• Treat the root cause, not just the symptoms.

• Identify pragmatic, actionable solutions to problems.

• Be part of the solution, not just an identifier of problems.

17


Case Study

South Carolina State Treasurers Office

Communication toolsOverview of the State Treasurers Office NVEST and DEBT Projects

– Retire the STO's legacy Investment Management System and Debt Management System

– Implement SAP Treasury Management modules for investments (NVEST) and debt management (DEBT)

– NVEST and DEBT Project Go Lives Scheduled for September 2015

18


Case Study


Communication toolsWhy did you decide to engage a vendor for Independent Verification & Validation services for your project?

19


Case Study


Communication toolsWhat are some of the barriers to bringing in a vendor for Independent Verification & Validation services?

20


Case Study


Communication toolsWhat are your expectations of the Independent Verification & Validation vendor?

21


Case Study


Communication toolsWhat are some of the important considerations for selecting and managing an Independent Verification & Validation services vendor?

22


Case Study


Communication toolsCan you tell us about the benefits you've seen from the use of Independent Verification & Validation services? Challenges?

23


Independent Verification & Validation


Communication toolsWhat recommendations would you have to share with us?

24


Independent Verification & Validation


Communication toolsWhat trends have you seen related to Independent Verification & Validation services?

25


Questions?

26


Contact Information

Cynthia Dannels, Deputy State TreasurerT 803.734.2655

E [email protected]

Anthony Hernandez, PrincipalT 850.201.7288E [email protected]

Julie Batchelor, DirectorT 919.881.5930E [email protected]

27

mailto:[email protected]

mailto:[email protected]

© grant thornton llp. all rights reserved. independent verification and validation ensuring project...

Documents

grant thornton llp

project business case

project budget

project lifecycle

project successes

project governance

project moves

project success nasact