edsp cp master test plan sit uat 11092011

5/11/2018 Edsp Cp Master Test Plan Sit Uat 11092011 - slidepdf.com

http://slidepdf.com/reader/full/edsp-cp-master-test-plan-sit-uat-11092011 1/16

Virginia Department of Social Services Master Test PlanEnterprise Change Management EDSP - CP

v 0.1 – 11/09/2011 of 16

Enterprise Delivery SystemProgram – Customer Portal

VDSS Master Test(SIT and UAT) Plan

Initial Draft Version 0.1

Project Name: Commonwealth of Virginia Department of SocialServices Enterprise Delivery System Program – Customer Portal

System Test Plan Prepared By: Stephanie Napper

Date ST Plan Prepared: November 10, 2011

Test Scripts Created By: DIS, Security, and Business

System Testing Performed By: Staff of DIS, Security, Business, andLDSS Staff




v 0.1 – 11/09/2011 of 16

Table of Contents

1. Introduction ......................................................................................... 4 1.1 Acronyms .......................................................................................................... 4 1.2 Definitions ......................................................................................................... 5 1.3 Purpose............................................................................................................. 5 1.4 Scope ................................................................................................................ 6 1.5 System Testing ................................................................................................. 6 1.6 Assumptions/Dependencies/Pre-Requisites/Risk ........................................... 11

2. Test Approach ................................................................................... 12 2.1 Test Approach ................................................................................................. 12

2.1.1 Requirement Testing................................................................................ 12 2.1.2 Regression Testing .................................................................................. 12 2.1.3 Interface Testing ...................................................................................... 12 2.1.4 Information Security Testing .................................................................... 13

3. System Testing Process .................................................................. 13 3.1 Test Entry/Exit Criteria .................................................................................... 13 3.2 Test Pass/Fail Criteria ..................................................................................... 13 3.3 Test Suspension/Resumption Criteria ............................................................. 14 3.4 Test Deliverables ............................................................................................ 14 3.5 Test Execution ................................................................................................ 14 3.6 Staff Responsibilities ....................................................................................... 10 3.7 Draft Schedule ................................................................................................ 15

4. Defect Tracking and Resolution Process........................................ 14 5. Communication ........................................Error! Bookmark not defined.

5.1 Communication Process ................................................................................. 16 6. Change Management ........................................................................ 16

6.1 Changes to Master Test Plan .......................................................................... 16 7. Plan Approvals .................................................................................. 16

7.1 Changes to Master Test Plan .......................................................................... 16




v 0.1 – 11/09/2011 of 16

Revision History

System Test Document: Revision HistoryVersion # Date Description Author

0.1 11/09/2011 Initial Draft Stephanie Napper

Approvals

Department Name and Title Approval Date




v 0.1 – 11/09/2011 of 16

1. Introduction

An examination of the Virginia Social Services System (VSSS) business model through business

process re-engineering (BPR) revealed several issues and operational challenges within the stateoffice and 120 local departments.

Minimal customer self-service capabilities, paper and manual based processes, and limitations oninformation sharing among social services activities and partners were identified as operational

challenges requiring a more progressive resolution.

It was determined that these concerns could best be addressed by leveraging of more currenttechnology. These concerns led to the inception of a new business process model; an efficient,

state of the art, enterprise-wide VSSS servicing customers holistically. The Virginia Department

of Social Services (VDSS) has entered into a partnership with Deloitte to create a web-enabled

system that will move VSSS to service integration, improved efficiency, and improved outcomesfor the citizens of Virginia.

1.1 Acronyms

The following acronyms may be found within this document.

Acronym Description

ACH Automated Clearing House

ACS Affiliated Computer Systems, Inc.

ADAPT Application Benefit Delivery Automation Project

BA Business Analyst

BPR Business Process Re-engineering

CP Customer Portal

DIS Division of Information Systems

EAL Enterprise Audit Log

EDSP Enterprise Delivery System Project

GIS Geographic Information System

IVR Interactive Voice Response

LDSS Local Department of Social Services

MMIS Medicaid Management Information System

RQM Rational Quality ManagerSOW Statement of Work

SIT System Integration Testing

SNAP Supplemental Nutritional Assistance Program

SPIDeR Systems Partnering in a Demographic Repository

TANF Temporary Assistance for Needy Families

UAT User Acceptance Testing

UCD Use Case Document




v 0.1 – 11/09/2011 of 16

Acronym Description

URD User Requirement Document

VaCMS Virginia Case Management System

VDSS Virginia Department of Social Services

1.2 Definitions

Pre-Condition- The state in which the application must be before the test case is run. This mayinclude pre-building test data prior to running a test.

Post Condition- The state in which the application must be after the test case is run. If the post

condition is met, the test passes.

Test Case- Specific test record to be utilized in a defined test scenario and script (e.g. specific

case/ component, known input, etc.).

Test Scenario- High-level description of a test to be conducted (e.g. set of conditions, variables,

positive/ negative scenarios, circumstance/ situation within which a case/ component is beingprocessed, etc.).

Test Script- The detailed steps/ instructions provided to the testers on how to operate theapplication and execute a test scenario to ensure it works according to the approved

requirements. It contains a sequence of events and expected results.

Application Requirement- The specific requirements the developers must program in order tomeet the overall requirements of the test scenario.

1.3 PurposeThe purpose of this Master Test Plan document is to define and outline processes, activities,

tools, and responsible parties for the System Integrated Testing (SIT), Security, and UserAcceptance Testing (UAT) of the Enterprise Delivery System Project - Customer Portal (EDSP –

CP).

This form of testing is to validate that Customer Portal, ADAPT and all interfaces meet

documented requirements as stated in the following documents:

Statement of Work (12/10/10)

Business and Requirement Document

Use Case Documents

Development Clarification Meeting Documents




v 0.1 – 11/09/2011 of 16

1.4 Scope

The System Integrated Testing (SIT) jointly performed by Deloitte, and the Division of

Information Systems (DIS) will be followed by User Acceptance Testing (UAT) performed by

Security and Business. The test schedule below outlines the two phases:

Release 1

Orientation and Training for UAT (Magic Team) November 8 – 9, 2011

SIT/MIT Developing of Test Scripts (11/18/2011)

SIT Training for testers (regular ongoing meetings)

Test Execution: MIT/SIT (November 21, 2011 – January 27, 2012)

UAT Developing Test Scripts (December 6 – 14, 2011)

UAT Building Cases in ADAPT (December 15, 2011 – January 12, 2012)

UAT Orientation for LDSS Testers (January 18, 2012)

UAT Introduction to Test Scripts and new ADAPT screens (January 19, 2012)

Test Execution : UAT - Business and Security (January 24 – March 1, 2012)

Release 2

Test Execution: SIT ( March 12 – May 4, 2012

Orientation and Introduction to Renew, Changes, and Check My Benefits for Testers(May 2 – 3, 2012)

Test Execution: Business and Security (May 7 – June 7, 2012)

1.5 System TestingThis System Test Plan document covers the methods of testing used to verify that the Enterprise

Delivery System Project – Customer Portal delivered by Deloitte meets the system requirementsoutlined in the Statement of Work, Requirement Documents, Use Case Documents, andDevelopment Clarification minutes.

This document describes the comprehensive approach to system testing, including specific areas

of the system that will be tested, areas that will not be tested, the roles of the individuals involvedin the testing, the documentation involved in testing, and the procedures for reporting and

retesting defects found during this phase of testing.

Testing of the system will be predicated upon the releases for this project.

Release 1: This phase of testing will focus on all aspects SIT, Security, and UAT for the“Screening”, “ Apply for Benefits”, and “ My Work Space” modules of the application for TANF,SNAP, Medical Assistance, and Energy Assistance. Testing for this deployment to include SIT,

Security, and UAT will be conducted from 11/21/11 through 3/1/12.

Release 2: This phase of testing will focus on all aspects; SIT, Security, and UAT for the

“Check My Benefits”, “ Report Changes”, and “ Renew My Benefits” modules of the applicationsfor SNAP, TANF, Medical Assistance, and Energy Assistance.




v 0.1 – 11/09/2011 of 16

MIT/SIT

Definition: MIT includes initial testing of the individual components/ modules

and their preliminary integration into the solution’s back-end framework. MIT

will be independent of the front-end solution and will focus on the back-endsolution and related components.

SIT includes initial testing of the individual components/ modules and their

preliminary integration into the solution’s back-end framework. MIT will be

independent of the front-end solution and will focus on the back-end solution and

related components.

SIT takes an optimistic approach to testing, because it expects to have few

problems with the individual components due to Unit Testing and MIT. The

strategy relies heavily on the component developers to perform the isolated unit

testing for their product before being released to MIT and then SIT. The goal of the strategy is to avoid redoing the testing done by the developers and initial MIT

testers and instead determine problems caused by the interaction of thecomponents in the environment.

SIT includes system to system and interface to interface testing. It includes thetesting of the integration of the front-end solution with the back-end solution. To

be more efficient and accurate, user-like workloads (e.g. for customers, for

workers) will be defined for creating realistic scenarios in exercising the

environment. This allows for confirmation that the integrated environment willwork as expected for the target customers/ workers.

Participants:

IT Test ManagerMary Disse

IT Requirements Manager

Iesha Tiller4 Business Analysts

Jan Ellison

Fernando MillerWillie McWhite

John Vaughan

Method: MIT/ SIT will conduct requirements-based testing focusing on thefollowing test scenarios and their associated requirements:

Security (e.g. including authentication, authorization, associations, EAL andSAMS Reports)

Apply For Assistance through Customer Portal (CP) Interface

o Including Reapply for assistance through CP




v 0.1 – 11/09/2011 of 16

Renew My Benefits through CP Interface

o Including Add person or program with renewal of benefits through CP

Report My Changes through CP Interface, including:

o Address change

o Add a person

o Remove a persono Add a program

o Begin/end pregnancy or disability

Check My Benefits through CP Interface (including Application Statuschecks)

File Clearance through My Workspace (MWS) and ADAPT

FIPS Verification through MWS (using GIS)

Reference Table Validations (ADAPT and CP)

Changes to ADAPT required to accommodate the above scenarios (e.g.including errors, screen help, alerts, inboxes, summary screens, reports, etc.).

Test scenario documents, requirements traceability matrices, screen mockups, designdocuments, and help files in the form of the CP/ADAPT VDSS Back-End Design

Document and supplemental documents/ data matrices will be provided to testers. The

Test Scripts which include steps the testers will follow to test the scenarios and theirrequirements will be developed by and provided to all testers.

Security

Definition: testing encompasses the back end of this project as it relates to both

citizens and staff of local departments of social services entering and accessing

secure information, to include creating accounts, strong passwords, resettingpasswords and access to information is provided as associated with client IDs..

Separate tests will be conducted on the SAMS CP functions with existing Security

Staff. An established SAMS regression test process will be utilize as part of these

tests. Separate test account for each worker role in CP will be tested. COV

Auditors will assist in conducting tests on the CP.

Participants

Test Manager

Matt Teasdale

Existing Security Staff 10 COV Auditors

Method: Security will conduct requirements-based testing.

Each tester will enter data and log the inputs as they make them for comparisons

with the audit logs.Testers will purposely attempt to enter values that should be rejected.

Testers will process file clearance actions to see how they are logged in EAL.




v 0.1 – 11/09/2011 of 16

Testers will add, update and delete accounts to see how the system responds and

updates to EAL.Testers will create a CP account and test the password reset options.

Separate tests will be conducted on the SAMS CP functions with existing Security

Staff.

UAT

Definition: Business UAT as are all components of testing has a primary

expectation to test with the goal of releasing to the users; both the citizens of the

Commonwealth and DSS staff an efficient new technology that will enhance theway we do business the first time out of the gate. To that extent UAT focuses on

the testing of business rules for SNAP, TANF, Medical Assistance, and Energy

Assistance utilized in the EDSP – CP and the ADAPT System to ensure that

programming of rules and logic result in accurate outcomes in line with the

Requirements Document.

UAT includes testing of navigational tools, password generation, access to clientinformation, system and screen flow, the accuracy of the passing of information

from the Customer Portal to ADAPT to include the filing of an application

utilizing the Spanish language, and the generation of reports in My Work Space.

UAT includes ease of use testing through the partnering with The Virginia

Department for the Blind and Vision Impaired.

Participants:

Co-Test ManagersStephanie Napper

Delores Veal3 Subjects Matter Experts (Regional Consultants who have been with the

project from the first JAD session)

Sherry Sinkler-CrawleyNan Foster

Cathy Trimble

2 Testers from local departments of social ServicesGail Kelley – Norfolk Department of Human Resources

Julie Chalupsky - Norfolk Department of Human Resources

Possible Testers from VDSS Business Operations Unit

Method: UAT will conduct requirements-based testing conducted in a test lab.

All testers will test from this lab. The lab environment will provide computers for

each tester. Orientation will be held for all testers to provide guidance for testerswho will develop test cases and test scripts. An additional orientation will be held

for all testers who will perform system testing. Testers developing scenarios and




v 0.1 – 11/09/2011 of 16

scripts will be provided access to Use Case Documents, hard copy of the VDSS

Back End Design Document, and the Requirement Document.

Testing of the Customer Portal; Screening, Apply for Benefits, My Work Space,

Filing Renewals, Reporting Changes, and Check My Benefits will include:

Accuracy of information on screens

Input of information

Expected results vs. Actual results

Testing of Requirements

Test to ensure they cannot get off the screen without a strong password

Test to ensure that only the identified individual(s) can access caseinformation

Testing of the ADAPT System to ensure information pushed from the Customer

Portal to ADAPT is correctly populated in the appropriate subsystems and that no

facet of eligibility determination and benefit calculation is automatically ran.

LDSS testers will have primary role in ensuring that screens other thanidentified screens have not changed.

LDSS tester will have primary role in ensuring that the system functionsthe way it did prior to the new application other than identified changes.

Testers will add program to both pending applications and ongoingcases.

Testers will add/delete a person to both pending applications andongoing cases.

1.6 Staff Responsibilities

VDSS/DIS/Security/Business staff will be responsible for creating the System Test Planand creating and maintaining System Test Case Scripts using the documents noted in

above sections. The assigned staff will be responsible for securing test documents and

securing production data used for testing.

Test Managers are responsible for the overall environment, operations, and process of

the unit’s system testing. Test managers will review and key defects into ClearQuest.

Test Managers will monitor the lifecycle of defects.

Testers are responsible for accurate and precise testing of test scripts and regression

testing as needed. Testers are also responsible for reporting defects to Test Managers.

Testers will maintain a daily count of test scripts tested to include pass/fail results.




v 0.1 – 11/09/2011 of 16

1.7 Assumptions/Dependencies/Pre-Requisites/RiskAssumptions

The following assumptions have been made for this plan:

Deloitte, DIS, Security, and Business will commit adequate resources to support

the testing effort as listed in the project schedule. All components of testing (Deloitte, SIT, Security, and Business)

All testing will be manually performed.

Testers, Test Managers, and Project Co-Leads will be made aware of all defects,new and outstanding.

Review of defects will be review by Test Managers to ensure the defect is valid,supporting documentation is provided to explain defect, and that an identical defect

has not been previously submitted.

Defects will be corrected in a timely manner to ensure the success of the projectschedule.

Regional and LDSS personnel participating in testing understand 100% of their

effort is required.

Deloitte, MIT/SIT, Security and UAT agree upon the severity definition for defectsidentified during testing.

Deloitte, MIT/SIT, Security and UAT will have access to defects created inClearQuest.

All defects will be entered in ClearQuest daily by 3:00 p.m.

Defects with a critical or high severity level will also be called in to the identifiedperson for Deloitte and DIS.

Dependencies and pre-requisites

The following dependencies and prerequisites exist:

Defects found during SIT have been resolved before UAT can begin.

All testing resources (computers, requirements, SOW) must be in place.

All test environments (ADAPT, SPIDeR, MMIS, VaCMS, Energy AssistanceAutomated Program, LDSS interface, etc) must be properly established

Test environment matches “real world”

Testers are experts in their own area

Testers from LDSS are knowledgeable of all benefit programs

All defects found during testing have been resolved to an agreed upon acceptablelevel.

Risks

The following risk may exist:

Resources not being available when needed to test

Users and/or developers trying to make changes to previously approved requirements

Findings on pre-existing production issues and requests for resolutions (potentialscope creep).

Non-CP related legacy system or interface changes impacting this project.




v 0.1 – 11/09/2011 of 16

2. Test Approach

2.1 Test Approach

System testing will be performed manually for SIT, Security, and UAT.

Test scripts will be executed using valid and invalid data to verify the expected results occur.

Each test script or test case will be created, tested, and documented as pass or fail. Test scriptsmay be combined into one document or maintained as individual documents with the goal of

entering all test scenarios and scripts in Quality Manager. The IT, Security, and Business Test

Managers should be abreast of all test scenarios and scripts.

Each test case/script will be traced to a Business Requirement. There may be rare situations

where a test can not be located in the Business Requirements because discrepancies may exist.

While the Rational Suite, including Rational RequisitePro, ClearQuest, and Quality Manager, are

the systems of record for the EDSP-CP project a back-up method will be implemented at the

beginning of the process. All requirements are loaded and managed in RequisitePro; all defects

must be logged/ tracked in ClearQuest, and all test items will be included in Quality Manager.This will ensure traceability and consistency throughout the entire project lifecycle.

Test Cases and Test Scripts are to be written and compiled on the agreed upon format forconsistency. Test Cases and Scripts will include a unique numbering identifier, the associated

requirement number(s), and a pithy description of the intent of the scenario and script. Paper

documents will be electronically filed in a determined folder on the R: / drive.

2.1.1 Requirement TestingEach applicable numbered requirement will be traced to a test script/case for traceabilityto validate each function properly works and meets the needs of the user.

2.1.2 Regression TestingMIT/SIT, Security, and UAT will each conduct regression testing. Regression testing

will be performed on defects introduced during system testing only. Regression testing

will also will include testing of existing legacy system functionality to verify that it was

not impacted unexpectantly when other aspects of the system/interface were changed to

meet project requirements.

2.1.3 Interface TestingMIT/SIT will test not only the individual components/requirements, but will also test the

interfaces between the CP and ADAPT/ MMIS/ Energy system/ LDAP/ SPIDeR/ EAL/

GIS, both by module and by end to end process. This includes the CP data, ESB

translations, VDSS Holding Tables, and legacy system databases. This is to ensure thatthe data entered into the CP is accurately transmitted and displayed in ADAPT. Also, it




v 0.1 – 11/09/2011 of 16

will test that the data in ADAPT/ MMIS/ Energy system is accurately transmitted and

displayed in the CP during the Renew My Benefits, Report My Changes and Check MyBenefits processes.

2.1.4 Information Security Testing

Information security testing will be performed differently because of its complexity andthe number of roles versus groups versus user types. Testing will consist of validating

system access and allowing of information to clients through the CP.

3. System Testing Process

3.1 Test Entry/Exit Criteria

System Testing will begin after tester training has been completed, test case scripts

documentation have been initiated and approved in draft, and system access and logonhave been obtained for the testers from SBE. All resources must be in place and all test

environments must be properly established and connections available.

System Testing will be considered complete when all planned test scripts have been

executed, all known system testing defects have been documented, tracked, and correcteddefects have been re-tested with no outstanding Critical or High defects or at the end of the

contract period, whichever comes first. A “critical” defect is a defect that prevents further

testing. A “high” defect” is a defect that that does not functions as expected/designed or

causes other functionality to fail to meet requirements. Otherwise, system testing will beconsidered complete on March 2, 2012 for Release 1 and June 8, 2012 for Release 2 per

the current Statement of Work Contract.

3.2 Test Pass/Fail Criteria

The pass/fail criteria for each test case/script is described by its expected results and will

be noted as pass or fail (with a defect number assigned), the test date, and tester name. If the expected results are received, the test will be marked as “pass”, if the expected results

are not received, the test will be marked as “fail”.

Upon defect correction, the test case/script will be retested and again noted as pass or fail

during retest. Each test case/script will continue to be retested until it passes or is deemedacceptable by the EDSP – CP Business Manager.




v 0.1 – 11/09/2011 of 16

3.3 Test Suspension/Resumption Criteria

If a problem is found that impedes the continuation of any phase of testing (50% or more

of the test cases can not be completed), System Testing will be suspended until appropriate

corrections are made and testing can resume. If this type of suspension is warranted, the

EDSP – CP Test Manager will immediately be notified.

3.4 Test Deliverables

The following items are work products that will be outputs of the testing process.

System Test Plan

System Test Case Scripts that includes a traceability matrix

3.5 Test Execution

System Testing will be performed in a test environment at VDSS by testers identified byDIS, Security, and Business. A stable URL will be provided by Deloitte. Testers will

have access to the unit Test Manager on a daily basis (in person, phone or email) during

the testing period to discuss issues and questions.

Workstations will consist of the hardware setup used by VDSS within the department and

access to the test environment will be secure and only accessed by the individuals taskedwith this activity.

Testing will be executed manually by using test case scripts that will be captured in MS

WORD or MS Excel Spreadsheets. Each script will contain the object of the test, the

conditions to perform the test, the expected results, the actual results received, the tester’sname, the date of test, and the execution status of the test (pass/fail). The execution of test

scripts are governed solely by the tester assigned and can be executed in any order deemednecessary by the tester.

The test database will consist of production and test data; therefore, data will be handled

securely:

Production data will not be accessed nor viewed by anyone in Deloitte orVDSS/DIS.

4. Defect Tracking and Resolution Process

Any defects encountered will be reported by the testers at the end of the day, with the exception

of Critical or High defects and entered directly into ClearQuest. Critical and High defects are to

be reported to the Test Manager immediately. Defect tracking and resolution will occur at the

test script level. System testers will execute the test scripts and record within the script the stepat which the script failed. If a script fails, it will be recorded within the defect tracking system

and attachments will be supplied as evidence.




v 0.1 – 11/09/2011 of 16

The following table describes the Defect Severity levels:

Severity Level Definition

Critical A defect that prevents further testing of the system,

High A defect that does not function as expected/designed or causes

other functionality to fail to meet requirements.

Medium A defect which does not conform to standards and conventions.

The application or system may function properly, but the actual

test results are not the same as expected results. (Should be

evaluated on a defect by defect basis.)

Low Cosmetic defects which does not affect the functionality of the

application or system.

It is expected that 100% of all defects be resolved.

ClearQuest statuses will be used to track the defect through the testing and retesting processes.

This status will be used to meet project reporting needs.

5. Schedule

5.1 Draft Schedule

The following dates are estimates based on the project plan and deadline date for completion of

system testing.

SIT Release 1 Completed: January 27, 2012

Security and UAT Release 1 Completed: March 2, 2012

SIT Release 2 Completed: May 4, 2012

Security and UAT Release 2 Complete: June 8, 2012




v 0.1 – 11/09/2011 of 16

5.2 Communication ProcessA weekly communications meeting with Deloitte, DIS, Security and Business will be held

to discuss status of the system testing effort, status of interface and web services, and

outstanding items that affect system testing.

Deloitte should also be available to for additional meetings and phone calls as needed.

Defects identified by a VDSS team member that are determined to be related to Deloitterequirements will be submitted to the team’s Test Manager who will submit the defect tothe Deloitte PM in ClearQuest. The VDSS Tester will notify the appropriate Test

Manager, and the Test Manager will notify the Deloitte PM by email of the entry inClearQuest.

Such defects will be linked/ traced to the Deloitte Requirements as entered in RequisitePro.

The Deloitte PM will then assign the defect to the appropriate Deloitte technical lead forresolution. Upon resolution, the Deloitte PM will re-assign to the IT Test Manager for any

required review or retesting.

Deloitte identified defects that are determined to be related to VDSS requirements will beassigned by the Deloitte team member to the appropriate Test Manager in ClearQuest. The

Deloitte team member will notify the Deloitte PM, and the Deloitte PM will notify the IT

Test Manager by email of the entry in ClearQuest. The IT Test Manager will work withthe Technical Leads to assign the defect to the appropriate DIS team member for

resolution. Upon resolution, the IT Test Manager will re-assign to the Deloitte PM for any

required review or retesting.

6. Change Management

6.1 Changes to Master Test PlanChanges to the Master Test Plan will be made by Stephanie Napper or Delores Veal, Co-TestManagers. Test Managers for SIT and Security should submit requested changes via email.

7. Plan Approvals

7.1 Changes to Master Test PlanThe Master Test Plan and any changes will be approved by the Project Co-Leads.