for... · web viewapplication penetration testing ... design - word, excel, visio4. unit testing...

ASPEN M&O RFPQuestions and Answers

# Page Question Type Question Response

1

40 Process

As mentioned in the RFP "Offeror's must fully describe how they will be fully operational by July 1, 2015."

We assume that the vendor will be provided a 3 months’ time period to complete the transition phase. (i.e. starting from award start date of 03/31/2015 to 01/07/2015). Please confirm.

The vendor will be provided the period of time from contract award start date through June 30, 2015 to complete the transition phase. It is anticipated that this will be three months.

2

40 Scope

As mentioned in the RFP " How it will confirm it has transitioned all assets and materials needed to continue systems operations."

Considering the above Statement, does the State expects the vendor to perform any mandatory activity like: Operational readiness testing (ORT) to ensure Operation Readiness & Business Continuity?

The State will perform a readiness assessment with the vendor near June 1, 2015 in order to validate Operational preparedness.

3

General Other

We assume that the vendor will be provided with the existing test artifacts (Manual/automation test scripts, test results, etc.) which will help to bring in reusability during the M&O and enhancement testing phase. Please confirm.

The vendor will have available all existing test artifacts.

1



4

58 Administrative

As part of the Staffing Experience mentioned in the RFP, the Quality Assurance Testing Analyst shall be responsible for “Development of automated test scripts utilizing Commercial-Off-the-Shelf (COTS) tools such as HP Quick Test Pro; and Development of automated load testing scripts utilizing COTS tools such as HP Load Runner"

Will the State agree for multiple resume's for the QA position (more than 2 resources) 1. With manual testing Exp. & With QTP experience (Automation) 2. With HP load runner experience (Performance Tester). As availability of resources having both manual/automation and performance/load testing experience is scarce. Hence we may have to look for different profiles for each of the above asks. Please confirm.

The State will agree to multiple resumes for the QA position.

5

38 Cost

We assume that 2 tester mentioned as part of open positions for the Quality Assurance Testing Analysts are only for functional testing requirement (Manual & Automation). As this RFP requires other testing like Performance testing, Security testing - We assume that we will require additional QA staffing. Please confirm

The vendor should propose resources to meet the RFP requirements.

6General Cost

Is the State willing to accept any open source (free of cost) tool like selenium for automation testing?

The vendor should be prepared to use existing tools however HSD is open to recommendations of changes or additions to these tools.

2



7

130 Other

As mentioned in the RFP "The Contractor shall create and maintain manual test scripts when the automated tool cannot be utilized allowing common areas, such as interfaces that receive files from trading partners to be regression tested."

Do we have any analysis for the coverage of automation v/s manual scripts for the existing regression test suite, if so, please provide details?

There are approximately 81 QTP scripts written for ASPEN functionality that are used in automated regression testing for each release. The State would like to increase the number and usage of these in the future. There are hundreds of manual scripts that can be used based on the functionality being tested. We do not have a current analysis of the coverage of automation v/s manual.

8

36 Scope

The scope of testing mentioned in the RFP describes the vendor to perform the following testing types: Testing will include:a. Unit/Integration Testingb. System Testing (i.e. Quality Assurance Testing)c. Regression Testingd. Performance and Load Testinge. Security Testingf. Disaster Recovery/Business Continuity Testing

Will the vendor be responsible to perform any penetration or network based security testing or is the scope of security testing limited to role-based security validation. Please confirm.

HSD conducts in-house as well as independent security scanning and testing on a regular basis. The vendor will be responsible for correcting any items found from these scans related to the application or their supported infrastructure. The vendor is not responsible for conducting penetration or network based security testing.

9General Other

Please provide the average number of incidents/break fixes that the State is currently handling for each month as part of the ongoing Operations and management phase.

Current monthly builds on average include 40 work requests that are related to incidents/break fix needs.

3



10

General Scope

Please provide the scope for the following Application Security testing:SAST - Static Application Security Testing (Source Code Level review of the code)DAST - Dynamic Application Security Testing (UI level testing for identifying vulnerabilities at the runtime)Application Penetration Testing

SAST/DAST – Static application Security Testing must meet all applicable controls from NIST 800-53.

Application Penetration Testing – Will be done as part of the annual HSD security assessment and not part of ASPEN M&O

11General Other

Do we have application component Categorization (simple/medium/complex) of the given applications that are in scope for security testing?

Yes. CMS has categorized this system as moderate.

12 General Scope Do we need to include re-scan and remediation support efforts as part of scope?

Yes.

13 General Other Will State provide the security scanning tools to perform the security assessment?

Yes. State has a continuous monitoring program and annual third party assessment.

14

32 Scope

"HSD introduced an Interactive Voice Response (IVR) service in February 2014. It allows clients to call a toll-free phone number and receive predetermined information about their cases. The information comes from the YES-NM database"We assume that IVR performance testing is part of scope. Please confirm.

IVR performance testing is included in scope as it relates to the database performance. The performance of the external call tree and telephonic operations is outside the scope of this procurement.

15

32 Scope

"With ASPEN implementation, HSD has introduced a web portal with a client-facing system allowing clients “self-service” capabilities"Do we have other interfaces (handheld/mobile) to access the ASPEN application other than web interface? We assume that only the web interface is part of performance testing. Please confirm.

YES NM is currently the only web portal access into ASPEN that includes handheld/mobile access. There is a web service supporting Department of Work Force Solutions as a real time scan from their systems - not for use by clients directly and not used via a mobile device.

4



16

32 Scope

"Section 504 and 508 accessibility requirements, and Section 1561 recommendations from the Department of Health and Human Services (DHHS)."We assume that accessibility testing is part of the scope. Please confirm.

The external facing YES NM portal requires accessibility testing and compliance - this is outside the scope of this RFP as that portal is supported by HSD/ITD staff. ASPEN is not required to meet Section 504 and 508 accessibility requirements.

17

33 Scope

"ASPEN is used by approximately 1,300 end users"

How many of these are concurrent users?

We assume that end-user volume would be 1300 only as part of the performance testing scope. Please confirm

is the number of concurrent users.Do we have data for end users volume growth for next 2-5 years? If yes, please provide the details.

Concurrent users can be as high as 900 on a regular basis. We do not have data for end user volume growth for next 2-5 years at this time.

18

33 Scope

As stated, "ASPEN to retrieve case information or reports from ASPEN"How many different type of reports are getting generated/available? Are we using COGNOS for generating reports? Please confirm.

Over 300 different types of reports are being generated from ASPEN. Some of these reports are built and run as canned reports within the application, some are done as Ad-Hoc 'one off' reports, some are built by another HSD/ITD team out of a data warehouse using extracted ASPEN data, and some are done via an Oracle tool 'APEX' by other HSD/ITD staff using ASPEN data. COGNOS is not being used to generate any reports.

19

33 Scope

As stated, "The majority of HSD staff are housed in 36 field offices located throughout New Mexico as well as in the Central Administration building located in Santa Fe" We assume that the Performance testing from different geo-location and from cloud LG is out of scope? Please confirm.

Performance testing from different geographical locations has been conducted in the past and could be in scope depending on the size of a future functionality change or enhancement.

5



20

33 Scope

As stated "Currently HSD provides at least one public assistance benefit to more than 800,000 low-income New Mexicans"We assume ASPEN would support overall member volume of 800,000 plus. Are we expecting overall member volume %growth in next 1-5 years? If yes, Please provide details of the expected % growth

We only have projections for FY15 and FY16 and at this time we don’t see an overall projected growth, however some programs may see growth, mainly Medicaid.

21

33 Scope

As stated "In addition, field staff members complete eligibility determinations on new applications, a significant portion of which will not be eligible for benefits"How many applications are getting ineligible for benefits in year/month/day period and what is the %growth? Please provide the details

Approximately 76,587 applications were denied during the last State fiscal year, with an average monthly decrease of 16%. Detailed information to be provided upon contract award.

22

33 Scope

As stated and provided the count under "Active Case and Recipient Counts by Program as of JULY 2014"Are these active cases, Recipient clients processed through web interface only? Do we have any other interface entry into the system for processing the active cases, recipient clients? If yes, Please provide details

These are not processed ONLY through interface.

Interface entry in to ASPEN can come from field office Lobby kiosk entry, Federally facilitated Marketplace and Yes-NM.

23

33 Scope

As stated and provided the count under "Active Case and Recipient Counts by Program as of JULY 2014"Are expecting any %growth in the active case and recipient count in next 1-5 years? If so, Please provide the details

We only have projections for FY15 and FY16 and at this time we don’t see an overall projected growth, however some programs may see growth, mainly Medicaid.

24

33 Scope

As stated" Use of the term “ASPEN” in this RFP shall include the ASPEN application, YES-NM web service support with ASPEN, Electronic Document Management (EDM), all Interfaces, and xml support to the IVR system"We assume that performance testing on YES-NM web service and EDM and other interfaces alone is out-of-scope. Please confirm

Performance testing for the YES NM web services within ASPEN, all EDM functionality, and all ASPEN interfaces are within scope.

6



25

33 Other

"The support and enhancements to be provided for ASPEN include but are not limited to: Point 4"We assume that State will provide performance test tool, profiling tool, monitoring tool for the performance testing. Please confirm.

The State will provide all tools required.

26

35 Process

As stated, Purchase of hardware, software, hosting, license fees or other commodities is not within the scope of the RFPWe assume that State will provide all necessary software, tool. Please confirm.

The State will provide all necessary software tools.

27

38 Scope

As Stated," 2. Testing will include: Point f. Disaster Recovery/Business Continuity Testing"Do we need testing to support "Disaster Recovery “exercise and Business Continuity Testing?

The State conducts Disaster Recover testing at a minimum annually and more often as needed based on new functionality. The vendor must support this testing as an integrated partner within the scope of this RFP.

28

121 - Point # 6 Process

Below point represents SLA."The Contractor shall respond to and comply with HSD’s direction and timeline for remediation of problems and incidents. ASPEN will continue to change in reaction to business needs, and federal and state legislative mandates and new enhancements will be added through the application maintenance process."Please share current SLAs for various priority incidents

There is no current SLA in place for the Maintenance and Operation of ASPEN. The State will establish an SLA with the vendor following contract award.

29 122 - Point # 14 Process

Are there any tools available to check & monitor system availability? Please provide details

Yes, HSD has industry standard monitoring tools in place for monitoring availability. Specifics will be shared with vendor after contract awarded.

30122 - Point #

15 Process

Are there any tools available to check & monitor application environments, logs, etc.? Please provide details

Yes, HSD has industry standard monitoring tools in place for monitoring application environments and log consolidation. Specifics will be shared with vendor after contract awarded.

7



31

126 - Point # 47 Process

What are all the tools used for (Other than Clearquest & JIRA) 1. Requirement gathering2. Data Models3. Design4. Unit Testing5. System & Integration Testing6. Source code management

The RFP procurement library includes a list of all tools. The following are used currently for items listed: 1. Requirements gathering - Word requirements documents, Excel, ClearQuest and JIRA2. Data Models - Word, Excel, ErWin, Visio3. Design - Word, Excel, Visio4. Unit Testing - Clover (YES NM) 5. System & Integration Testing - QTP, ClearCase, and Atlassian tools6. Source code management - ClearCase, Subversion

32 133 - Pint # 106 Scope

Please share past 12 months ticket count by priority, request type (incidents/Service Requests/etc.), function, etc.

A slide deck for November 2014 information has been added to the Procurement Library.

33

135 - Point # 127 Technical

What is current release management process and release planning?How many releases are happening in a month for1? Enhancement2. Incidents3. Break fixes

Current release management process includes weekly meetings with vendor, IT and business to determine what Work Requests (Incidents and Break Fixes) will go into next major and immediate releases if needed. This is planned for 2-3 releases in the future. Each month has one major release and 1-3 immediate releases if needed. Enhancements are determined with monthly Steering Committee meeting of all Divisions and are scheduled into releases based on priority and hours required to complete the change request. These are determined 2-3 releases out as well. In additional daily data fixes are submitted by vendor for approval and run as required.

34 137 - Point # 140 Technical What tools are used for batch scheduling and batch

monitoring?OpCon tools are used for batch scheduling and batch monitoring currently.

35138 - Point #

142 Technical

Please provide details of OpCon environment Please refer to OpCon Integration documentation in the Procurement Library in the Technical Architecture folder (Deliverable A4_Technical Architecture Plan-Appendix L Opcon Integration)

8



36

142 - Point # 157 Scope

How will the contractor be notified of any Ad hoc report requests?Provide some examples/cases for Ad hoc reports and Recurring report requests.

Ad Hoc report requests are submitted via the Help Desk ticket tracking tool - Cherwell. They are submitted with requirements and requested due dates. They are managed via the ISD Business owner and the vendor as to priority and due dates. See new items posted in the procurement library Ad Hoc Request Summary Spreadsheet and Sample Ad-Hoc Request.

37

General Other

Do the applications owners have specific expectations by moving towards Managed Service model

We understand your question as asking what are the Department’s expectations of a vendor maintained system versus a State staff maintained system. The ISD eligibility system has always been primarily vendor maintained so this is not a new concept for the application owners. There are state staff supported various components alongside the vendor.

38 General Other Are there any regulatory/compliance and audit requirements for these apps?

Yes. CMS Mars-e requirements as well NIST 800-53, SSA, FNS, and HSD security directives

39

General Other

What is the current level of documentation application-wise? Do you have support documentation available?

The application documentation is extensive and includes storyboards, decision tables, requirement and design documents, Operations documentation and a Disaster Recovery plan that includes extensive support documentation. In addition the procurement library has a section detailing the training that is available for the actual application usage documentation.

9



40

General Other

What are the definitions of L1/L2/L3/L4 for the applications in scope

Level 1 support is the ASPEN Help Desk and/or ASPEN Customer Service Center depending on whether question is coming from HSD internal staff or from HSD customers using the YES NM Portal. Level 2 support is via ASPEN Help Desk Support Supervisor and Manager, Level 3 support is via the current support contract vendor staff and Level 4 support is via the current support contract vendor manager. The scope of this RFP includes levels 3 and 4 although resolutions are directed back through the ASPEN Help Desk staff and not directly between vendor and requester.

41

General Other

What are the definitions of P1/P2/P3/P4 incidents? Are these definitions common across different applications?

Priorities are assigned via the help desk tool (Cherwell) by either ASPEN Help Desk staff or by HSD staff if submitted via the self-service portal. Priorities are determined based on a matrix that uses impact and priority as follows:

Dept. Wide Office Bus

Unit Single User

Now 1 1 2 3ASAP 1 2 3 4Soon 2 3 4 4Later 3 4 4 4

In addition there are priorities related to if it impacts benefit issuance, has a work around, is coming from a constituent complaint, etc. The definitions are common across the applications within scope of this RFP.

42

General Other

What is the definition of Minor/Major enhancements? - What is the effort consideration to classify enhancements into Minor and Major enhancements?

There is no strict definition between Minor and Major enhancements currently. Each enhancement is given to the vendor with high level requirements for a level of effort estimate. Based on this estimate it is determined when it can fit into a release or if additional resources need to be pulled from break fix work to assist.

10



43

General Other

What are the key KPIs/Metrics and SLAs for the applications in scope

What are the key SLAs that need to be monitored and reported across applications?

Current M&O year contract does not contain specific SLAs. Please review the Monthly Report sample for KPIs/Metrics that are expected to be tracked and reported. The State is interested in developing additional metrics in the future including methods to track rework.

44General Other

What is the SLA trend and KPI/metrics trend over the last 6 months for the applications in scope

A slide deck for November 2014 information has been added to the Procurement Library in response to this question.

45

Page 32/System

BackgroundScope

Are there any other major enhancements currently going on which may affect the project scope?

There are three known major enhancements that could affect maintenance scope. One is replacement of the current Correspondence module to use HP ExStream tools instead of current OPUS toolset. Two is the planned move of the current Master Client Index (MCI) into a separate service architecture that can be utilized by other applications instead of its current ASPEN embedded architecture. Three is adding real time eligibility to the interface between ASPEN and YES NM for Medicaid.

11



46

Page 40/Require

mentsOther

Current contractor M & O staff count is 26. Can we get the title (skill set) of these resources.

The current M&O staff count required by the contract is 26. The current vendor has additional staff currently supporting the system (as needed) that are not accounted for. The contracted staff by track are:3 - Technical Support3 - Benefit Management1 - Correspondence6 - EDBC3 - Front Office/Self Service2 - Interfaces1 - Project Manager2 - Reports1 Production Support2 - Testing1 - Admin Support1 - Application Development ManagerThe skill sets include track leads, business analysis and java development spread among each team.

47 Page 39/Require

mentsScope

What is the current size of the "Aspen Enhancement" team? The new 28 resources will be added in addition to the current count or will they be replacing the current contractor.

Current size of the vendor ASPEN Enhancement team is 24. These will be replaced by RFP awarded vendor. There is additional State ASPEN Enhancement staff that support YES NM and will not be replaced with this RFP.

48General Contract

Will State be prepared to negotiate a contract based on industry-standard terms applicable to top tier providers

Please refer to Section II, C, parts 15 and 16 under “Sample Contract Terms and Conditions” and “Offeror Terms and Conditions.”

49

General Cost

Please clarify if proposed costs should include New Mexico sales tax (Gross Receipts Tax). Please clarify the I.T services subject to the tax and confirm the statutory rate.

The proposed costs should include the NM Gross Receipts Tax. The rate may vary based on an Offeror’s status under NM tax code, location of services, etc. Due to various aspects as to how the NM Tax and Revenue Department (TRD) establishes tax rates, Offerors should confirm their tax status and liability with the TRD.

12



50

Page 28 Cost

It is our understanding that a Contractor may lease the space at the location and cost indicated on page 28 but is however not required to do so. Please confirm that the should the successful Contractor elect to lease this space pursuant to this RFP, the Contractor would not be required to enter into a lease arrangement for a period prior to the start of delivery or during a period of any delay caused solely by the Department

A signed contract would be required to allow the successful contractor to both enter into a lease agreement as well as collocate.

51 189 /Table B.1. Key

Contractor Technical Tasks and Activities

Technical

Please validate the tools name /provide the tools used to manage the ASPEN environment? ITSM - Cherwell Monitoring - Nimsoft & Oracle Ent MangerBackup /recovery - ?Batch jobs processing /automation - ?Others - ?

ITSM - Cherwell with interface to ClearQuestMonitoring - Nimsoft & Oracle Enterprise Manager as well as custom scripts.Backup/Recovery - Oracle tools as well as NetBackup and DataDomainBatch Jobs Processing/Automation - OpCon

52 191 / Table B.1 - System

patch and upgrade

Technical

How many OS upgrade is expected per month? There can be many devices that can have an Operating System upgrade - i.e. desktops, Vblock, Exadata, etc. - on average we are doing Operating System upgrades 3-4 per year across all devices.

5334 / Other

general information

Technical

What is the failure rate of batch jobs? Are these batch jobs running off business hours?

Batch jobs are rarely failing unless an interface file is not received - then they are cancelled. We have less than one batch job failure per month currently. Batch jobs are primarily run off business hours - exceptions are those batch jobs supporting EDM and YES NM.

54

Page 120 Administrative

Within Appendix 2-B Functional Specifications, the table of contents provided on page 121 indicates that the appendix contains A.8 Staffing and Functional areas; however the appendix doesn't contain any requirements for that category. Can you please confirm there are no requirements under thissubheading of A.8?

The Reference to A-8 was a typographical error and should have been deleted.

13



55

Page 28 Administrative

Could you please clarify that if a Contractor selects the collocation option, would HSD consider providing 15 additional spots at the Siler location? What is the associated cost for additional spots?

Per Section IV, E., page 47, the Siler location only has space for 45 contractor staff. The state would not be able to host an additional 15 spots at the Siler location.

56 Page 65 Administrative

Could you please clarify that there is not any preference points associated to firms that have a Local Business Certification? If so, can you please provide details of this preference as it relates to additional points within the evaluation criteria.

There are no preferential points associated with this procurement. Procurements which include federal funding do not qualify for Local Business or Veteran Owned Business preferences.

14

for... · web viewapplication penetration testing ... design - word, excel, visio4. unit testing...

Documents