overview - international test and evaluation association...• the dcgs‐a pmo can publish a...
TRANSCRIPT
Test.Forge.mil
Overview
© 2011 All Rights Reserved
Improving Software Development
Reduce Cost
More Testing
Faster Delivery
Agile Development
????
Virtualization
Cloud
© 2011 All Rights Reserved
Current Process
Program Office
Development Contractor(s)
TestContractor(s)
IV&VAgency(s)
Ops Team
© 2011 All Rights Reserved
Test.Forge.mil Goals
Provide access to the all resources necessary…
Desktops & servers
…while leveraging automation and virtualization to expand resources.
Operating Systems & updates
Applications & code
Test cases
… so they can support &perform rapid, repeatable, agile, on‐demand testing of software systems…
Virtual services
Functional Performance Regression DT OT
Virtual machinesVirtual users
...to all parties involved…
Program Office Developer Tester Community Member
© 2011 All Rights Reserved
O/S & Application Provisioning
Machine VirtualizationCloud
Virtual Services
Application Lifecycle
ManagementSource Code
Build Test Tools
What it is
Test.Forge.mil
* some products for illustrative purpose; not all listed are implemented at this time
Hosting
Messaging
Backb
one
Data Center
PKI AuthenticationForge.mil SSO Database
Community.Forge.milIntegration
Remote Access Solutions
UI & Other Requirements
CONS3RT PaaS BaaS TaaS
© 2011 All Rights Reserved
Local Cloud Private Cloud Public Cloud
Test as a Service
Tools
How it Works
Asset Library
O/S Test AssetsAppsVirtual ServicesConfigs
Scenarios/AppliancesVMs Servers Devices
Scenario #7
Web Collaboration Scenario
Source Code
Web Collaboration Scenario
Platform as a Service
Provisioning
Virtualization
Build as a Service
© 2011 All Rights Reserved
Sample Project
Sample Project: ObjectiveBuild a web dashboard for senior leadership that displays DoD activities alongside current events and weather for situational awareness purposes.
DCAPES
Map
Weather
News
Troops
Location: Kabul, Afghanistan
84,375
21,222
18,153
34
111
Use of program/organization names are for illustration purposes and do not represent any official involvement with Test.Forge
© 2011 All Rights Reserved
Test.Forge Asset Library
Test.Forge Dashboard Project
O/S Test AssetsAppsVirtual ServicesConfigs
Scenarios/Appliances
• The Master Dashboard PMO can define the standard configurations for the systems to be used in development & test.• The DCGS‐A PMO can publish a standard DCGS system to be used in development & test.• The TBMCS PMO can publish a virtual service model with standard inputs/outputs for development & test.• JITC can provide standard interoperability tests tat are run using existing test tools.• The developer can build & save virtual service models of some of the industry news feeds to expedite efforts.• The developer can run continuous integration testing with every daily build.• The testers can begin creating & validating test cases in parallel with development.• The developers & testers can quickly evaluate a “what‐if” scenario by replacing one part of the system (e.g. IBM vs. JBOSS).
Forge.mil
Master Dashboard Project
External CloudTest.Forge Services
CONS3RT
Infrastructure(NTP, DNS, DCHP)
Test as a Service(LISA, Selenium, W3AF, etc.)
Virtual Service Engine(s)
DCGS‐A Project TBMCS Project
Test.Forge Cloud
External CloudMaestro
FAP
VSManager
GUI
TestManager
Messaging
Dashboard v0.2
Program Office Developer Tester Community Member
Use of program/organization names are for illustration purposes and do not represent any official involvement with Test.Forge
DCGS v10.1
Dashboard v0.3
Demo
© 2011 All Rights Reserved
What it does
Test.Forge.mil enables the ability to …– build virtual machines
– install multiple software applications onto these virtual machines
– pull, build & deploy source code onto these virtual machines
– mange virtual service models
– combine virtual machines & virtual service models into complex “systems of systems” scenarios
– deploy (turn on) these scenarios
– select the destination cloud for the deployment
– run automated tests against a deployment
– save and reuse all the components, scenarios and deployments
– execute these tasks • on demand
• with any combination on Windows and Linux components
© 2011 All Rights Reserved
How to use it
Test.Forge.mil can be used for…– Platform as a Service
• build workstations for developers to write code• build workstations for testers to write test• perform manual tests• “what if” analysis
– Test as a Service• execute regression testing on new builds• perform smoke tests• performance testing• “what if” analysis
– Continuous Delivery• continuous integration (big & little)• compliance testing• Build as a Service
– Collaboration• share knowledge, techniques & technology across communities • connect non‐developers & non‐testers
© 2011 All Rights Reserved
Benefits
Goals & Benefits:– On‐demand environments to support all levels of testing
– Shorten timelines, reduce cost for testing
– Push quality down to source of problems
– Test early, test often (every week, every day, every hour)
– Pull in all parties involved in development (not just developers)
– Act as a “Test Range” for IT Systems
– Incorporate best of breed COTS/GOTS test tools
– Transform “cloud” from buzz word to useful tool
– Reduce risk
– Work with common tool sets for testing, automation
© 2011 All Rights Reserved
Contact Information
John CaseboltPresident, CTOJackpine [email protected]: (978) 263‐6025Cell: (978) 394‐2195
Peter WalshProgram Manager, NPLACEWVHTC Foundation, [email protected]@hanscom.af.milPhone: (781) 377‐7211DSN: 478‐7211Cell: (617) 816‐6001
ESI3 Lab15 Eglin StreetBuilding 1607, Shelter 10Hanscom AFB, MA 01731(781) 377‐7184DSN 478‐7184https://www.esi3l.ceif.hpc.mil
Backups
© 2011 All Rights Reserved
Future PhasesPilot Phase
Test.Forge Feature Roadmap*
Scenarios, Assets & Clouds
• build & deploy virtual machines
• build & deploy virtual services
• deploy into other cloud(s)
• connect your cloud
• deploy on hardware
• config & deploy devices
• nested scenarios
• automated O/S updates
• dynamic resource management
Testing
• automated test workflow
• execute iTKO LISA tests
• execute soapUI tests
• execute Selenium tests
• execute W3AF scans
• execute Retina scans
• execute Fortify code scans
• execute HP LoadRunners tests
• consolidated results reporting
Other
• integration in Forge.mil
• single sign‐on
• users & groups
• CAC authentication
• asset visibility rules
• NIPR accreditation
• project level resources
• billing model metrics
• license management
*Feature set subject to change during the development process
© 2011 All Rights Reserved
How We are Building it
local (IDE) build
Jenkins & MavenNew Code
Good build?
Deploy in INT
Debug & test
Pass tests?
Deploy in CI
Deploy in QA
Deploy in Prod
Smoke/Regression
Smoke/Regression
New tests
Pass tests?
New stuff?
Jenkins & Maven
Develop
erIntegration
Continuo
usIntegration
QA
Prod
uction
Pass tests?
New stuff?
Jenkins & Maven
Tag
Tag
Delete artifact
svncommit