realizing business value quickly from data center automation in a large, multi-organization...
DESCRIPTION
This case study illustrates how a large IT organization utilized HP Operations Orchestration, Network Automation, Server Automation, and Service Automation Reporter to deploy data center automation capabilities across multiple business units and multiple organizations within business units. You’ll see how proper team structure, organizational accountability, and a focus on user adoption help accelerate the realization of business value of data center automation. The presenter will explain how automation delivered value at various stages within the software development lifecycle and how standardizing development helped to accelerate the adoption across business units and organizations.TRANSCRIPT
1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
Realizing business value quickly from data center automation in a large, multi-organization environment
Dave Cadoff & Jimmie NeighborspureIntegration
Pure Integration, LLC Confidential & Proprietary
2
Agenda
Introduction
Partnering with pureIntegration
Automation Approach
Challenges We Faced
Results Achieved
Lessons Learned
Pure Integration, LLC Confidential & Proprietary
3
During Spring/Summer 2008, a large MSO initiated a campaign to maximize business
value from their existing Data Center Automation (DCA) investment. This journey
uncovered several business processes and organizations that could benefit from
broader DCA capabilities. Ultimately, MSO committed to deploying DCA across their
enterprise, and this culminated in an ELA transaction.
pureIntegration played an active role in the initial discovery phase:
1) Exploration and analysis of business processes that could utilize DCA (Ex/Softswitch)
2) Provide a service delivery framework for business-focused DCA enablement
Greenfield DCA Install
•Buyer – Single
•Focus – Local
•Use Cases – Limited
•DCA Products – 1 to 2
Expansion of DCA Capability
•Buyer – Multiple
•Focus – Departmental
•Use Cases – Many
•DCA Products –Limited
Enterprise DCA Deployment
•Buyer – Multiple
•Focus – Enterprise
•Use Cases – Many
•DCA Products - Many
DCA Lifecycle
SW Deal Profile
MSO
2008
MSO
Future Vision
Introduction
Pure Integration, LLC Confidential & Proprietary
4
Early in the rollout, MSO recognized that DCA program success would be largely
dependent on factors above and beyond the technical capabilities of the DCA software
products. MSO needed a delivery partner who could successfully manage parallel
initiatives, inter-dependent tasks, and cross-organizational work efforts.
MSO chose to partner with pureIntegration based on our:
Proven track record of successfully delivering complex projects at MSO
Differentiated service delivery methodology designed to maximize DCA program
success
DCA Program Success
Project Execution
Organizational Alignment
Capability Delivery
Solution Sustainability
User Acceptance
Business Value Attainment
Partnering with pureIntegration
Pure Integration, LLC Confidential & Proprietary
5
AUTOMATION APPROACH
Pure Integration, LLC Confidential & Proprietary
6
Rapidly deploy Data Center Automation (DCA) capabilities to
realize savings associated with a business case that required a
$M+ initial investment.
One of the main use-cases driving the overall business case involved
automating video headend configuration management. This capability
development required the client to work across multiple applications within the
video headend, as well as multiple organizations within the SDLC
(Dev/QA/Ops). MSO encountered many challenges such as:
• Poor Cross-organization Communications
• Lack of Automation Adoption
• Unable to Accurately Identify Resource Requirements
The Challenge
Pure Integration, LLC Confidential & Proprietary
7
Initiative Assessment
Process
Requirements Management
Process
Value Metrics Definition
Enterprise Architecture
Process Standards &
Project Management
Overall Process
Definition and
Execution
Individual Capability Delivery
Data Center
Automation
Use Case Definition
Design & Development
QA Deployment Operations
This component was missing!
Establish a single entity that is accountable for delivering DCA capabilities
across the enterprise.
pureIntegration was engaged to support the DCA enterprise deployment
program, utilizing their Data Center Automation Deployment Services
(DCADS) Methodology.
The Solution
Pure Integration, LLC Confidential & Proprietary
8
Automation Category Target Use Cases
Configuration ManagementProvides ability to create scheduled complex audits
against a golden standard configuration.
Incident Management
Provides automated identification and remediation
of production incidents. Also provides automated
execution of scheduled maintenance tasks.
Release Management
Provides lifecycle management for enterprise servers
and applications. SA and OO will contribute to
reduction in operational costs by automating manual
deployments.
Reporting, Admin, and
Infrastructure
SA and SAR provide the extensive reporting
capabilities required to manage an IT infrastructure.
This also includes DCA infrastructure upgrades
required to support new initiatives.
We structure our capability development, as well as resource alignment, based on
the following automation categories. This allows us to maximize delivery team
effectiveness based on the unique skills and toolsets required for each automation
category.
Automation Approach
Pure Integration, LLC Confidential & Proprietary
9
CHALLENGES WE FACED
Pure Integration, LLC Confidential & Proprietary
10
DCA
Program
Success
Challenges Mitigation Strategy
Project
Execution
• No dedicated program
manager
• No additional headcount to
design/build the DCA
framework
• Engage pureIntegration to
lead/drive program execution
• Utilize DCADS methodology
Organizational
Alignment
• Obtaining sponsorship and
support from senior
management across
business units and teams
(Eng/QA/Ops)
• Work directly with key
management stakeholders to
understand MBO’s and how
DCA directly, or indirectly,
affects them
Challenges We Faced
Pure Integration, LLC Confidential & Proprietary
11
DCA
Program
Success Challenges Mitigation Strategy
Capability
Delivery
• Developing and deploying DCA
was initially silo’d amongst
teams
• Capability development without
requirements or feedback from
end-users
• Centralize and standardize process
for requirements, design,
development, testing and
deployment of DCA across all teams
(Eng/QA/Ops)
Solution
Sustainability
• Architecting for scale – utilizing
cores and satellites
• Central vs distributed DCA
resource support model
• Incorporate capacity planning as a
front-end design activity
• Perform detailed work scoping to
determine the number of resources
required to support each
organizations DCA plan-of-record
Challenges We Faced
Pure Integration, LLC Confidential & Proprietary
12
DCA
Program
Success Challenges Mitigation Strategy
User
Acceptance
• Skepticism of new tool
introduction
• Resistance to process
change
• Facilitate monthly brownbag
sessions with various user
groups to promote DCA
awareness
• Focus on delivering quick wins
to each user group
Business
Value
Attainment
• No definitive way to
measure program success
• Delivery timelines were
being missed, but the cost
of delay was not known
• Defined value metrics across
hard/soft savings categories
• Publish value to senior
management to communicate
current status and justification
for future initiatives
Challenges We Faced
Pure Integration, LLC Confidential & Proprietary
13
RESULTS ACHIEVED
Pure Integration, LLC Confidential & Proprietary
14
MSO initiated the DCA Enterprise Deployment Program in March 2009. As a result,
MSO has achieved key program milestones and significant business savings.
HSD
Video
Network
Eng QAVideo
Ops
Field
Ops
Eng QASustaining
EngineeringRDCs
Eng QA SysOps NOC
DCA Metrics 2008 2009 2010 (Plan)
Data Centers 9 51 67
NA (Devices) 0 9000 18000
SA (Devices) 1800 2200 3100
OO (Flows) 0 43 100
Annual Savings $1.2M $2.4M $4M
DCA Program Adoption
2008
2009
2010 Target
Legend
Results Achieved
Pure Integration, LLC Confidential & Proprietary
15
Automation Category
SysOps Network Video Voice
ConfigurationManagement
- 1800 managed servers- DHCP Audit- LDAP Audit- DNS Audit- DTFTP Audit- Mailstore Audit- License/Maintenance
Mgmt
- 8000 managed devices
- 25 Network Policies- Automated routine
configuration changes across data centers
- 550 managed servers- BMS Server Audit- MAS Server Audit - Web Based reporting
tool for displaying dashboards and reports (Packages, RPMs, STBs OOS, etc.)
- Limited device management
- POC built to audit Cisco BTS softswitches in lab env
ReleaseManagement
- Mail 4.0 OS/application profile
- Commercial mail OS/application profile
- Mail 4.3 OS/application profile
- Cisco CMTS IOS deployment
- Automated installation/upgrade Mystro BMS 3.1
Results Achieved
Pure Integration, LLC Confidential & Proprietary
16
Results Achieved
Automation Category
SysOps Network Video Voice
IncidentManagement
- Restart failed app processes
- Repair corrupt mailbox- Veritas disk replacement- SAN attached host health
check- DTFTP bootfile.conf- Log file parsing (root
cause diagnosis)- Additional NOC
automated workflows
- Syslog integration (root cause diagnosis)
- Event notifications for all configuration changes
- PMG Status- VOD troubleshooting- Start Over
troubleshooting- MAS Health
verification
Reporting, Admin & Infrastructure
- Executive style status reports for audits emailed weekly with SAR
- All changes made to network devices in last 24 hours
- Devices with different startup and running configurations
- Agents installed on BTS softswitches in lab env
Pure Integration, LLC Confidential & Proprietary
17
LESSONS LEARNED
Pure Integration, LLC Confidential & Proprietary
18
Secure high-level executive support for DCA…..even if you are initially
starting small. Bottom-up adoption is tough without top-down support.
Treat DCA initiatives like application rollouts vs toolset enablement.
DCA is a framework and requires rigor around processes such as
requirements, design, development, QA & deployment.
DCA capability development should be value-driven vs capability-
driven. Avoid the trap of building automation for the sake of
automation.
Enterprise DCA delivery is NOT a product consulting engagement. If
approached in this manner, there is a high probability that DCA
deployment will stall and user acceptance/adoption will not be realized.
Lessons Learned
20 ©2010 Hewlett-Packard Development Company, L.P.
To learn more on this topic, and to connect with your peers after
the conference, visit the HP Software Solutions Community:
www.hp.com/go/swcommunity
21