bmc capacity optimization - best practice webinar ... · designing the bco data integration...
TRANSCRIPT
© Copyright 12/4/2012 BMC Software, Inc 1
BMC Capacity Optimization - Best Practice Webinar Designing the BCO data integration architecture
Int'l Toll: 913-312-0828 US/CAN Toll free: 866-816-1982
Argentina: 0800 444 6440
Australia: 1 800 612 415
Austria: 0800 295 780
Bahamas: 1 800 389 0491
Belgium: 0 800 75 636
Brazil: 0800 891 0266
Bulgaria: 00 800 115 1141
Chile: 123 0020 6707
China, Northern Region: 10 800 714 1509
China, Southern Region: 10 800 140 1376
Colombia: 01 800 518 1171
Czech Republic: 800 700 715
Denmark: 80 883 277
Dominican Republic: 1 888 752 0002
France: 0 800 914 176
Germany: 0 800 183 0299
Greece: 00 800 161 2205 6440
Hong Kong: 800 968 066
Hungary: 06 800 112 82
India: 000 800 1007 613
Indonesia: 001 803 017 6440
Ireland: 1 800 947 415
Israel: 1 80 925 6440
Italy: 800 789 377
Japan: 00348 0040 1009
Latvia: 8000 3523
Lithuania: 8 800 3 09 64
Luxembourg: 800 2 3214
Malaysia: 1 800 814 723
Mexico: 001 800 514 6440
Monaco: 800 39 593
Netherlands: 0 800 022 1465
New Zealand: 0 800 451 520
Norway: 800 138 41
Panama: 00 800 226 6440
Peru: 0800 54 129
Philippines: 1 800 111 010 55
Poland: 00 800 112 41 42
Portugal: 800 827 538
Russia: 810 800 2915 1012
Singapore: 800 101 2320
Slovenia: 0 800 80439
South Africa: 0 800 982 304
South Korea: 003 0813 2344
Spain: 900 937 665
Sweden: 02 079 3266
Switzerland: 0 800 894 821
Taiwan: 00 801 127 186
Thailand: 001 800 156 205 2068
Trinidad and Tobago: 1 800 205 6440
United Kingdom: 0 808 101 7156
Uruguay: 0004 019 0348
Venezuela: 0 800 100 8540
INTERNATIONAL TOLL FREE
December 4th, 2012
BMC Capacity Optimization - Best Practice Webinar Designing the BCO data integration architecture Speaker: Giuseppe Nardiello Sr Mgr BCO Product Management Panelists: Sudheer Apte Marco Colombo Charlie Geisler Giorgio Gasparini Mike Paska Mike West Evan Wetstone
© Copyright 12/4/2012 BMC Software, Inc 3
BMC Capacity Optimization - Best Practice Webinar Legal Notice
The information contained in this presentation is the confidential information of BMC, Inc. and is being provided to you with the
express understanding that without the prior written consent of BMC, customers and partners may not discuss or otherwise disclose
this information to any third party or otherwise make use of this information for any purpose other than for which BMC intended.
All of the recommendations and information described herein are at the sole discretion of BMC and are subject to change and/or cancellation, and in no way should this content be viewed as
guarantees or warrantees on BMC’s part.
© Copyright 12/4/2012 BMC Software, Inc 4
BMC Capacity Optimization - Best Practice Webinar Topic: “Designing the BCO data integration architecture”
We will discuss topics such as: How do I select the right data source for managing a specific
environment? Do I need to deploy new data collection capabilities or can I rely on already deployed management tools? How do I design the data integration architecture from a logical perspective?
We will NOT discuss topics such as: How to integrate BCO to extract data from external reporting
tools or applications How do I size the BCO infrastructure based on the number of
systems to manage? How many connectors do I need? How do I make it scalable? How do I design the BCO infrastructure from a physical perspective?
How do I configure a connector to a specific data source? How do I manage historical recovery? How to ensure BCO resilience?
© Copyright 12/4/2012 BMC Software, Inc 5
BMC Capacity Optimization - Best Practice Webinar Agenda
A (short) introduction to BMC Capacity Optimization (BCO) architecture
data integration
Designing data integration in BCO general criteria
some real-world use cases
Conclusions
Some of your use cases will be selected to be discussed – please have them at hand !
F I R S T P A R T
S E C O N D P A R T
© Copyright 12/4/2012 BMC Software, Inc 6
technical and business-oriented dashboards & scheduled, on-demand and by-exception reports
BMC Capacity Optimization High-level architecture
Web Interface (OOTB and customizable views)
Capacity Database (CDB)
Data Integration Engine
Financial Metrics Facility Metrics Performance Metrics Business KPIs Configuration Metrics Events
Storage Data Center Facilities Physical/Mainframe/Virtual/Clouds Databases & Middleware Applications Networks Business Drivers
Reporting Engine Analytical and Modeling Engines
Capacity Manager Storage Admin Service Manager IT Financial Manager IT Operation Manager Cloud Admin
BMC Capacity Optimization
© Copyright 12/4/2012 BMC Software, Inc 7
AIX Topas / CEC
BMC Atrium CMDB
BMC Atrium Discovery and Dependency Mapping
(ADDM)
BMC Patrol
BMC Performance Assurance
BMC ProactiveNet Performance Mgmt
BMC Cloud Lifecycle Management
BMC Capacity Management for Mainframe (CMM)
Citrix Xen server
EMC Storage Scope Control Center
nMon for AIX
Microsoft Performance Monitor (perfmon)
Microsoft SCOM 2007
NetApp Data Fabric Manager (DFM)
Oracle Enterprise Manager (OEM)
Solaris Prstat
Solaris Extended Accounting
UNIX OS configuration (AIX, Solaris, …)
UNIX CLI tools (DiskFree, IOstat, …)
UNIX System Activity Report (SAR)
VMware vCenter
BMC Capacity Optimization Connectors
CA Unicenter TNG
HP SiteScope
HP Business Availability Center (BAC)
HP Load Runner
HP OVO EPC Agents (Coda Agents)
HP OpenView Performance Agents
HP OpenView Performance Insight
HP OpenView Performance Manager
NetIQ AppManager
Oracle AWR / STATSPACK
SolarWind Orion NPM
TeamQuest Enterprise Database
Zenoss
(*) OOTB connectors are supported by BMC - please verify the availability for the specific platform, version and metrics with BMC representatives
(**) Market Zone Direct (MZD) connectors are supported connectors that require a separate license
(***) Template connectors are provided as-is (at no additional cost) and are nor supported by BMC - please read BMC support policies for template connectors
CSV file
NCSA and custom text log files
SQL queries to RDBMS (e.g. Oracle, IBM DB2
and MS SQL Server)
SNMP interfaces (standard/proprietary MIBs)
HTTP and JMX interfaces
Template Connectors (***)
OOTB Connectors (*)
Custom connectors can be developed from an
Eclipse-based Integrated Development
Environment (IDE) provided as part of BCO SDK
Specific Connectors General-purpose Connectors
Custom connectors (SDK)
MarketZone Direct Connectors (**)
IBM Tivoli Monitoring
HP Reporter
HP uCMDB
MS SCOM DW
Entuity
EMC SMI-S
Hitachi Device Manager
more to come...
© Copyright 12/4/2012 BMC Software, Inc 8
Movìri Integration for BMC Capacity Optimization
Moviri Integration for BMC Capacity Optimization v3.0
HP Reporter
IBM Tivoli Monitoring provides supported connectors to:
• HP Reporter HP Reporter 3.8, 3.9
• HP uCMDB HP uCMDB 9 and 10
• IBM Tivoli Monitoring ITM 6.2.1, 6.2.2, 6.2.3
• MS SCOM Data Warehouse MS SCOM DW 2007, 2012
• Entuity Entuity EYE 2012 and 12.5
Entuity
MS SCOM DW
BMC Capacity
Optimization uCMDB
NEW
ENHANCED
ENHANCED
ENHANCED
© Copyright 12/4/2012 BMC Software, Inc 9
Sentry Software Integration for BMC Capacity Optimization
provides supported connectors to
• EMC SMI-S provider EMC Symmetrix DMX™ Series (DMX-4, DMX-3, DMX-2) EMC Symmetrix V-Max Series EMC CX4 Series, CX3 Series, CX Series EMC VNX series
• Hitachi Device Manager Hitachi Adaptable Modular Storage 2000 family Hitachi Universal Storage Platform (USP, USP-V/USP-VM) Hitachi Virtual Storage Platform (VSP)
Sentry Software Integration for BMC Capacity Optimization v2.0
NEW
NEW
BMC Capacity
Optimization
EMC
Hitachi
© Copyright 12/4/2012 BMC Software, Inc 10
BMC Capacity Optimization Visibility, Planning and Optimization
Technical and business oriented views
Scheduled, on-demand and by-exception reports Operation Manager
IT Financial Manager
Service Manager
Storage Admin
Capacity Manager
Database Admin BMC
Capacity Optimization
© Copyright 12/4/2012 BMC Software, Inc 11
BMC Capacity Optimization OOTB Views
DataCenter Overview
vSphere datastore allocation AIX PowerVM frame
vSphere infrastructure
© Copyright 12/4/2012 BMC Software, Inc 12
BMC Capacity Optimization for Mainframes OOTB Views
z/OS Image Summary View – 4 Hour Rolling Average
4 Hour Rolling Average MAX value KPIs
Mid – Short term mainframe capacity summary
z/OS Image – MIPs consumed
© Copyright 12/4/2012 BMC Software, Inc 13
BMC Capacity Optimization - Best Practice Webinar Agenda
A (short) introduction to BMC Capacity Optimization (BCO) architecture
data integration
Designing data integration in BCO general criteria
some real-world use cases
Conclusions
Some of your use cases will be selected to be discussed – please have them at hand !
F I R S T P A R T
S E C O N D P A R T
© Copyright 12/4/2012 BMC Software, Inc 14
BMC Capacity Optimization General criteria
When designing the BCO data integration archicturecture, you are required to evaluate several factors, such as: - data availability (e.g. integration of already adopted management tool vs
new data collection mechanism to be deployed) - data quality (e.g. data collection mechanisms guaranteed to return CPU
utilization values <=100% for all managed platforms) - data richness (e.g. more comprehensive set of metrics or at higher
granularity that can satisfy already known or potential requirements) - easiness of integration (e.g. access to repository with public data schema vs
parsing of exported reports in complex formats) - robustness of integration (e.g. parsing of files located on a remote file
system or pushed to a local file system) - cost of integration (e.g. parsing exported CSV files versus creating a custom
connector to leverage complex APIs)
In general, there is not a «right» solution as these and other factors need to be balanced carefully. Also, a “one fits all” approach may not be suitable and different design may apply to different environments to be managed.
In the following, we will discuss some general criteria that can be used as a guidance to design the BCO data integration archicturecture.
© Copyright 12/4/2012 BMC Software, Inc 15
BMC Capacity Optimization General Criteria
BCO has the ability to virtually integrate any existing data source thanks to BCO flexible and scalable data integration layer
Therefore, when discovering the existing management environment and identifying potential data sources to be integrated, you might be tempted to consider any availble data source
However, only data sources that are would provide useful data to support Capacity Management use cases should be considered
In general, make sure that you understand how existing or new data sources are supporting your Capacity Management use cases of interest
Also take into account that your environment can change over time in terms of available data sources
#1 – It is not about data integration – it is about Capacity Mgmt
Best Practices: Don’t integrate data sources that you are not sure how to leverage
Be driven by your Capacity Management use cases
Adopt a stepwise approach – perfect knowledge is not always available
© Copyright 12/4/2012 BMC Software, Inc 16
BMC Capacity Optimization General Criteria - Real-World Use Case
Case 1: Can we integrate WebSphere Application Server (WAS)? First of all, integrating “WAS data” in BCO may mean different use cases:
perform a seasonal analysis on WAS instances (e.g. heap / thread pool size) execute extrapolation models on applications running on WAS containers
based on technical proxy (e.g. number of servlet calls) of Business KPIs Depending on which use case is of interest, the available integration options
can be very different (e.g. a custom connector to IBM Tivoli Monitoring for WAS or a custom connector to JMX provider or more simply parse an application log file, respectively)
While these use case are valid, do not forget that Capacity Management is done OS level (e.g. to right size CPU or memory resources) not to identify AS level configuration
Case 2: Can we integrate BMC EUEM? Measured response times provided by BMC End-User Experience
Management (EUEM) or similar tools can be collected to provide trending analysis or to calibrate what-if scenarios but the latter constitutes quite an advanced use case
#1 – It is not about data integration – it is about Capacity Mgmt
© Copyright 12/4/2012 BMC Software, Inc 17
BMC Capacity Optimization General Criteria
BCO can provide support for easily geneating seasonal and trending analyses and reports on any metric collected from the integrated data sources
However, not all metrics that might be potentially available from a data source need to be collected unless useful from a Capacity Management perspective
BCO supported connectors usually collect only a subset of all potentially available metrics, that correspond to typically useful deliverables (e.g. views)
BCO supported connectors usually also provide a way of filtering specific Datasets of interest (e.g. filtering out NIC-related metrics)
While BCO allows additional custom metrics to be added, existing BCO Data Sets usually provide a quite comprehensive set of metrics can be expected to be collected for each managed resource type
#2 – Horses for courses
Best Practices: Don’t integrate metrics that you are not sure how to leverage
Be driven by your Capacity Management use cases
Perform a match & gap analysis before adding custom metrics
© Copyright 12/4/2012 BMC Software, Inc 18
BMC Capacity Optimization General Criteria - Real-World Use Case
#2 – Horses for courses
Case 1: Can we integrate NetBackup metrics? Again it is important to identify the intended use cases in order to evaluate
whether it might be useful to integrate specific metrics, for example: job execution times: to execute trend analyses volumes of backup data: to be correlate to number of clients
While some other metrics may not be integrated as being more performance oriented or more useful for an administration purpose, as for example: disk performance, drive performance, ... real-time library drive status
Case 2: Should I create a new custom metrics for these NetBackup metrics (and in case for each different backup technology of interest)? BCO Datasets in #13 WKLGEN («Business Driver Generic Metrics») contains
many (56) predefined metrics that can used to map the previously listed metrics when used as Business KPIs: ELAPSED_TIME or BYSET_ELAPSED_TIME TOTAL_EVENTS or BYSET_EVENTS
© Copyright 12/4/2012 BMC Software, Inc 19
BMC Capacity Optimization General Criteria
Data sources providing the “same metric” can still differ in terms of: Support only for specific platforms (e.g. not for HP IVM) Support for different time granularities (e.g. 5m or 1h, possibly depending
on how original data collection has been configured) Support also for «by-metrics» (e.g. storage utilization by tablespace, or
process-level CPU utilization, by custom workloads or transaction classes) Availability of historical data (based on configured retention policies, that
usually cannot be changed easily)
Since BCO provides decoupling between data collection and data model for managed systems, it is possible to have different data sources providing different set of metrics (provided corresponding connectors are shared)
#3 – Not all data sources are created equal
Best Practices: Evaluate your specific needs in terms of metrics
Be driven by your Capacity Management use cases
Consider using different data sources for different platforms
© Copyright 12/4/2012 BMC Software, Inc 20
#3 – Not all data sources are created equal
Case 1: Can we leverage BPPM data for AIX workload characterization? BCO 9.0 can leverage BPPM (9.0) data collection for performing capacity
management activities on AIX platforms (both standalone and partitioned) However BPPM does not provide support workload characterization (that is
resource utilization breakdown by custom group of processes running on an LPAR) that is supported by BPA (a BPA agents needs to run on each LPAR)
Case 2: Do I need to use BPA for collecting vSphere data or can I use BCO? BCO 9.0 collects an extensive set of vSphere metrics, but there are cases
where an higher level of detail is required, for example for accounting or chargeback purposes
BPA the ability to collect (via agent-less or agent-based data collection depending on the specific platform) data from inside the VMs (at the “virtual node” level) related to transaction classes
Example: banking applications accessed via Citrix terminal servers with SAP (Linux) servers virtualized with VMware - each Citrix server runs multiple applications that are accounted individually and also Linux guests run multiple services (e.g. WAS JVMs) that need to be accounted individually
BMC Capacity Optimization General Criteria - Real-World Use Case
© Copyright 12/4/2012 BMC Software, Inc 21
BMC Capacity Optimization General Criteria
BCO may collect metrics related to a managed system (aka BCO entity) from diverse data sources - and this is what usually is used to combine configuration data (e.g. from a CMDB) with system metrics (e.g. from BPPM)
However, notice that BCO does not support multiple datasources for the same metric for the same systems (e.g. to compare their data quality) – BCO has been designed to supports Capacity Management use cases, including ability to generate automated forecasting and automated by-exception reporting, to provide resource utilization visibility also from a service/business perspective and to provide IT cost visibility (showback/chargeback) that require
Therefore, it is highly discouraged to configure BCO to collect the same set of metrics from different datasources – unless this is done in a test environment, for a limited time period and only to evaluate the best data sources to integrate in the production environment
#4 – Data sources should not be multiplied beyond necessity
Best Practices: Evaluate in advance the quality of data sources (in case, in your test env)
Don’t collect twice the same metric in your production env
© Copyright 12/4/2012 BMC Software, Inc 22
BMC Capacity Optimization General Criteria - Real-World Use Case
#4 – Data sources should not be multiplied beyond necessity
Case 1: Can we collect the same VM metrics both from the hypervisor level (e.g. vSphere or Hyper-V) and from inside the VM? In general, metrics measured from inside the VM and at the hypervisor level
may not provide the same values - as matter of fact they do not represent exactly the same metrics
In order to make this distinction clear, metrics collected at the VM level are referred to a different entity (Virtual Node) that is distinct from the VM entity associated to vSphere or Hyper-V hosts and clusters
Case 2: Can we collect the same metrics from two data sources (in our test environment) whitout creating custom «shadow» metrics? The simplest approach consists in not sharing the connectors extracting data
from the different data sources so that managed entities are populated as completely separated entities
Notice that also with this appoach, the complexity of the BCO environment (and the size of the database) increases – for example, entities of the same type can be counted twice and be visible under different names
© Copyright 12/4/2012 BMC Software, Inc 23
BMC Capacity Optimization General criteria
When multiple equivalent BCO data integration options are available, always balance the advantage of the most complex and complete possible integration with the benefits of a simpler while still robust integration approach
BCO provides several mechanisms that can ensure an automatic and reliable data integration: «last counter» mechanism: retrieve only new data or data that were missed
in a previous run (also used to retrieve historical data) «done» mechanism: parse only new files not already parsed in previous runs
Better control of connector execution, support for migration from test to production environments and for database cleanup is being introduced in the upcoming Service Pack
#5 – Keep it simple
Best Practices: Perform a cost / benefit analysis
Be driven by your Capacity Management use cases
Consider also how your environment may change over time
© Copyright 12/4/2012 BMC Software, Inc 24
BMC Capacity Optimization General criteria
#5 – Keep it simple
Case 1: How can we integrate network metrics from a Network Management solution that does not provide any interface that can be leveraged? This is a real case that was solved by having reports scheduled on a regular base from
the Network Management solution, so as to allow BCO to automatically parse all reports files via a custom connector (few lines of code)
In some cases it is required to keep the integration simple as there are no other way of integrating closed solutions
Case 2: How can we integrate storage management metrics from a Storage Management solution that had a complex API (WBEM) for extracting data? This is a real case that was solved by having reports scheduled on a regular base from
the Storage Management solution, so as to allow BCO to automatically parse all reports files via a custom connector (few lines of code)
Even when some different options are available, do not understimate how effective a simple integration approach can be – at least as a first integration step that allows to quickly demonstrate BCO value
As a matter of fact, as high as 80% of data integrations can be easily done via general purpose connectors or via simple parsing
© Copyright 12/4/2012 BMC Software, Inc 25
BMC Capacity Optimization General criteria - summary
#1 – It is not about data integration – it is about Capacity Mgmt
#2 – Horses for courses
#3 – Not all data sources are created equal
#4 – Data sources should not be multiplied beyond necessity
#5 – Keep it simple
Be driven by your Capacity Management use cases
b u t m o s t a n d f o r e m o s t
© Copyright 12/4/2012 BMC Software, Inc 26
BMC Capacity Optimization From criterias to a decision tree
Identify the use cases of interest
Is there an OOTB or a MZD connector?
Identify the required metrics
Identify the potential datasources
Identify potential integration mechanisms and interfaces
Is there a template/community connector?
Is a general-purpose connector viable?
create a custom connector
PHASE 1: use cases
PHASE 2: metrics and data sources
PHASE 3: connectors
/
© Copyright 12/4/2012 BMC Software, Inc 27
BMC Capacity Optimization - Best Practice Webinar Agenda
A (short) introduction to BMC Capacity Optimization (BCO) architecture
data integration
Designing data integration in BCO general criteria
some real-world use cases
Conclusions
Some of your use cases will be selected to be discussed – please have them at hand !
F I R S T P A R T
S E C O N D P A R T
© Copyright 12/4/2012 BMC Software, Inc 28
Question
Answer
BMC Capacity Optimization Your cases
© Copyright 12/4/2012 BMC Software, Inc 29
BMC Capacity Optimization - Best Practice Webinar Upcoming Webinars
January 8th
“Managing BPA to BCO data integration”
February 5th
“Deploying and configuring BCO-CLM integration”
March 5th
“BCO for Storage Capacity Management”
Thanks for attending…
and ... completing the survey!
© Copyright 12/4/2012 BMC Software, Inc 30
Learn more at www.bmc.com