1.

• Case Study: How Caixa Econmica in Brazil Uses IBM Rational Insight and Performance Measurement to Support Outsourcing to Its Software Factories

• Wagner Arnaut IT Specialist, IBM [email_address]

• Paulo Lacerda IT Specialist, IBM [email_address]

• Rodrigo Evangelista Caixa National Architecture Manager [email_address]
• PPM-1565

2.

Agenda

Performance Measurement

3. IBM Rational Insight 4. Insight at Caixa Econmica Federal

• • • • - Customer Profile
    • 5. - Problem Description
  • 6. - Proposed Solution
• 7. - Solution Deployment

8. - Architecture Overview

9. - Reports Delivered 10. - CAIXA Benefits 11. - Next Steps 12. - Lessons Learned 13.

What is an IT Performance Measurement System?

The use of measurement to determine and manage the performance of organizations against their goals

Challenges in performance measurement

• Connection between improving high-level goals and software delivery strategy

14. Set of measurements that best address the goals, development goals and delivery strategy

15. Rolling out the right amount of measurement at the right level for the right audience 16. Making data available to support decisions

Measured Capability Improvement (MCI) Supportsperformance measurement by providing a 3-tieredperformance measurement system and the relationshipsbetween metrics in each tier

17.

MCI: Leverage a control framework to manage to expected business results

OperationalLevel

PracticeLevel

BusinessLevel

Business Objectives

Measures (Automatic Collection)

Efficiency

Control

Value

Feedback Feedback Feedback

Process Enactment / Governance Enforcement / Process Awareness (Jazz Platform or other non-Jazz-based products)

Performance Measurement

Control Metrics (Subjective) (Self-Check)

Control Metrics (Objective) (Rational Insight ) Velocity, Backlog, Density Growth, SPI, CPI

Value Metrics (Objective) (Rational Insight / Cognos) Cost, Revenue, Time-to-ROI, etc

Operational Effectiveness Metrics (Objective) (Rational Insight) Productivity, Quality, Time-to-Value, etc.

Process Definition / Practices (RMC)

Operational Objectives

Other Data Sources

18.

3-Tiered Performance Measurement

Business Level Metrics

• Usage: Provide specific information that has impact on the business mission

19. Audience: CxO

20. Example: Revenue, Time-to-ROI, KPIs (specific to LOB)

Operational Level Metrics

• Usage: Measure and manage improvements to development performance

21. Audience: VP of development, Development Mgmt., Operations Mgmt.

22. Example: Productivity measurements, Quality measurements, Time-to-Value measurements

Practice Level Metrics

• Usage: 1) Demonstrate the level of adoption, 2) Track and monitor progress

23. Audience: Project manager, team lead and team members

24. Example: Iteration velocity, project backlog, defect density, growth chart, SPI, CPI 25.

Agenda

Performance Measurement

26. IBM Rational Insight 27. Insight at Caixa Econmica Federal

• • • • - Customer Profile
    • 28. - Problem Description
  • 29. - Proposed Solution
• 30. - Solution Deployment

31. - Architecture Overview

32. - Reports Delivered 33. - CAIXA Benefits 34. - Next Steps 35. - Lessons Learned 36.

Introducing Rational Insight

Automatic collection, measurement and analysis of real-time data for informed decision making

37. Measures tailored and optimized by specific role providing relevant timely information 38. Visibility into your data across projects, products, and geographical boundaries to monitor status and progress in a single view 39. Continual process improvement with the adoption of best practice frameworks to assess, measure, and improve results. 40.

Cognos DataManager

Cognos FrameworkManager

ETL (Extract, Transform, Load)

Rational Data Warehouse

Focal Point

Rational Insight Architecture Overview

ClearQuest

ClearCase

RequisitePro

RTC

RQM

MS Project

DOORS

HP Quality Cntr

Web architecture to retrieve product data in XML (REST get)

Project, Process and PortfolioManagement

Requirements Management

Change and Release Management

Quality Management

Rational InsightProvide objective insight and transparent governance for software and systems delivery

XML

TestManager

DataStream

DataStream

Enterprise Reporting Server (Cognos 8 BI Server)

CIO

Project Manager

Development Team Lead

Release Manger

41.

Agenda

Performance Measurement

42. IBM Rational Insight 43. Insight at Caixa Econmica Federal

• • • • - Customer Profile
    • 44. - Problem Description
  • 45. - Proposed Solution
• 46. - Solution Deployment

47. - Architecture Overview

48. - Reports Delivered 49. - CAIXA Benefits 50. - Next Steps 51. - Lessons Learned 52.

Customer Profile

Business

Largest public bank in Latin America *

53. Customer base of 33.6 million people* 54. More than 17,000 points of attendance in all 5,561 Brazilian cities * 55. The bank in charge to support Brazilian Government social initiatives

IT

Software development geographically distributed in three cities:

• Brasilia

56. Rio de Janeiro

57. So Paulo

Development outsourced to 5 external software factories

58. More then900applications

*source: http://www.caixa.gov.br

59.

Problem Description

It is difficult to understand the status of each development unit.

• Braslia, Rio de Janeiro and So Paulo

Manual work to extract metrics from development units

• Completion of worksheets to consolidate information from different systems

60. Reports are always outdated Offline Reports

Executives do not have enough visibility into development performance

• They needed to create a comparison view between:

• Development units

61. Teams

62. Projects

Metrics are required to support key business decisions

• Investment/Enablement

Difficult to align IT KPIs and Corporate KPIs

Caixa need to have visibility about Software Factories productivity

• Need to compare Software Factories results

63.

Proposed Solution

Use Rational Insight to provide executive dashboards and reports with consolidated performance information from development units/software factories/teams/projects.

64. Initially we wanted to see the following metrics:

• Total function points delivered

65. Software factory payments

66. Open x closed demands 67. Demand backlog 68. Production environment Incidents

Considering the following dimensions:

• development area, software factory, system, department, date

It was necessary to get data from the following support systems:

• SIGCP: An ASP/SQL Server application used to manage software factory contracts.

69. SISOL: A ClearQuest workfow used to manage software development demands.

70. Rational Team Concert

Rational Insight

71.

Software Development Governance Quick Wins

1

2

3

4

5

6

7

8

9

10

Software Development Metrics Dashboard

Installation and Configuration

Metrics from SISOL (CQ) + SIGCP + TeamConcert

Operational Support + Enablement

11 Fev

11 Mar

11 Abr

11 Mai

11 Jun

11 Dez

11 Nov

11 Out

11 Set

11 Ago

11 Jul

Architecture and Metrics Definition.Initial Customization

72.

Solution Deployment

Insight and Data ServicesInstallation and setup

73. SIGCP Live data reports

• Reports Definition

74. Reporting Metadata

75. Reports Creation

SIGCP data warehouse reports

• Dimensional modeling

76. ETL Jobs

77. Reports Creation

1 week

2 weeks

6 weeks

78.

Solution Deployment

ClearQuest Connection

• Dimensional Modeling

79. ClearQuest data extraction customization

80. Reports Creation

Rational Team Concert Connection

• Dimensional Modeling

81. TeamConcert data extraction customization

Custom fields

Reports Creation

Performance Tuning

4 weeks

4 week

3 weeks

82.

CAIXA Performance Measurement View

Caixa's measurement portal is here Caixa's measurement portal is here Caixa's measurement portal is here 83.

Architecture Overview

84.

Reports Delivered Payment Dashboard

Home GrownApplication

Data Source

85.

Reports Delivered Function Points Dashboard

Home GrownApplication

Data Source

86.

Reports Delivered On-time delivery rate

Home GrownApplication

Data Source

87.

Reports Delivered Business Demands Dashboard

Home grown

88. application 89. + 90. ClearQuest 91. Data Source 92.

Reports Delivered Comparing Development Units

Home GrownApplication

Data Source

93.

Reports Delivered Software Development Tasks Dashboard

RTC Data Source

94.

Reports Delivered Incidents Dashboard

Customized RTC Data Source

95.

CAIXA Benefits

CAIXA now has a set of dashboards and charts that provide a high level view of its software development organization performance, that includes:

• Single point of access to the development metrics, KPIs and related reports.

96. Consolidate data from all development areas that can be easily compared along the time.

97. Automated data extraction eliminating the need of manual data collection and consolidation. 98. Cross tool reporting, allowing the association of data extracted from different software development tools and administrative applications to provide valuable information. 99. Measures aligned with IT KPIs and Corporate KPIs 100. Data updated on daily basis 101. Software Development Decision Making tool in many levels:

Business

102. IT Operational 103. Software Development Operational 104. Project - Practices 105.

Next Steps

There's more to come:

Implement CAIXA software development performance measurement system.

• CAIXA has a list of about 30 key performance indicators that are going to be implemented using Rational Insight.

Connect Insight to Rational ClearCase to provide change management metrics;

106. Connect Insight to Rational RequisitePro to provide requirement related metrics; 107. Connect Insight to Operational Deployment Management Systems (IMPACT or RTC) to provide metrics about release promotion to production environment. 108.

Next Steps Work in progress

Project dashboard

RTC, SISOL(CQ),RequisitePro and ClearCase Data Source

109.

Lessons Learned

1.Take some time tounderstand customer requirements and the data availablefrom your data sources. It can be risky not giving it the right importance, this will save you of unnecessary rework.

• Try to getData Modelsfrom external data sources

2.Do aquick st art with Live Data reportsand then go to data warehouse reporting. 3. Test, test, test.

• Working with a lot of data and calculations may be tricky and some mistakes may happen at some point in the report development process (ETL, metadata, report creation). It's very important that you spend some time testing the source data, the ETL jobs and the reports you create before publishing them.

4.Understandingdimensional modeling conceptsis very useful when customizing Rational Insight ETL Jobs and data model. It will help you with some basic tasks and avoid unnecessary changes in your model. There are some good books on this subject. 110.

Lessons Learned

Performancemay become an issue.

• Databasesskills makes the difference here to do query performance analysis, index creation, etc.

111. Some reports may not be need to run against the database every time the user access them, generally because their data is update weekly or daily. In those cases, for performance reasons, it may be better to use astatic versionof the report weekly or daily updated.

112. Insight ArchitectureOptimization

• • Distribute some Cognos services between more then one server

• • • content manager, report service, presentation service, etc...

• • Add HTTP Server (IIS) to the architecture

• • • Reduce application server workload
  • 113. Enable static files compression
• 114. Caching

• • Enable query parallel execution
• 115. IncreaseBIBusTKServerMain.exe idleTimeLimitSec.

Unleash the labs teamplayed a key role in our deployment to do the

initial data extraction and report customization.

116.

www.ibm.com/software/rational

117.

Daily iPod Touch giveaway

Complete your session surveys online each dayat a conference kiosk or on your Innovate 2011 Portal!

118. Each day that you complete all of that days sessionsurveys, your name will be entered to win the dailyIPOD touch! 119. On Wednesday be sure to complete your full conference evaluationto receive your free conference t-shirt!

SPONSORED BY

120.

Copyright IBM Corporation 2011.All rights reserved.The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied.IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials.Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreementgoverning the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.Product release dates and/or capabilities referenced in these materials may change at any time at IBMs sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way.IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

www.ibm.com/software/rational