practical usage of fpa and automatic code review piotr popovski

1

Orange Polska for IWSM Mensura, Cracow 2015

Orange Polska IWSM Mensura 2015

Practical usage of FPA and automated code review Piotr Popowski & Bogusz Jeliński

IT Applications

2


What we measure, count or look for in Orange

Function Points

Manual, IFPUG-based counting for pricing

Quality of code (static review)

Violations of patterns

Weighted average of low-level grades

Defects per FP

Software process maturity, repository, continuous integration

Automated FP

Complete source code

3


Measurement in figures

MANY TECHNOLOGIES JAVA, C/C++, PHP, PL/SQL, WEB METHODS, PEOPLE SOFT, POWER BUILDER, ABAP, ORACLE FORMS, etc.

1 000 000 FP COUNTED

800 PROJECTS (GO-LIVES)

1 500 CODE AUDITS

7 DOMAINS

5 VENDORS

4 800 ENHANCEMENT COUNTS (incl. CRs)

200 IT SYSTEMS

5 YEARS

EXPERIENCE

4


Part 1 Estimating size and price for IT projects using FPA method Piotr Popowski – IT Applications, Orange Polska Contact: [email protected]

5


Context

In 2010 Orange Polska transformed its sourcing strategy from multi-vendor to consolidated vendor agreements. In return for a long-term contract, the company offered its suppliers exclusive rights to deliver software changes in particular IT areas.

Vendors consolidation

Solution

Challenge Lack of free market pricing mechanisms, as implied by the contracts, imposed the necessity to implement a dedicated sourcing and pricing method.

To meet the new challenges Orange Polska adopted a new method of estimating IT development size, workload and finally, pricing.

6


From FP to price In four steps

FP and SNAP count based on IFPUG and custom rules

FP adjustment

Adjusting number of FP. Impact ratios based on COCOMO II

effort calculation

Contract-based productivity ratios related to vendor, system and technology

price

Price calculation using rates defined in contracts

FP SP

Calculated by Orange and vendor analysts

Verified and authorized by OPL IFPUG certified team

Orange and vendor analysts

Sourcing division Sourcing division

7


Customization Custom rules

The custom rules cover the areas where IFPUG FPA proved to be in some way deficient

Custom rules

5% of FPs is „Rule 27” algorithmic complexity - internal data processing where data does not cross application boundary

7% of FPs is „Rule 30” reusability – a reusable component, where development is made once but analysis and tests are conducted for many elementary processes

20% of FPs – dedicated rules for 26 systems in the following areas

Integration

Business Intelligence (ex. Data Warehousing)

Wholesale

8


Customization FP adjusting

The main goal of adjusting is to assess effort

Adjustation is made separately for each BFC (Base Functional Component)

Impact ratios are defined based on COCOMO II Equation 4

Impacts are expert-estimated and mutually agreed

Also using NESMA impacts as reference

Creation or huge change Moderate change Small change Suppression or COTS

Eff

ort

9


Customization Productivity

Productivity ratios were established during vendor selection process

COCOMO II factors were used as reference for determining productivity

Productivity

Technology System Vendor Year of

contract

10


Knowledge base and tools

FP calculation

CARTO – Excel spreadsheet for doing detailed calculations based on HLD

Full decomposition: project > system > functionality > BFC (or SCU for SNAP)

Additional information for effort calculation (e.g. technology, impact)

FP adjustment

Document and report

FP Repository – all data from CARTOs recorded in a dedicated system

Over 30 on-line generated reports available from the repository

Progressive build of Application Count

HLD

CARTO

REPOSITORY

REPORTS

11


Common pricing model Instead of tenders, quotes and negotiations – objective pricing

Budget and capacity early estimation Based on historical data

Project Roadmap management Project assignment to particular path depending on path capacity

Path management – schedule and capacity

KPIs Cost efficiency – spending/FP

Quality – errors/kFP

Process optimization Better and more precise analytical documents (HLD)

Tracing volume of changes in IT projects (e.g. number of projects, number of change requests, changing size of average project in particular domain)

Benefits

12


Challenges

Introduce another measuring method in order to verify or improve current model (e.g. Use Case Points)

Measuring Agile projects

Improving the way to measure configuration changes

Introduce NESMA impacts

13


Orange Polska experience

Since 2010 Has a dedicated, internal, IFPUG certified Metric Team

Uses IFPUG FPA (and SNAP since 2012 ) for pricing IT projects

Uses metrics to manage Project Roadmap and IT budget

Dedicated Metric Team provides the following services Design and implementation of effort estimation and pricing models for IT contracts

FPA, SNAP and custom trainings

Methodology, contract and project audits

Services are offered internally and externally, both on-site (currently Warsaw and Cracow) as well as remotely

14


Part 2 Automated code review

dr Bogusz Jeliński – IT Architecture Strategy, Orange Polska Contact: [email protected]

15


What we measure, count or look for in Orange

Function Points

Manual, IFPUG-based counting for pricing

Quality of code (static review)

Violations of patterns

Weighted average of low-level grades

Defects per FP

Software process maturity, repository, continuous integration

Automated FP

Complete source code

16


Code review – history & inspirations

1976 - Fagan’s classic article in IBM Systems Journal

1979 - Lint for Unix V7

1991 - ISO/IEC 9126

1992 - Technical debt metaphor by Ward Cunningham

90’s - Sun Java Code Conventions

Our own manual review (WebMethods, Drools)

17


Tools used for the static review of source code pros & cons

Many supported technologies

Tracks cross-layer dependencies

Semi-automated count of function points

Cost of license

Needs extensive training to operate

Not automated (as its manufacturer might claim)

Cannot be run by any vendor before code hand-over

CAST Sonar

No license fee

Integrated with dev tools

Supports few technologies

Some plugins are not open-source

HP Fortify

Security oriented

Only security oriented

Not free of charge

18


Quality metrics Content of contracts with vendors

Low level rules

Aggregate

Total Grade

Expected improvement

< 3.10 0,25

3.1 - 3.5 0,15

> 3.50 0,10

Expected yearly improvement

Tree-based aggregation with weights

Values from 1 (bad) to 4 (very good)

19


We prevented the increase of technical debt

Technical Quality

Index

Discovered acts of sabotage – removing comments to hamper the change of vendor

Created rules for the analyzer to prevent technology-specific anti-patterns (based on test reports)

Fewer production incidents by 4%

Increase of code performance prevented infrastructure spending

Benefits

20


Challenges

We found that vendors differ vastly in terms of delivered quality

Reviews may be used to create a ranking of software vendors

Functionalities always prevail over internal quality

We need quality metrics in contracts

Vendors tend to claim too many false-positives

Anti-patterns remain in the code, aggregate metrics (CAST TQI) may be used to exert pressure on the vendor

There have to be penalties in contracts for poor quality. Or at least KPIs inspected by top management

Good-will assumption does not work

21


Service for third parties – scope and tools Quality audits by Orange Polska*

Static code review

CAST & Sonar (cyclical & one shot), including adding new rules

Productivity review

CAST (automated function points)

Open-source licenses review

Palamida

Security audit

HP Fortify

Software process improvement

SVN, Git, Jenkins (continuous integration)

* provisioned in English, French and Polish

22


Thank you

Bogusz Jeliński [email protected]

Piotr Popowski [email protected]

IWSM Mensura 2015

Estimating size and price for IT projects using FPA method

Automated code review