qa 2010 stki

QA Trends

Galit FeinEVP & Senior Analyst

Office of the CIO Strategies

www.galitfein.blogspot.com

Your Text hereYour Text here

Galit Fein’s work Copyright 2010 @STKI Do not remove source or attribution from any graphic or portion of graphic 2

Agenda

QA & Testing Services Israeli Market

Factors Influencing QA Staffing Level

IT Trends Influencing QA Staffing Level

Pricing Models

Galit Fein’s work Copyright 2010 @STKI 3Do not remove source or attribution from any graphic or portion of graphic

QA & Testing Services Israeli Market

2008240.0 -10.42% 215.0 13.95% 245.0 2.04% 250.0 2.00% 255.0

20.0 -10.00% 18.0 38.89% 25.0 40.00% 35.0 20.00% 42.0 Project Management Office

2009 2010 2011 2012Q/A & Testing

Software TestingOutsourcing:

An IdeaWhose Time Has

Come?



QA & Testing Main Players

Matrix

MALAM/TEAM

NESS

Tesnet

Tescom

Mertens Hoffman

Calanit Carmon

Taldor

Top Group

One1

Netwise

EL AD Systems

Category 1

Category 2

Category 3



QA Staffing Level

The skills of a capable QA group are essential to

improving application development & maintenance:

• Early discovery of inconsistencies

• Errors reduce

• Rework prevention

Yet, the appropriate level of QA

staffing is often difficult to determine



Factors Influencing QA Staffing Requirements

When ITO requires higher level of QA staffing:

Orgs that produce their own SW other than orgs that

implement commercial SW packages

Orgs relying on SW for mission-critical functions

Highly regulated industries

Orgs that produce SW as a part of their services to customers

Companies devoting their efforts to steady process

improvement (Six Sigma/ CMMI…)


QA & Testing Ratios

1.70%25 percentile

3.40%Median

5.30%75 percentile

Per FTE

425 percentile

6Median

1875 percentile

• SW developers per QA FTE

• QA unit* ratio to total IT

*Includes employees who are part of a dedicated QA or testing function. It does not include employees who perform testing as part of their other job responsibilities, such as application programmers who also perform their own unit testing

Source: ComputerEconomics

- Do some orgs have lower needs & are they more efficient in their use of QA staff ?

OR- Are they understaffed in this position?



QA Staff Ratio / total IT

3.90%

2.20%

0.70%

Finance

High Tech

Government

Source: ComputerEconomics



Find The Right Balance

Too often QA is viewed as an obstacle to getting work done

Desire to

meet

deadlines &

enhance

systemsNeed to

ensure

quality,

reduce

waste &

enforce

standards



IT Trends Influencing QA Staffing Requirements

New regulations

Cloud computing

Agile

Outsourcing

Office of the CIO

Agile



New Regulations Requirements

In recent years, the quality & integrity of information

systems are being examined more closely because of

regulations that address the security and privacy of

information

QA function becomes even more important in light of

these requirements



Office of the CIO Israeli Trends

After many years of talking OTC is actually happening!

Financial crisis & budget reduction in 2009 forced better ITG:

• Increased focus on IT costs - efficiency increasing

• New regulations requested better transparency

Most ITOs have expanded their OTC/ PMO department:

• IT budget control & mng, PPM, rewrite methodologies & WF definition,

tools evaluation, HR - resource allocation & mng, QA, GRC…



The PMO goals are:

Keep all this in the right balance:

• Scope of Work

• Budget

• Time

• Quality



SW metrics

Bugs - How many bugs (including severity)

• When the bug was open, when it was closed

• Number of bugs correspondingly to closing date of the version

• When was the bug found(dev, functional test, regression, prod = escaping bugs)

Test coverage (what part of SW was tested)

Percentage of automatic tests (from the complete project = automatic +

manual)

Progress of project compared to plan

How many (and percentage of) passed builds

Development of new functionality vs. effort to correct bugs from prod &

from tests

STKI recommends: change your metrics once in a while



SW Measurement Insights

Bugs are the most easy metric to measure & track

Sometimes too much effort is spent on bugs (example:

minor memory leak) while important functionality is

delayed

How much effort should developer put in “non-dev”

missions (like detailed progress report)?

Should maintenance be part of the general development

team or should maintenance have a special “team”?



Agile Project Management

Analy

sis

Code

Test

Implem

ent

Analy

sis

Code

Test

3 6 9 12

36

912

Source: Cutter



Agile SW Development Israeli Trends

Agile is catching up in ITOs

Clients see the benefits of Agile in SMPs

Large Agile projects (50 people) are

more challengeable mainly when there is high

interdependencies between the SW modules

Israeli ISV’s report that sometimes their clients demand Agile

Agile requires change in the hierarchical structure that is

extremely difficult to large conservative organizations



Outsourcing

While Professional Services market significantly dropped

last year, the QA & testing PS remained stable

QA - rapidly growing sourcing segment

Acknowledging the advantage & cost saving (up to 50%)

by local nearshore



Israeli Nearshore

Israeli ITOs do not offshore QA, except global orgs

Israeli nearshore existing clients have broaden their

scope

Added Value Professional Services :

• Best Practices – ISTQB, ISO 27001

• Automation Testing Tools

ISTQB

ISOCMMI

Six

Sigma



Israeli Nearshore Main Players

Matrix Global (Talpiot)

Ness

Malam Team

Ness

Aman – Tehila

Manpower

Galil Software

Many more…



Pricing Models

Traditionally QA was priced on a fixed-time/fixed-price

basis (most tenders still are)

Today:

There is a real difficulty to evaluate QA scope of work,

therefore most contracts priced on time & material basis

Israeli ITOs prefer in-house responsibility & QA PS, both

• On premises

• Off premises – nearshore



Developers vs. Testers

Advantage:

Already knows the system

- Driven by “delivery”

Disadvantage:

Must learn the system

- Seek the problems

- Driven by “hours”

But But

qa 2010 stki

Documents

portion of graphic office

portion of graphic factors

qa fte qa unit

dedicated qa

qa staffing levelit

higher level of qa staffing

work donedesire

capable qa group