15504 spice article

13
The relationship between ISO/IEC 15504 process capability levels, ISO 9001 certi®cation and organization size: An empirical study Ho-Won Jung a, * , Robin Hunter b,1 a Department of Business Administration, Korea University, Anam-dong 5Ka, Sungbuk-gu, Seoul 136-701, Republic of Korea b Department of Computer Science, University of Strathclyde, Richmond Street, Glasgow G1 1XH, UK Received 13 November 2000; received in revised form 31 December 2000; accepted 12 February 2001 Abstract The gradual spread in the use of ISO 9001 and ISO/IEC 15504 also known as software process improvement and capability determination SPICE)) has raised questions such as ``At what ISO/IEC 15504 capability level would one expect an ISO 9001 certi®ed organization's processes to be?'' and ``Is there any signi®cant dierence between the ISO/IEC 15504 capability levels achieved by the processes of ISO 9001 certi®ed organizations and those of non ISO 9001 certi®ed organizations?''. This paper provides answers to those questions as well as to the following question ``Is there any signi®cant dierence in the capability levels achieved by the ISO/IEC 15504 processes of organizations with a large information technology IT) sta and those with a small IT sta?'' In order to answer these questions, we analyzed a data set including 691 process instances PIs) taken from 70 SPICE phase 2 trial assessments performed over the two years from September 1996 to June 1998. Results show that the ISO/IEC 15504 processes of the ISO 9001 certi®ed organizations attained capability levels of around 1±2.3 in 15504 terms. Results also show dierences between the capability levels achieved by ISO 9001 certi®ed organizations and non ISO 9001 certi®ed organizations, as well as between organizations with a large IT sta and those with a small IT sta. Ó 2001 Elsevier Science Inc. All rights reserved. Keywords: Bootstrap; Con®dence interval; Capability level; Permutation test; ISO 9001 certi®cation; SPICE assessments 1. Introduction The software process improvement and capability determination SPICE) project is an ongoing project supporting ISO/IEC JTC1/SC7/WG10 2 in the devel- opment and trialing of the emerging standard ISO/IEC 15504 for software process assessment, capability de- termination, and software process improvement. ISO/ IEC JTC1/SC7/WG10 has developed a technical report 2 TR2) ISO/IEC TR2, 1998) consisting of the docu- ment set shown in Table 1. In the development of these documents, the SPICE project has empirically evaluated successive versions of the document sets of the emerging international standard through a series of trials El Emam and Goldenson, 1995; Goldenson and El Emam, 1996; Maclennan and Ostrolenk, 1995; Smith and El Emam, 1996; ISO/IEC, 1999; Woodman and Hunter, 1998; El Emam and Jung, 2000). More information about ISO/IEC 15504 may be found in ISO/IEC 15504 1998) and in El Emam et al., 1998). ISO 9001 1997) contains 20 clauses see Appendix A) that collectively provide the minimum requirements for setting up a quality management system for use in soft- ware development and maintenance, as well as in other industries. Satisfaction of all the requirements leads to ISO 9001 certi®cation. ISO 9000-3 1997) contains soft- ware speci®c guidelines for the use of ISO 9001, and TickIT 1999) is an initiative, which originated in the UK, to promote the application of ISO 9001 to software. Although ISO 9001 and ISO/IEC 15504 have dier- ent origins, i.e., ISO 9001 is a generic standard for quality management and assurance while 15504 was created solely for software process assessment, capabil- ity determination, and process improvement, the two The Journal of Systems and Software 59 2001) 43±55 www.elsevier.com/locate/jss * Corresponding author. Tel.: +82-2-3290-1938; fax: +82-2-922-7220. E-mail addresses: [email protected] H.-W. Jung), [email protected] R. Hunter). 1 Tel.: +44-141-548-3585; fax +44-141-552-5330. 2 Working Group 10 of Subcommittee 7 Software Engineering Standardization) under a Joint Technical Committee 1 for the international organization for standardization ISO) and the interna- tional electrotechnical commission IEC). WG10 is working for development of standards and guidelines covering methods, practices and application of process assessment in software product procure- ment, development, delivery, operation, maintenance and related service support. 0164-1212/01/$ - see front matter Ó 2001 Elsevier Science Inc. All rights reserved. PII: S 0 1 6 4 - 1 2 1 2 0 1 ) 0 0 0 4 7 - 4

Upload: kalebasveggie

Post on 06-Nov-2015

7 views

Category:

Documents


0 download

DESCRIPTION

15504 SPICE Article

TRANSCRIPT

  • ! !" # $ %

    &'( )% * + &%

    ! "#

    $ ## %# "#% &' & ()

    + , -$ ./ 0$ , 1$ ./ . 2% .

    % % 0 C40 4 0 $$

    $ C40566 7% % 88# % 9

    !: ;:: 88 / E

    4 / $ $$ >/ / ($ &%@/ $$ )% .6= E 0$

    % $ 0% C40@6 C40 $$ = @6=

    C40?6 . % C40 #9 #6 $$%$ 7%$ 0 % 7% $ $ $ 0 % 0' $ $ %= 0 0 7%$ = ', C40?6 0' % 0 % 0 4 C406 FG $ 0 0=

    #% ' . . C40 6 ? '

  • % $ $ % 0 C40& @6= 0 % 4 % 0 $ % $ % 0 $% $ C40EE6 C405%4 = ,6% C405%4 6= % 5%4 '$ 7% % 88#

    EE % $ !

    ;:: 88 EE . C40 ,6 ! $ ;::=

    $ 0 % 0 7% # %

    9 !: ;

    " D% $ 0

    5 ?" D% 0 % $$

    5 @" D% 0 % $ %

    5 " L%

    )-0- 1 "- ) 2 .% 1 ' 3 C40,!!5 67

  • $

    $ 5#= C40 0$ %6= # 0 = .A % $ ,= # $ ' 0 % 0% " 5 . =

    % $% 2 C402%6 M C40M 6 5 C4056 - C40-6 9 0 $ 0 %= C '= # C4056 % 0 %7% % 0$ $

    C40 / $$ =@6=

    0 $ 0$ 0 5#= $ 0 7% 5# 2 % 2 M= #

    9$ $ 0 , 7% 2 0 5# = 5# .= 5# .=. 2 M 05# ,= 5# ,=.= " 5 $ 0$ $ =

    -- #

    0 % % 5 . = 5 C

    .

    $

    $7 #

    # C40$5 $ %$ % $ 0

    %$ % 0 0 % = "

    F=" #7% 0

    F=." E %$

    F=," % 0

    F=" 0

    F=" 5 %$

    + #

    # C40+8&5 0 $$ $ 0 % $

    %$ %$= "

    -D=" 1 $ 7%$

    -D=." 1 0 7%$

    -D=," 1 0

    -D=" $$ 0

    -D=" 0

    -D=>" $

    -D=?" E $ 0

    #

    # C4095 $ $ 0 C40% % 6 %

    0 0 = "

    F5=" 1 %$

    F5=." 50$ % $ $

    F5=," 50$ 7% %

    F5=" 50$ 4 %

    F5=" 50$ 4 %

    F5=>" 50$ C

    F5=?" 50$ %

    F5=@" 50$ $ %

    : #

    # C40:85 $ % $ 0 C

    0 0 = "

    E#-=" E C

    E#-=." E 7%

    E#-=," E 4

    E#-=" E %

    ;/ #

    # C40;"&5 % 0 ! % %

    % C ! ! % = "

    +D=" %

    +D=." 1

    +D=," $

    +D=" 5 4 %$ %

    +D=" 5 0 0%%

    )-0- 1 "- ) 2 .% 1 ' 3 C40,!!5 67

  • $ % $ ' % 0 0 =5 . 0 0 C4051+6 C40 @6 4 0$ $'

    > )% @= 51+ 0 ' + . =

    0$ ? $ 0 !' 0$ " % C40. 6 % #'5 C40, 6 - #'5 C40 6 F# C406 E9 C40 6 C40&% @/ 6= - %$ 0 $ 0 F % $% $

    $ != 0 > C > 5=

    %$ ' C406 0 % 0$ $ 0 7%'

    $ F 'C = $ F0 $= # % C40M6 ' F % : 7% $4 7%

    7%% 7%

    % $ ' 0 =

    ,

    5 % 0 C40 " 5 6

    5 % C405#6

    !" $ 0% % 0 = 4

    % %% 0 = % 5#=

    !" 50$ % 0 = $ $ %

    4=

    4 % 0 0 $ 0 %=

    # $ # % & 9 %$ 0$ % 4 % % %% 4 %=

    !" E 4 % %

    4= (4

    % 0$ 7%$=

    # $ # ' " 9 0$ 0 $ % 4 % $ C=

    # $ () % ' " 9 0$ 0 $ % 4 % %$ =

    !" 0$ $ % 0 =

    % $$ 0 % 0 %$

    %$= %

    =

    # $ # * " 9 0$ 0 %

    % %$=

    # $ # " 9 % % % C4009$ %$ % 0%%6

    =

    !" 5 0$ $ =

    1 $% 0 0$ != 7% % 0

    $ $ 0$= 50$ 7%

    $ = 7% 0 4 % 7% 4=

    # $ + & 9 % $% % % 0$ 0 % $ 0 % 0 %

    =

    # $ # " & 9 % % 0 % $% 0$ 0

    % =

    !" $! 5 0$ $! $ % 0%% %

    $ % = 50$ 0 7%

  • -,-

    -,-- +

  • ! 0

    != % ! ' ! = # % $ 0 0 9 0 % 6= 2 = . , > 0 4 0 $ %

    0 =

    % 0 5$% 0"

    . % $ %

  • C406 % $% 0 $ $ $% C40 -%

    B 6=

    ! % % 88 :: 88 :: = $$ B4C40.6 $ 0 ' $'%$= & ' $ $% 0 $ = 2%$ % 0

    $ $ %9% $% %0% % C40-%

    B6= E % $0 %0% % % 0 ' C40B = >/ D?/ / L$ (4 ,6= # % 0 % 0 $$ B4 C40.6=

    -,-6- +A# #

  • 4 0 0 . % = #9 0

    9 0 9 4 =

    &0 0 F C40.. F6 '= 2 > 5 , 5 0$ F , 5 0$

    F= > 5 ' F < .> 5 F $ ,=? ?=

    2 = % 0 0 5 F F= $ $ F F=2 9$ ,P 0 F 0 0 F ,=

    2 = % 0 < != #% F $ < P 0 $' F <

    $ $+ #

    >; 3!!#= /

    7% '%9>$+ # ' ; 3!!#= /? #

    ; 2 = > ' 0 , 5 C40% . 5 6 0 != # 0 ! $$%$ '

    0 9 $ F=. C40E %$ 6 F=, C40% 06 F=C40 06 -D= C401 $ 7%'$ 6 -D=? C40E $ 0'6 +D= C405 4 %$ %6=2 ., 5 !=

    %$ ' 0 0 . ' 0 != 0 5 9F= C400 6

    .=,= C40 $6 -D=? C40E $ 06 F5=. C4050$ % $ '$6 F5=, C4050$ 7% %6 F5=?C4050$ %6 F5=@ C4050$ $ %6E#-= C40E %6 +D=. C401 6= 0 F=C40#7% 06 = & $! $ 0 % ' = 2

    ! -D=> C40 $6 7% .=

    & 9

    0 5 0 !; 9'"

    2 = = 1% 0 !

    < ! $

  • 5 $

    $

    4 =

    5 $ '7%$=

    5 $ $ %0 $ $'$=

    $ $$ ! C40%$ 0 5 6=

    6-- A# % # >; 3!!#= >; 3!! #=

    7% >% =# A# % 9>$+ # #% % #

    >; 3!! #=/ % >; 3!! #=/@

    %$ 0 5 ' 0 0 . 5 0

    != ' 0 ! 0 '! 0 0$ F= -D=>=1

  • ! , 5 0$ !=

    % %! Q'C40@6 9 0 5$% 0 = Q' $'% 9 '% $ $ C40 6 % 0 %= # 0 F=, -D=-D= F5 F5=. F5=, F5=@ E#-= E#-=,+D=. +D= 0 P 0 ! 0 ! '% 7% == 2 $ '% =

    %C40 ! '!6 . A / '% >. A

    7% > % =# A# % # #% % 9>$+ #

    / '% >.

    A % '% >. A@

    $ % $ '%

    < ! $ % '% = F0% C $= % % 0 >. 5 > ! < .> ! $

  • % 0 . ' 0 !

    0 !=

    0 ! <

    0 ! $

  • . 0 4 %$0 $ 0 % = 5$% 0 4 = # 5$: % = . %

    9 '% $% %$% % 0 . % %$

    .= % . % % . % . = .. C40 .6 % % 0 .= 7% 4% % =

    5$% . 9$0 $ = 9 '' 9 '%

    0 0

    % C40 . 6 C % '%= '% $ 0 % C 0 % = & 0 '% % C 0 = % % P=

    0 % 0 $ $ '7% % $ '% $ % 0 $ '%= 9 '% $% % % % 0 % = 2 $ %

    9 $ '%

  • 1$= $% 5

    E1=

    $$ G= )% &='(= .= # % 0

    $ $= )% 0 $ 0 .,A=

    2 -= 5 = ,= % 0 0

    = " 5 0 0

    $%$ = A?=

    2 -= M B= 5 = ,= % 0

    $= " += ED4 #=

    C40=6 0 " # % 5=

    $% 5 E1=

    D 5= ?= = + 0 %

    + C406 ,A?=

    D

    )=1= @= -$ 0 . =

    E 144 - V4=

    D 1=+= $$ G= >= 5 "

    5C %= " 5 0 @

    0 5 D% 0=

    D 5= ,= 5$% # 5 D% +$

    E 0 &= - V4=

    & L= @= # $ 0 5

    0$4= " $$ G= 1% )='-= E (= C40=6

    5" 5 0 0 5 $$

    1$= $% 5

    E1=

    &% +=B= @= 5 $ = 0

    5 - . .A@=

    ', ?= U% E $ U% #%

    A 5 ," D% 0 # 0

    " 1$ % E'

    0 $% 0 D !=

    ?= U% $ A E 0 U% #%

    1 1$ 5%

    D !=

    )?(D = 5 5 . 2 +

    = =

    51+ 0$ A 0 5

    #$" 5 A5 @=

    +. @" 5 A5 0$ A

    0 5 #$ D !=

    G +== O4 = @= E % " 1

    0 U% += 1%9% 5 5 D

    # C40 ?6=

    G%C 5= = B+#5 ,= A # 5 0$ 0

    $ $ = 0 U% )% @ ?A

    =

    E

    2= 4 D= = 5 " L

    0$4= " 5 0 . 5

    $%$ B=

    E $ 1== +% D== &% -=2= @=

    = ( - V4=

    -%

    )== B =&= = 5$ = ED'

    & - V4=

    5%4 E== (

    =L= D =E= E= B% E=

    ,= G 5 0 E% E ==

    0 % EF','+'.=

    5%4 E== = & $ EE=

    0 ?A@,=

    5K ='M= 2 -= 5 = = % 0

    = $% ?A?=

    $ B= $$ G= >= . 0 5

    = " 5 0 5:> = A=

    Q' 0 (= @= 0 0 9 -$

    0= 0 $ E#=

    = = E$ $%$ = "

    = C40=6 &4 0 9$ 5 = (

    - V4=

    5+ C= @= 5+ 4" 0

    0 $ $ $= 5C

    .,@?,=

    4 = # D% 0 U% E $ $

    % F -. % = FG

    4 "=4= =

    L$ 5= (4 M= ,= -$

    $ = #$ ? >A?.=

    ($ = &% += @= # 0 #$ + 0$

    = " $$ G= 1% )'-= E (= C40=6 5"

    5 0 0 5 $$

    1$= $% 5

    E1 = ,?A,.=

    78( 9' B== C40% 6 0$G F E== 0$ G# C40G # '% 0 6 5=1= E 0$ F 0 #!= & $

    F 4 -# C40- $%!

    # 6 G= & % 0 0 1$ 0B% #$ G F= & $$ 05 $ $$ 0 $%= & % 0 7% $ $ % 0$ 0 $$% 4=

    7 B== 5=1= F 0D = & M% $% F 0 D $$ 0 $% = & % 0 0 7%= & 0 5+ C 5 0 '% 5 0 ! C $$ 0 $=

    )-0- 1 "- ) 2 .% 1 ' 3 C40,!!5 67