efficient mobile network acceptance testing

Download Efficient Mobile Network Acceptance Testing

Post on 03-Jan-2017

215 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • P R E S E N T A T I O N

    International Conference On

    Software Testing, Analysis & ReviewDEC 4-8, 2000 COPENHAGEN, DENMARK

    Presentation

    Bio

    Return to Main Menu T7

    Thursday, Dec 7, 2000

    Efficient Mobile Network

    Jorg Paul

    Paper

    Acceptance Testing

  • mmobilfunk

    mannesmanntelecommunications

    1

    Efficient Mobile Network Acceptance TestingEfficient Efficient Mobile Network Mobile Network Acceptance TestingAcceptance Testing

    Jrg PaulJrg PaulMannesmann Mobilfunk GmbH

    Am Seestern 1D-40547 Dsseldorf

    Germanyjoerg.paul@d2mannesmann.de

    DrDr. Dieter. Dieter Kreuer KreuerEricsson Eurolab Deutschland

    GmbHEricsson-Allee 1

    D-52134 HerzogenrathGermany

    eeddik@eed.ericsson.se

    Thomas GrtnerThomas GrtnerEricsson GmbH

    Fritz-Vomfeldestr. 26D-40547 Dsseldorf

    Germanyeddthga@edd.ericsson.se

  • mmobilfunk

    mannesmanntelecommunications

    2

    ContentsContentsContents

    1. Mobile Network Acceptance Testing2. The NEXT/AIMS Tool Platform3. The GSM R8 Project4. Results5. Conclusions

    1. Mobile Network 1. Mobile Network Acceptance TestingAcceptance Testing2. 2. TheThe NEXT/AIMS Tool NEXT/AIMS Tool PlatformPlatform3. 3. TheThe GSM R8 Project GSM R8 Project4. 4. ResultsResults5. 5. ConclusionsConclusions

  • mmobilfunk

    mannesmanntelecommunications

    3

    GMSC MSC

    MSC

    HLR

    VLR

    AuC

    EIR

    PSTN

    SSS

    BSC BSC

    BTS BTS BTS BTS

    BSS

    MSMS

    OMC

    OSS

    Basic Structure of a GSM NetworkBasicBasic Structure of Structure of a GSM Networka GSM Network

  • mmobilfunk

    mannesmanntelecommunications

    4

    differentdifferent releases forreleases for SSS, BSS SSS, BSS and and OSS OSS at at leastleastshipped once shipped once a a year by the vendor year by the vendor ((EricssonEricsson))

    Test Test phasesphases forfor thethe newnew softwaresoftware::

    1. Mobile Network Acceptance Testing1. Mobile1. Mobile Network Network Acceptance TestingAcceptance Testing

    Unit T

    estin

    g

    Func

    tion T

    estin

    gSy

    stem

    Test

    Type

    Acce

    ptanc

    e Tes

    tFS

    A / Ro

    llout

    Featu

    re Te

    sting

  • mmobilfunk

    mannesmanntelecommunications

    5

    any newany new HW HW or or SW SW release tested before integration release tested before integrationinto the commercial network into the commercial network ((no harm no harm to to the networkthe network))

    joint test activity between vendor and operatorjoint test activity between vendor and operator testing done testing done in in the actual the actual live live network configurationnetwork configuration SUT not SUT not tested standalonetested standalone, , but as but as a a part of the full part of the full GSMGSM

    networknetwork emphasis on emphasis on end-to-end end-to-end functional testingfunctional testing both blackboth black--box and whitebox and white--box testing performedbox testing performed

    Type Acceptance TestingTypeType Acceptance Testing Acceptance Testing

  • mmobilfunk

    mannesmanntelecommunications

    6

    MSC

    BSSPSTN HLRVLREIR

    AuC Billing

    PBX

    ACD

    VMS

    CTISMSC VRU VRU SGSN

    IN SCP

    W-MSC

    WAP

    VoIP

    Growth of Network ComplexityGrowth ofGrowth of Network Network Complexity Complexity

  • mmobilfunk

    mannesmanntelecommunications

    7

    Conflict Situation for TestingConflict Situation for Testing

    Testing

    Quality

    Costs Time

  • mmobilfunk

    mannesmanntelecommunications

    8

    Network Complexity and Test CoverageNetwork Network Complexity and Complexity and Test Test CoverageCoverage

    Complexity

    Test Coveragewith Test Automation

    TAcc duration

    Test Coveragewithout Test Automation

    10

    15

    20

    25

    30

    35

    40

    1990 1992 1994 1996 1998 2000

  • mmobilfunk

    mannesmanntelecommunications

    9

    resource savings due to automation of regressionresource savings due to automation of regressiontestingtesting

    speed-up ofspeed-up of test execution test execution more efficient test more efficient test plant plant utilisationutilisation (e.g. (e.g. at nights andat nights and

    weekendsweekends)) higher higher reproducibilityreproducibility of automated testing of automated testing enable testersenable testers toto concentrate on the new features of concentrate on the new features of aa

    releaserelease

    Automated Testing: Our goalsAutomatedAutomated Testing: Our goalsTesting: Our goals

  • mmobilfunk

    mannesmanntelecommunications

    10

    originaloriginal target application target application: : Statistical Usage TestingStatistical Usage Testing end-to-end functional testing ofend-to-end functional testing of mobile mobile networksnetworks both blackboth black--box and whitebox and white--box testing possiblebox testing possible suitable platform forsuitable platform for automation of automation of Type Type AcceptanceAcceptance

    testing performed by Ericsson andtesting performed by Ericsson and Mannesmann MannesmannMobilfunkMobilfunk

    3. The NEXT/AIMS Tool Platform3.3. The The NEXT/AIMS Tool NEXT/AIMS Tool Platform Platform

  • mmobilfunk

    mannesmanntelecommunications

    11

    Principle of NEXT/AIMSPrinciple of Principle of NEXT/AIMSNEXT/AIMS

    NEXT AIMSServer

    AIMSHardware

    SUT

    TENUX

    SRSMS

    Commands Actions

    Calls

    Commands Commands

    SUT StatusSUT Status

    Results Signals

  • mmobilfunk

    mannesmanntelecommunications

    12

    Integrated Automated Testing Platform for GSMIntegrated Automated Testing Platform for Integrated Automated Testing Platform for GSMGSM

  • mmobilfunk

    mannesmanntelecommunications

    13

    4. The GSM R8 Project4. 4. The The GSM R8 ProjectGSM R8 Project

    CSS PSS BSS OSS

    GSM R8

  • mmobilfunk

    mannesmanntelecommunications

    14

    Automated Test Objects: Automated Test Objects:

    Data / Fax33%

    SMS3%

    Call Tests50%

    Other Tests14%

  • mmobilfunk

    mannesmanntelecommunications

    15

    AutomatedAutomated Test Test Objects Objects ( (continuedcontinued):):

    Call HandlingCall Handling Interworking withInterworking with BSS BSS and and GSM R7 GSM R7 SupplementarySupplementary Services Services ChargingCharging Statistics and Traffic MeasurementsStatistics and Traffic Measurements Data andData and Fax Tests Fax Tests ShortShort Message Service Message Service APZ Regression TestsAPZ Regression Tests

  • mmobilfunk

    mannesmanntelecommunications

    16

    5. Results5. 5. ResultsResults

    Grade of Automation: Test Objects

    24%

    52%

    24%

    fully automated TOsfully manual TOsmixed TOs

  • mmobilfunk

    mannesmanntelecommunications

    17

    Test Case Execution Times

    5:04:22 5:04:225:04:22

    0:00:00

    0:05:00

    0:10:00

    0:15:00

    0:20:00

    0:25:00

    0:30:00

    0:35:00

    0:40:00

    0:45:00

    0:50:00

    0:55:00

    Anno

    unce

    mts

    APZ R

    egres

    sn

    Call B

    arring

    s Ba

    sic C

    all Hn

    d Ch

    arging

    Fa

    x/Data

    Fra

    ud Pr

    evntn

    Int

    erwrk

    R7/R

    8 Int

    rw M

    SC/BS

    C

    Statis

    t. Sub

    stm.

    Overa

    ll

    Test Object

    MinimumMean Maximum

  • mmobilfunk

    mannesmanntelecommunications

    18

    Number of Repetitions

    0,0%2,7% 0,3% 0,0%

    8,0%

    89.0%

    0%

    10%

    20%

    30%

    40%

    50%

    60%

    70%

    80%

    90%

    100%

    1 2 3 4 5 6

  • mmobilfunk

    mannesmanntelecommunications

    19

    Execution Time Distribution

    0%

    2%

    4%

    6%

    8%

    10%

    12%

    14%

    00:0

    0-01

    :00

    01:0

    0-02

    :00

    02:0

    0-03

    :00

    03:0

    0-04

    :00

    04:0

    0-05

    :00

    05:0

    0-06

    :00

    06:0

    0-07

    :00

    07:0

    0-08

    :00

    08:0

    0-09

    :00

    09:0

    0-10

    :00

    10:0

    0-11

    :00

    11:0

    0-12

    :00

    12:0

    0-13

    :00

    13:0

    0-14

    :00

    14:0

    0-15

    :00

    15:0

    0-16

    :00

    16:0

    0-17

    :00

    17:0

    0-18

    :00

    18:0

    0-19

    :00

    19:0

    0-20

    :00

    20:0

    0-21

    :00

    21:0

    0-22

    :00

    22:0

    0-23

    :00

    23:0

    0-24

    :00

  • mmobilfunk

    mannesmanntelecommunications

    20

    Implementation Progress

    0%

    20%

    40%

    60%

    80%

    100%

    120%

    43/9

    9

    45/9

    9

    47/9

    9

    49/9

    9

    51/9

    9

    01/0

    0

    03/0

    0

    05/0

    0

    07/0

    0

    09/0

    0

    11/0

    0

    13/0

    0

    15/0

    0

    17/0

    0

    19/0

    0

    21/0

    0

    Calendar Week

    Coded TCsVerified TCs

  • mmobilfunk

    mannesmanntelecommunications

    21

    Grade of Automation: Test Cases

    2

Recommended

View more >