ETSI Testing Projects ICT-OSA/Parlay Workshop

Brazil, March 2006

Anthony Wiles

Manager

ETSI Protocol and Testing Competence Centre

ETSI Testing Projects - ICT Workshop, Brazil March 2006 3

The ETSI PTCC

Protocol and Testing Competence Centre In-house team of experts providing direct support to ETSI

Technical Bodies (for over 12 years) Development of protocols and profiles and the application

of modern specification techniques (35%) ‘Standards Engineering’

Validation of standards (5%) E.g., testbed activities (IPv6, SIP/H.323)

Development of test specifications (60%) Conformance tests Interoperability tests Network Integration tests Others (PHY, performance …)

http://www.etsi.org/ptcc

ETSI Testing Projects - ICT Workshop, Brazil March 2006 5

Typical ETSI Test Specifications

Cellular: GSM and 3G (UMTS) terminals Now includes IMS

WiFi: HiperMAN, HiperACCESS, WiMax

VoIP: H.323, SIP, SIGTRAN

Service Creation: OSA/Parlay (API, IDL)

IPv6: Core, Security, Mobility, v4-v6

Cordless phones: DECT

Radio communications: TETRA, DMR

Access terminals: FSK, SMS

Broadband: ISDN, DSL

Smartcards: Readers, cards, security modules

Intelligent Transport Systems (ITS): DSRC

Future: More Security, more ITS, GRID ...

ETSI Testing Projects - ICT Workshop, Brazil March 2006 6

IPv6 Test SpecificationsSTF 276

Joint ETSI – eEurope project

Supported by the ETSI membership Mainly vendors Require IPv6 test specifications using ETSI methods

and test languages Designed to be part of 3GPP and TISPAN NGN IPv6

testing strategy (where applicable)

European Commission: Mandate for IPv6 device interoperability Amplify Europe’s voice in IPv6 testing

Liaising with the IPv6 Logo Program

For more info visit: www.ipt.etsi.org

ETSI Testing Projects - ICT Workshop, Brazil March 2006 7

Two Phases (2005 – 2007)

Phase 1: Core IPv6 Total value approx. 0,5M€ Jan – Dec 2005

Phase2: IPSec, Mobile IPv6, IPv4-6 Transitioning Total value approx. 1,0M€ Jan 2006 – May 2007

Provided as Funding Provision of experts Hosting costs Travel

All technical work done at ETSI by STF276 Currently 11 participants in the STF

ETSI Testing Projects - ICT Workshop, Brazil March 2006 8

Outputs

IPv6 Testing Framework Published (update due end 2006)

IPv6 Requirements Catalogue On line as part of IPT Library

Test specifications Interoperability test specifications Conformance test specifications Library of TTCN-3 test code freely available On-line, extensible and reusable

All tests are validated (probably 30% of the effort) On ETSI IPv6 Testbed By member companies

For more info visit: www.ipt.etsi.org

ETSI Testing Projects - ICT Workshop, Brazil March 2006 9

Results Phase 1

Over 1100 Core IPv6 Requirements identified Including 3GPP

300 Core IPv6 conformance tests fully validated

120 Core IPv6 Interoperability Tests fully validated

All these test specifications including TTCN-3 code are free and open source

ETSI Testing Projects - ICT Workshop, Brazil March 2006 10

STF276@Work

ETSI Testing Projects - ICT Workshop, Brazil March 2006 11

3GPP UMTS (terminal) Test Specifications (STF 160)

Nobody can make money without interoperable handsets (UE)! UE Interoperability towards access technologies is the key issue

for mobility

Conformant UE = Interoperability (of UE in 3G networks) Good experience from GSM

Solution: conformance testing Very detailed test specifications Based on the 3GPP core specs Before commercial products enter the market Voluntary declaration, but certified

• One of the most important UE quality benchmarks

ETSI Testing Projects - ICT Workshop, Brazil March 2006 12

Methodology for 3G Testing

Test Cases

(TTCN)

34.123-3, 51.010-5

PICS (capability

applicability)

34.123-2, 51.010-2

Test Purposes,34.123-1 Descript (prose), 34.108,

51.010-1

Test Report

logging and

analysis

Core specs

25.331, 25.321, 2 24.008, 44.018, 44.060

implementation

testing

SSMobile Under Test

compilation

Executable Test Suite

(e.g., C++)

ETSI Testing Projects - ICT Workshop, Brazil March 2006 13

3G Test Industry Commitment is Vital

The TTCN test cases are implemented in the System Simulators (SS) on different platforms Must secure the commitment of the SS vendors

SS vendors organise verification and regression testing Each and every test case is validated Bug-free tests have a high real value Important to ensure that a TC will execute correctly even

after updates Early identification of clients who deploy the tests

Prioritising TCs Scheduling the deployment Certifications

ETSI Testing Projects - ICT Workshop, Brazil March 2006 14

Funding

Test specs developed at ETSI by STF160 Duration of several years

Shared costs to ensure the funding 3GPP funding from Organisation Partners

Main source, approx 1M€ (61 mm) /year)

Voluntary contributions from the marketing partners

Range from 65k€ - 150k€ (5 – 12 mm) /year

Man-power (free-of-charge) Range from 15 - 18 mm /year

ETSI Testing Projects - ICT Workshop, Brazil March 2006 15

Typical 3GPP UMTS UE Tests

Protocol and signalling tests via radio interface Largest part of the testing process Full conformance tests

USIM functional tests Identification, PIN, account charging, network priority, phonebook

UICC tests (SIM electrical test) power on/off, 1.8v/3v detection, presence detection bit/character transmitting duration, command processing and error

handling Acoustic tests

Loudness rating, idle noise, echo … Radio Frequency & Radio Resource Management tests

Characteristics and performance tests of transmitter & receiver

ETSI Testing Projects - ICT Workshop, Brazil March 2006 16

Users of 3GPP Tests

Global Certification Forum (GCF) PCS Type Certification Review Board (PTCRB )

GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs

TD-SCDMA Industry Alliance (TDIA) TDIA is the major clients for TD-SCDMA (LCR TDD) test

specs

GCF WI-10 selected 400 TCs as high priority Divided in 4 packages Progressing sequentially one by one (milestone 80%)

UMTS mobile certification will be started if 80% of each package are validated

ETSI Testing Projects - ICT Workshop, Brazil March 2006 17

Approval Process

Test cases are approved by e-mail No quality compromise Two week consultation period TCs are approvedone by one

Each test must run on at least one SS platform against two independent commercial mobiles Log files and radio conditions as support information

for approval

Reason for this TTCN TC, SS and UE are developed at the same time None of them is perfect

ETSI Testing Projects - ICT Workshop, Brazil March 2006 18

Results

400 TCs in 8 ATSs, 25 MB code Platform independent Running on > 4 platforms

We have reached the GCF 80% validated milestone 3G mobile certification starting at beginning of 2005 The light is green for mass 3G mobiles entering the markets 4 types of 3G mobiles are ready to go

Begun the real 3G test business 4 validated 3G SS platforms available in the markets 4 test houses in Europe opening 3G test services 3G TTCN experts becoming hot in the labour market

We know the price of each 3G coded TC 10 k€ /TC 3G investment for 4 years: 4 mil.€ for 400 TCs

All TTCN code can be downloaded free of charge Code is continuously maintained and delivered

Every three weeks for 3G members Every three months for everybody

ETSI Testing Projects - ICT Workshop, Brazil March 2006 19

VoIP-related Test SpecificationsSTF 269, STF 270 and others

SIP (RFC 3261) Over 400 conformance test cases Fully validated (executables available on commercial tools) STF 270 is doing the maintenance

H.225 About 300 conformance test cases Fully validated (executables available on commercial tools) No ongoing maintenance

SIGTRAN Test purposes for M3UA, M2UA, SCTP TTCN-3 available commercially (not ETSI)

IMS SIP (together with SDP) 3GPP RAN5 (STF 160) TTCN-3 will be used Work started March 2006 Based on the generic SIP test suite (see above) augmented to

concentrate on 3GPP SIP specific requirements

ETSI Testing Projects - ICT Workshop, Brazil March 2006 20

ETSI – WiMAX Test Specifications STF 252

WiMAX Forum Board meeting in January 2005 Agreed to harmonize the WiMAX/HiperMAN base

specifications Agreed to harmonize the WiMAX/HiperMAN test

specifications Agreed to exploit ETSI’s testing expertise Agreed to accelerate the work using the ETSI STFs Agreed that validation of the test specifications needed

• complex interplay of all parties involved• WiMAX Forum, ETSI BRAN, equipment vendors, ETSI STF,

test tool developers, test labs, etc.

ETSI WiMAX Co-operation agreement signed in 2005

ETSI Testing Projects - ICT Workshop, Brazil March 2006 21

Results

Wave 1: Joint test specifications approved and published Completed Dec. 2005 PICS – Protocol Implementation Conformance Statement TSS&TP – Test Suite Structure and Test Purposes ATS – Approx 60 test cases implemented, validated and used

in certification

First WiMAX products have been certified

Wave 2: New drafts reaching approval PICS – new updated version with corrections TSS&TP – new draft with close to 1400 Test Purposes ATS – new draft with close to 300 additional test cases

developed and entering validation

ETSI Testing Projects - ICT Workshop, Brazil March 2006 22

IPT LibrarySTF 295

IP Testing Library Developed by ETSI TC MTS All MTS IP and IP-related conformance and IOP tests PICS, TPs, Test Suites etc. Due to be published April 2006

SIP, IPv6, H.323, H.248

Includes IPv6 Requirements Catalogue Linked to TPs and TTCN-3 code

Powerful search and selection possibilities E.g., allows test suite tailored to specific search criteria

ETSI Testing Projects - ICT Workshop, Brazil March 2006 23

Screen Shot of IPT Library

ETSI Testing Projects - ICT Workshop, Brazil March 2006 24

PlugtestsTM ETSI Interoperability Service

Created in 1999, specialized in the organization oftest events

Open to all organizations Experts from ETSI and other technical communities Skilled ETSI Staff managing test events

In 6 years, 66 In 6 years, 66 EventsEvents

3500 engineers3500 engineers

1400 Companies1400 Companies

Usefulness & Usefulness & Service SatisfactionService Satisfaction

ranked at ranked at 4.35 / 54.35 / 5

ETSI Testing Projects - ICT Workshop, Brazil March 2006 25

Why?

Provides useful feedback to the standardization process

Accelerates time to market via quick product debugging

Improves both quality and features of implementations

Helps to ensure end users’ satisfaction

Reduces deployment costs

Allows networking with the market key players

COST & TIME SAVING!

ETSI Testing Projects - ICT Workshop, Brazil March 2006 26

Upcoming events in 2006

Q1 – 2006

SIM/Handset

Lawful Interception

3rd WiMAX Plugfest

Digital Identity

Q2 – 2006

IPv6 Asia

4th SQTE EU round

NGN

Mobile Application

NGN@Home

Q3 – 2006

OSA Parlay

UWB

SIGTRAN

Q4 – 2006

Human Factors (HF)

IPv6 French Summit & Interop (FR)

Powerline

RFID

ITS ecall

Thank You!

Anthony Wiles

ETSI Protocol and Testing Competence Centre

Phone +33 (0)4 92 94 43 26

e-mail ptcchelp@etsi.org

Website www.etsi.org/ptcc