1

TESTNET- Integration of Testing Methodologies

Ana Cavalli (GET/INT)Manuel Nunez (Univ. Complutense-Madrid)Edgardo Montes de Oca (Consultant)

2

European Commission 6th FP

EC initiated the 6th FP with new instruments to organize research activities:

Integrated Projects (IP)

Network of Excellence (NoE)

The programme started in 2002, and we have submitted an expression of interest to constitute a NoE, TESTNET, in June 2002

3

Networks of Excellence

Call of the IST priority: June 17 2002 Submission deadline:October 15 2003Budget: €1070 Millions

All documents for IST can be found at:

http://fp6.cordis.lu/fp6/

NoE description:

http://www.cordis.lu/fp6/instruments.htm

4

A network of excellence (NoE)

What is a NoE?

A set of research institutions (academic or industrial) excellent in a domain, that work together in a research topic

This set has the critical mass to carry out the proposed activities

5

A durable structuring

Structuring effect:

Reduce overlapping in research by coordination,

e.g. “Virtual Center of Excellence” concept

Sustainable:

Long term commitments from the institutes, labs,…;

e.g. by structuring their research with respect to the NoE

Key Elements

6

Network of Excellence (NoE): build around a “Joint

Program of Activities” (JPA)

Aimed primarily at creating durable integration of the

research capacities of network partners

While at the same time advancing knowledge on the topic

Key Elements

7

Strengthen excellence by tackling fragmentation

ExcellencePublications, conference organization, awards, patents, expertise for public and private organizations, project leadership, …

Integration Show how the network will be structured,e.g. how coordination and cooperation is facilitated

Key Elements

8

Structuring Effect

Partner 2

Partner 3

Partner 4

Partner 3Partner 1

Part

ner 1

Partner 2 Partner 4

Integration Activities Binding

Today Coordination NoE Target

9

Integrating activities coordination and adaptation of participants research

activities sharing of platforms/tools/facilities: coordination of

software development, integration of tools, joint use of infrastructures, …

staff mobility and exchanges integrated management of knowledge reinforced electronic communication networks:

development of tools and infrastructure to share information

Virtual Center of Excellence

The Heart of The NoE

Joint Program of Activities

10

Jointly executed research a strong program of joint research: Define joint research

projects, establish working groups, … co-advising of thesis, e.g. Ph.Ds, M.Sc., … joint participation in Integrated Projects

The Heart of The NoE

Joint Program of Activities

11

Activities to spread excellence training of researchers and other key staff: Summer

schools, joint lectures, etc … dissemination and communication; networking

activities to assist knowledge transfer: workshops, conferences, participating in standard bodies, etc …

Training SMEs researchers Building relationship with other groups

The Heart of The NoE

Joint Program of Activities

12

Hints for Success Address all evaluation criteria

Excellence of the participants Relevance to the objectives of the program Potential impact on strengthening Europe’s excellence Extent, depth and lasting nature of the integration Joint Program of Activities Contribution to spreading excellence Organization, Management and governance of the network

An NoE should last after the contract: It should be more difficult to disband it than to continue it

13

Hints for Success

Research as such is not funded:

show links with other projects, e.g. European or national projects, grants

Negotiation will be difficult:

financing, technical content

Develop a clear Consortium agreement

Review your ideas with Commission officials

14

What is the critical mass?

Concerning expertise:

the network must assemble the critical mass needed to achieve ambitious goals

gather the major European researchers in this area

Concerning duration of Community support:

5 years or more, if needed, to create a durable integration

Critical mass doesn‘t mean everybody

15

Financial Regime (General Principles)

Concept of an incentive to integration

will take the form of a fixed “grant for integration”

calculated on basis of total number of researchers

Example (by European commission):

a network of 200 researchers supported over 5 years would receive a fixed grant of €17.5 million

approx. 17.5 K€ per researcher

16

Flexibility and autonomy

Each year the NoE proposes a detailed program for the forthcoming 18 months

Freedom to distribute funds between partners and between activities

The partnership may itself decide to take in new partners (though without additional financing)

The Commission may decide to launch calls to add partners (with additional financing)

17

Why TESTNET?

Humans have created complex systems that put themselves and their environment at risk

Human-made catastrophes have increased in frequency and magnitude with industrialization

Testing, the process of checking that a system possesses a set of desired properties and behavior,

contributes to reduce the risk of failures and to guarantee the quality and reliability of the software used

is an integral part of innovation, production and operation of systems

18

Why TESTNET?

Testing is already a flourishing area with the active participation of a large community of researchers and experts

There is also an awareness of an increasing need:

to automate the testing activity in all application areas

that testing should be taken in consideration in all phases of the software and system life cycle

19

Different communities

Different testing communities use different methods: testing of software and testing of communication systems

Almost no interactions between them

Both communities have realized the need to unify their research efforts to define a common framework

20

Application in industry

Existing techniques and tools for testing: are not adequately applied in industry

Testing activities are: still very expensive relegated to the last stages of development, so when time and resources start

to be scarce, they are often partially sacrificed

Make the testing activities more cost/effective: by developing better tools by defining techniques seamlessly integrated into the development process

21

Necessity of a NoE

Research is reflected in: international conferences:IFIP TESTCOM, IFIP-FORTE/PSTV, IEEE-ICNP, ASE,

ISSTA, IEEE-COMPSAC, IFIP CFIP, etc. standard bodies: ETSI, ISEB, ITU-T, ISO, IETF

But these instruments lack the structuring effect that would arise from the organization of an European NoE

These are the reasons why we propose TESTNET: to undertake the past, on-going and necessary future activities in the area of

testing as well as the importance of it in all applications domains (communication services, nuclear power, transport, aeronautics, etc.)

22

TESTNET

The network will mobilise researchers and experts in the field of testing mobile communications, consumer electronics, embedded software and systems, aeronautics and nuclear energy

To assure that Europe remains a leader in the field and continue to offer innovative, integrated and adapted methods and techniques to tackle the growing complexity of software and communication systems

To prevent economic failures (maintaining cost-efficiency) or social catastrophes (maintaining reliability)

23

Issues to be addressed

To automate the testing procedure to reduce time to market and to improve software quality at the same time

To define best practices that take into account different existing techniques in different domains that are applicable for industry

Testing is not always taken into consideration in industrial environments as it should be

Universities and other educational institutions often do not include it in their curricula

24

TESTNET Objectives

Co-ordinate and bring together major European research groups from different testing communities and domains to strengthen and develop the collaboration in the area at the European level

by stimulating technological transfer and sharing of experience

by coordinating research activities and defining its roadmap (medium and long term strategy)

by adapting activities to integrate their competencies taking advantage of the multi-disciplinary character of participants

25

TESTNET Objectives

Integrate testing through all software life cycle

Promote testing as a well-defined and important activity in education, research, software engineering and industry

Promote the integration of different testing techniques and development tools. Stimulate the creation of open tool environments so that different test and development tools can be seamlessly connected

Promote innovation and take-up of new technologies through small and medium enterprises (SMEs) and through the organisation of Integrated Projects

26

Planned activities

Creation of working groups: WG1: Interoperability and conformance testing platforms and application to

heterogeneous networks (e.g., next generation communication networks, mobile and wireless communication systems)

WG2: Testing of embedded software and fault-tolerant and safety-critical systems (e.g., aeronautics and railway control)

WG3: Languages for systems and test description (e.g., UML, SDL, B, Z, MSC, TTCN)

WG4: Techniques and processes for the new paradigms of software development (e.g., component-based software testing and testing of product lines)

WG5: Theoretical foundations of testing (e.g., measures for quality of testing and quality of software)

WG6: Platforms for testing of industrial applications

27

Cross-working group actions

Roadmap for testing: elaboration of a strategy for research and development

Education and training: elaboration of new curricula to integrate testing methodologies into university degrees

Elaborate best practices applicable globally, but also specifically for each area, for obtaining reliable software and hardware

Participation in the definition of standards: ETSI, ISO, ITU-T, ISEB, IETF, etc

Promotion of common activities with enterprises, with special emphasis on SME. In particular, creation of academia-industry laboratories in the countries having the necessary critical mass

28

Other complementary activities

Organisation of joint workshops (i.e. including all the working groups) as well as open workshops (i.e. including participants outside the network).

Organisation of summer schools covering the different aspects and communities of testing.

29

Partners

40 academic partners

20 entreprises

Countries: Denmark, Finland, France, Germany, Great Britain, Hungary, Israel, Italy, The Netherlands, Poland, Russia, Spain, Sweden

30

Other information

Strong competition

the axe « Open development platforms for software and services » has 55 Meuros

they expect to finance 2-3 IP ’s and 2-3 NoE ’s

High level management is expected

a key point to obtain the financial support

31

Other information

The Joint Program of Activities is the most important part

they expect a real and strong commitment of the participants organizations

national projects should be integrated in the network

international projects with non European countries should collaborate with the network

32

Conclusion

Our strong points:

integration of testing of software and testing of communication systems communities

excellence of the consortium: includes theoretical and practical experience