4. validation - tu dresdenst.inf.tu-dresden.de/files/teaching/ss09/stii09/04-validation.pdf ·...

Fakultät Informatik, Institut für Software- und Multimediatechnik, Lehrstuhl für Softwaretechnologie

4. Validation

Prof. Dr. U. AßmannTechnische Universität DresdenInstitut für Software- und MultimediatechnikGruppe Softwaretechnologiehttp://st.inf.tu-dresden.de

Obligatory Reading

► Maciaszek Chapter 12 (is not enough)

TU Dresden, 21.04.2009

Sebastian Richly

Folie

2 von 66

Validation

Recommended Reading

► Uwe Vigenschow. Objektorientiertes Testen und Testautomatisierung in der Praxis. Konzepte, Techniken und V f h d k V l H id lb 2005 V d i l Verfahren. dpunkt-Verlag Heidelberg, 2005 Very good practical book on testing. Recommended!

► Axel Stollfuß und Christoph Gies. Raus aus dem Tal der Tränen. Hyades: Eclipse als Automated Software Quality Plattform. Eclipse Magazin 2/05. www.eclipse-magazin.de

► Bernhard Rumpe. Agile Modellierung mit UML. Springer, 2004. Chapter 5 Grundlagen des Testens, Chapter 6 Modellbasierte Tests

► Robert Binder. Testing Object-Oriented Systems. AWL.


Sebastian Richly

Folie

3 von 66

Validation

Verification and Validation

Verification:Mathematical proof that the implementation conforms to the specification (correctness proof)Without specification no proof

Validation:Plausibility checks about correct behavior (reviewing, tests, Code Coverage Analysis)

Test: Validation of runs of the system under test (SUT) under well-known conditions, with the goal to find bugs

Defensive Programming:Defensive Programming:Programming such that less errors occur

Testing shows the presence of bugs, but never their absence (Dijkstra)


Sebastian Richly

Folie

4 von 66

Validation

Komplexitätswachstum und Fehlerrate

20 800

1977 1994

0,2

1977 1994

10

1977 1994

Anzahl Fehler auf 1000 LOC

1977 1994

Programmgröße (1000 LOC)200

160160Echte Qualitätsverbesserungen

sind nur möglich, wenn dieSteigerung der ProgrammkomplexitätSteigerung der Programmkomplexität

überkompensiert wird !

1977 1994Resultierende

absolute Fehleranzahl


Sebastian Richly

Software Krise Folie

5 von 27

Defensive Programming

Reduce errors by safer programming...


Sebastian Richly

Folie

6 von 66

Validation

Defensive Programming: Contract Checking with Layers Around Procedures

Assertions in procedures can be used to specify tests (contract checks). Usually, these are layered around the core functionality of h dthe procedure

Programming style of “analyse-and-stop”: analyze the arguments, the surrounding of the arguments, and stop processing with exception if error occursoccursSome programming languages, e.g., Eiffel, provide contract checks in the language

Validating p econditions (ass mptions)Validating preconditions (assumptions):Single parameter contract checks

Polymorphism layer: analysis of finer typesNull check, Exception value checkRange checks on integer and real valuesState analysis: which state should we be in?Condition analysis: invariants fulfilled?Cross-relation of parameters

Object web checks (null checks in neighbors)Invariant checksPostcondition checks (guarantee)


Sebastian Richly

Folie

7 von 66

Validation

Invariant Checks

In a triangle, the sum of two sides must be larger than the third [Vigenschow]

public boolean isTriangle(double s1, double s2, double s3){return ((a+b > c) && (a+c > b) && b+c > a));

}

In a triangle-manipulating program, this is an invariant:

public void paintTriangle(Triangle t) {p p g ( g ) {// preconditionsassertTrue(t != null); assertTrue(t->s1 > 0 && t->s2 > 0 && t->s3 > 0);( );// invariant checkassertTrue(isTriangle(t->s1, t->s2, t->s3));// now paint.// p....// invariant check againassertTrue(isTriangle(t->s1, t->s2, t->s3));( g ( , , )).. postconditions...

}


Sebastian Richly

Folie

9 von 66

Validation

Implementation Pattern: Contract Layers

Contract checks should be programmed in special check-procedures so that they can be reused as contract layersAdvantage: entry layers can check contracts once, other internal layers need no longer check

paint() move() scale()p () () ()

isTriangle() isRectangle() isFigure() contract layer

paint() move() scale()


Sebastian Richly

Folie

10 von 66

Validation

Model-Based Contracts

Are specified in OCL (object constraint language), referring to an UML class diagram:

context Person.paySalary(String name)pre P1: salary >= 0 &&pre P1: salary > 0 &&

exists Company company: company.enrolled.name == namepost P2: salary = salary@pre + 100 &&

i t Cexists Company company:company.enrolled.name == name && company.budget = company.budget@pre - 100

These contracts can be generated to contract code for methods (contract instrumentation)(contract instrumentation)

Contract checker generation is task of Model-driven testing (MDT)More in special lecture


Sebastian Richly

Folie

11 von 66

Validation

Validation with Inspection and Reviews


Sebastian Richly

Folie

12 von 66

Validation

Checklists

Project managers should put up a bunch of checklists for the most important tasks of their project members and themselves

[Pilots use checklists to start their airplanes]

Question lists are a specific form of checklists to help during brainstorming and quality assurance

http://www.rspa.com/spi/chklst.html#Anchor-49575


Sebastian Richly

Folie

13 von 66

Validation

Example [Bazman]

1. Does a failure of validation on every field cause a sensible user error message?g

2. Is the user required to fix entries which have failed validation tests?

3 Have any fields got multiple validation rules and if so are all rules 3. Have any fields got multiple validation rules and if so are all rules being applied?

4. If the user enters an invalid value and clicks on the OK button (i e does not TAB off the field) is the in alid ent identified and (i.e. does not TAB off the field) is the invalid entry identified and highlighted correctly with an error message.?

5. Is validation consistently applied at screen level unless ifi ll i d fi ld l l?specifically required at field level?

6. For all numeric fields check whether negative numbers can and should be able to be entered.

7. For all numeric fields check the minimum and maximum values and also some mid-range values allowable?

8. For all character/alphanumeric fields check the field to ensure 8. For all character/alphanumeric fields check the field to ensure that there is a character limit specified and that this limit is exactly correct for the specified database size?

99. ……….


Sebastian Richly

Validation Folie

14 von 66

Internal Reviews

A colleague reads your code (you are the producer)Explain your code (walkthrough)!Check programming conventions, clarity of code, use of design patternsDetect problems, but don't solve them

A project leader should group her people to review pairsp j g p p p pDuration: 30-90 minutesProtocol: Email or formal document

Participants, time, durationName, version, variant of code sources inspectedReview issue listActions determined

A review issue database is also nice (similar to a bug tracking or requirements management system)

Bug Tracking Database http://www.mantisbt.org/ g g p // g/


Sebastian Richly

Folie

15 von 66

Validation

Pair Programming

Programming in pairsA programmerA reviewer

Change roles after some time

Psychology: Not everybody likes to program in pairsEgoless programming is desired

Pair programming is permanent reviewing


Sebastian Richly

Folie

16 von 66

Validation

External Reviews

Special review meetingPrepare meeting by distributing all relevant documentsAn unrelated colleague from another department reviews the codeA review leader (moderator) guides through the meeting

Formal protocol:pReview form is often standardized for a company

Specifications can also be reviewed (requirements specs, design specs)specs)

Find out inconsistencies with source code

Inspections differ from walkthroughsNot so oftenNot so oftenUnder management supervision

Reviews contribute to quality


Sebastian Richly

Folie

17 von 66

Validation

Audits

Most formal kind of reviewProfessional auditors (quality assurance people) from QA (q y p p ) Qdepartments, or even external companies

Producer may be absent, auditing can be done from documents alone

Audits take longer than reviewsAudits take longer than reviewsAudits contain several reviewsPlanning phase

Audits can also check the financial budgetsAudits can also check the financial budgetsAuditors check how the money was spent (time sheets, travel, labor cost, ...)


Sebastian Richly

Folie

18 von 66

Validation

A General Heuristic: Tight Feedback Loops

Software processes are highly dynamic. It is hard to pre-plan them.Install process guidelines that lead to tight feedback loops:

Feedback early, often, frequently.Better early light feedback than late thorough feedback

For reviews, this means: review early, review often.


Sebastian Richly

Folie

19 von 66

Validation

Validation with Tests


Sebastian Richly

Folie

20 von 66

Validation

The Problem

Programmers program under time pressure (on-demand)Programmers program on-demand, because programming is hardg p g , p g g

Domain problemsSpecial cases are forgottenThe effect of users is underestimated[The demo effect]A writer never finds his own bugs (Betriebsblindheit)

Tests have destructive, negative natureIt is not easy to convince oneself to be negative!Quality assurance people can ensure this, ensuring a reasonable software processQ y p p , g p


Sebastian Richly

Folie

21 von 66

Validation

The Cleanroom Model

The Cleanroom method divides the project members into programmers and testers.Developer must deliver a result almost without bugs

Testing forbidden!

Incremental processExperience with Cleanroom Model

Selby with 15 Student Teams 10 Cleanroom/5 non CleanroomSelby with 15 Student Teams, 10 Cleanroom/5 non-CleanroomCleanroom-Teams produce simpler code with more comments 81% want to use it againAll Cleanroom teams manage milestones 3 of 5 non Cleanroom teams not All Cleanroom teams manage milestones, 3 of 5 non-Cleanroom teams not. But: programmers do not have the satisfaction to run their code themselves

Only the problems with formal specificationOnly the problems with formal specification


Sebastian Richly

Folie

22 von 66

Validation

CleanRoom in the NASA

In 1987, the NASA developed a 40KLOC control program for satellites with CleanroomDistribution of project time:

4% Training staff in Cleanroom, 33% design18% Implementation (45% writing, 55% reviewing), 27% Testing, 22% Other things (e.g., meetings)

Increase in productivity 69%. Reduction of error rate 45%. Resource reduction 60-80% Resource reduction 60-80%. Developers, prohibited to test their code, read intensively. This catches many bugs (~30 bugs for 1KLOC).


Sebastian Richly

Folie

23 von 66

Validation

Microsoft's Software Process “Synchronize and Stabilize”

.. is a CleanRoom process:Microsoft builds software until 12:00 (synchronization)( y )In the afternoon, test suites are run by the test teams, i.e., separation of programmer and testerProgrammers get feedback the next dayProgrammers get feedback the next day[IBM tests in China]


Sebastian Richly

Folie

24 von 66

Validation

Standard Test Process as Right Wing of the V Model

Tests should be done bottom-up Defensive programming (contracts)Then unit tests (class tests)Then component tests (EJB, JSP etc)Then the system

Acceptance test

Then the beta testThen acceptance test

Analysis

Installation, beta test

Test cases

Architectural design

System testsTest casesg

Det. Design Component tests integration tests

Test cases

Implementiation Contracts, class tests


Sebastian Richly

Folie

25 von 66

Validation

Acceptance Test

Acceptance test cases are part of the SRSAre checked by the customer for fulfillment of the contractWithout passing, no money!

Acceptance tests are system testsRun after system deploymenty p yTest entire system under loadTest also non-functional qualities

After every evolution step, all acceptance test cases have to be After every evolution step, all acceptance test cases have to be repeatedRegression test:

Should-Be-outputs are compared with actual outputsShould Be outputs are compared with actual outputsConsists of a set of test cases (a test suite)


Sebastian Richly

Folie

26 von 66

Validation

Deriving Test Cases from Functional Specifications

Most often, acceptance tests are derived from use cases, function trees, or business modelsEvery use case yields at least one acceptance test case

For every test case, a test driver is written

Terminverwaltung

Organize

Organisator Move

gmeeting

Organisator

Find out about unused Room

Movemeeting

rooms managerPlan

personal date

Teammember


Sebastian Richly

Folie

27 von 66

Validation

Regression Tests

How to be sure that a change did not introduce errors...


Sebastian Richly

Folie

28 von 66

Validation

Regression Tests as Diffs on Outputs of Subsequent Versions

Regression tests are operators that check semantic identity between versions that have similar input/output relation

Regression TestTime Regression Test Enhancement Test


Sebastian Richly

Folie

29 von 66

Validation

A Poor Man's Regression Testing Environment

The UNIX tool diff is able to textually compare files and directories (recursively) 1.1

System under test (sut)

sut.output1

sut.output2

sut.input1

sut.input2 test (sut) sut.output2

sut.output3

sut.input2

sut.input3

diff

1 1subdir 0.9

1.1

subdir1.0sut.output1sut output1

sut.output2sut.output1

sut output2

sut.output1

sut.output2

diffsut.output2sut.output2

sut.output3sut.output3


Sebastian Richly

Folie

30 von 66

Validation

Diff Listings for Regression Tests

Diff shows lines that have been removed from first file (<) “went out” and added to the second (>) “came in”

diff file~1.1 file~1.0

< if (threshold < 0.9) stopPowerPlant();> if (threshold > 0.9) stopPowerPlant();

-- compares entire directory to subdirectory 1.0

diff -rq . 1.0diff rq . 1.0 ./subdir/f.c:< if (threshold < 0.9) stopPowerPlant();> if (threshold > 0.9) stopPowerPlant();

Diff invocations are wired together for test suites with shell scripts or makefiles

(t es o d 0.9) stop o e a t();

On windows, use cygwin shell (www.cygwin.org)


Sebastian Richly

Folie

31 von 66

Validation

Variants of Regression Tests

[Binder] distinguishes 5 coverage patterns for regression tests:

All (exhaustive): bestRisky use casesRe-test profile: profile code and re execute tests on most executed Re test profile: profile code and re execute tests on most executed codeChanged code (code that changed between versions)Ch d d d ll d d t dChanged code and all dependent code


Sebastian Richly

Folie

32 von 66

Validation

Importance of Regression tests

Regression tests are the most important mechanism to ensure quality if a product appears in subsequent versions

Without regression test, no quality

Companies sell test data suites for regression testsValidation suites for compilers (e.g., Ada or XML)Test data generators that generate test data suites from grammars

The more elaborated your regression test method is, the better your product will bey p


Sebastian Richly

Folie

33 von 66

Validation

GUI Regression Test with Capture/Replay Tools

A capture tool allows for recording user actions at a GUIRecording in macros or scripts

A replay tool reads the scripts and generates events that are fed into the system

The replay tool can be started in batch, i.e., can be integrated into a regression test p y , , g gsuiteHence, the GUI can be regression tested

Capture/replay tools can record the most important workflows p p y phow systems are used

Opening documents, closing, savingException situationsEven big office suites seem not to be tested with capture/replay tools

Examples:Mercury Interactive WinRunner www.mercuryinteractive.dey yRational Robot www-306.ibm.com/software/rational Abbot - http://abbot.sourceforge.net/doc/overview.shtmlJellytools is a JUnit-derivative for test of Swing-GUIy gweb2test from Leipzig http://www.saxxess.com/content/14615.htm


Sebastian Richly

Folie

34 von 66

Validation

The JUnit Regression-Test Framework

JUnit www.JUnit.org is a simple framework for regression tests of classes and components

Run test cases and test suites Plugins for IDE available (Eclipse, JBuilder, ..) Available for several languages

Nunit for .NETCppUnit for C++

Test result classification:Failure (Zusicherung wird zur Laufzeit verletzt)Error (Unvorhergesehenes Ereignis, z.B. Absturz)Ok

JUnit supports permanent regression testing during development:All test cases are collected and repeated over and over again


Sebastian Richly

Folie

36 von 66

Validation

JUnit Framework

run(TestResult)

TestComponent( )

TestCase

String name TestSuite

Composite

L f

run(TestResult)runTest()setUp()

String name

run(TestResult)add()

TestSuite

Composite

LeafTestResult

setUp()tearDown() setUp();

runTest();tearDown();TextTestResult

MyTestCase

UITestResult

... reason why failedtest data T l t

.. init of test datarunTest()setUp()tearDown()

... test data TemplateMethod

tearDown()


Sebastian Richly

Folie

37 von 66

Validation

Example: Test of Dates

// A class for standard representation of dates// A class for standard representation of dates.public class Date { public int day; public int month; public int

year;year;public Date(String date) {day = parseDay(date);month = parseMonth(date);month = parseMonth(date);year = parseYear(date);

}public int equals(Date d) {public int equals(Date d) {return day == d.day &&

year == d.year &&month == d.month;month d.month;

}}


Sebastian Richly

Folie

38 von 66

Validation

New TestCase

TestCases are methods, start with prefix “test”Test cases contain test data in a fixtureProblem: test data is integrated with test case

public class DateTestCase extends TestCase { p {Date d1; Date d2;Date d3; int length = 42;g ;

protected int setUp() {d1 = new Date(„1. Januar 2006“);d2 = new Date(„01/01/2006“);d3 = new Date(„January 1st, 2006“);d3 new Date(„January 1st, 2006 );

}public int testDate1() {assert(d1.equals(d2));assert(d2.equals(d3));assert(d2.equals(d3));assert(d3.equals(d1));.... more to say here ....

}public int testDate2() {public int testDate2() {.... more to say here ....

}}


Sebastian Richly

Folie

39 von 66

Validation

Example: Test Case in Nunit (.Net) and JUnit 4.0

Tags are used to indicate test case classes, set up, tear down methods

The tags are metadata that convey additional semantics for the items


Sebastian Richly

Folie

40 von 66

Validation

Use of Test Case Classes

A test case is created by calling the constructorThe constructor finds the method of the given method name and gprepares it for call (reflection)The run() method starts the test case with the fixture and returns a test resulta test resultIn case of a failure, an exception is raised

public class TestApplication {...TestCase tc = new DateTestCase(„testDate1“);TestResult tr = tc.run();

}


Sebastian Richly

Folie

41 von 66

Validation

Test Suites

A test suite is a collection of test cases (pattern Composite)

public class TestApplication {...TestCase tc = new DateTestCase(„testDate1“);TestCase tc2 = new DateTestCase(„testDate2“);T tS it it T tS it ()TestSuite suite = new TestSuite();suite.addTest(tc);suite.addTest(tc2);T tR lt t it ()TestResult tr = suite.run();// Nested test suitesTestSuite subsuite = new TestSuite();

fill subsuite... fill subsuite ...suite.addTest(subsuite);TestResult tr = suite.run();

}}


Sebastian Richly

Folie

42 von 66

Validation

TestRunner GUI

The classes JUnit.awtui.TestRunner, JUnit.swingui.TestRunner are simple GUIs to present test resultsTest suites can be given the class object of a test case, and it finds the test case methods on its own (reflection)

public class TestApplication {public static Test doSuite() {// Abb i ti t t ll T tC bj t// Abbreviation to create all TestCase objects// in a suiteTestSuite suite = new TestSuite(DateTestCase.class);

}}// Starte the GUI with the doSuite suitepublic static main () {JUnit awtui TestRunner run(doSuite());JUnit.awtui.TestRunner.run(doSuite());

}}


Sebastian Richly

Folie

43 von 66

Validation

FIT Testing Framework

FIT is an acceptance and regression testing framework

A software testing tool designed for customers with limited IT knowledge

Test cases can be specified in tables• Wiki• Excel• Excel• HTML• DOC• .….

Fit test tables are easy to be read and written by customer


Sebastian Richly

Folie

44 von 66

Validation

FIT Testing Framework

Story Tests

Parse input and invoke methodsthrought reflection

FitRunner to start the test (Command line)

Can be combined with GUI robots like Abbotbbot


Sebastian Richly

Validation Folie

45 von 66

Requirements Analysis

Stakeholder AnalysisFeasibility Study Stakeholder Analysis(Beteiligtenanalyse)

Feasibility Study(Lastenheft)

Domain Analysis(Domain concepts) Problem Analysis

Goal Analysis

Quality Analysis

“real” requirements analysis

Function Analysis

ProjectTask Planning

Quality Analysis(Non-functional Reqs)

Other Analysis

Task Planning

Acceptance CriteriaAnalysis Acceptance Test


Sebastian Richly

Validation Folie

46 von 66

Tests and Tutorials

Some test cases can be written in a user-friendly style (tutorial test cases). If they are enriched with explanations, tutorial threads resultHence, sort out some test cases for tutorial test cases[Java documentation][Java documentation]


Sebastian Richly

Folie

47 von 66

Validation

Test-First Development (Test-DrivenDevelopment)

Iterate:First, fix the interface of a methodSecond, write a test case against the interfaceThird, program method. Fourth, Run test case. If test case works, add it to the current test suite, ,

Add a testAdd a test

Run testsRun tests-> seefailure-> seefailure

RefactorRefactor

Run theRun the Write somecode

Write somecode

Run thetests ->

see all pass

Run thetests ->

see all pass


Sebastian Richly

Folie

49 von 66

Validation

TDD

AdvantagesPermanent regression test (test data integrated)Permanent regression test (test data integrated)Stable extension of the code: no big bang test, collection of test cases always runningFunctionality so far can always be demonstratedFunctionality so far can always be demonstrated

TDD is like automating the reviewer: the test case plays the role of the criticizing colleague!


Sebastian Richly

Validation Folie

50 von 66

Separation of Test Data and Test Cases

Instead of fixing the test data in a fixture, the test data can be separated from the application. Advantages:

Test data can be specified symbolically, instead of using constructor expressionsTest data can be persistent in files or in databasesTest data can be shared with other products in the product line

Disadvantages:Database must be maintained together with code (versioning)g ( g)

Example:Big compiler test suites (e.g., Ada)Database test suitesDatabase test suites


Sebastian Richly

Folie

52 von 66

Validation

Stubs and Co

Stub: Empty implementations behind the interface

Dummy: Simple, restricted simulation of the interface functionality

Mock: Dummy that also checks the protocol of the class-under-testMock: Dummy that also checks the protocol of the class under test


Sebastian Richly

Folie

53 von 66

Validation

Modality of Classes

Non-modal class (stateless)• no call protocol over method set

Uni-modal class (stateful)• call protocol exists (described by a Chomsky language: state chart, context free

language, context-sensitive, Turing-complete)g g g

Quasi-modal class (stateful with restrictions)• The class has additional semantic restrictions (e.g., limited buffer size)

Modal classModal class• business rules describe the call protocol. For instance, some accounts cannot

be overcharged, others until a certain credit limit


Sebastian Richly

Folie

54 von 66

Validation

Mock Object for (Uni-)Modal Classes

Mock objects simulate a modal class-under-test, implementing the life-cycle protocolIf the CUT is a unimodal class, with an underlying state chart, the test driver should test all paths in the state chart

The mock must check whether all state transitions are done right

<<mock>>plane

Test case tests:Path 1: parking->starting-> planePath 1: parking->starting-> flying->landing->parkingPath 2: parking->starting-> landing >parking (emergency

GO/rollouthalt

parking

landing->parking (emergency path)

Driver checks that after each Exception/ t ()

halt/halt()

starting

method that is called for a transition, the right state is reached

WheelsOff/takeOff()

/stop()landing

Mock object implements state transitions

PermitToLand/land()

flying


Sebastian Richly

Folie

55 von 66

Validation

Mock Object

Public class PlaneMock extends MockObject {int state;public enum { parking, starting, flying, landing };public PlaneMock() {

state = parking;}}

public class PlaneTestCase extends TestCase {

planeparking

pMock = new PlaneMock();public void setUp() { .. }public void tearDown() { .. }public void testPath1(){

GO/rollout

E ti

halt/halt()

starting

ppMock.rollout();assertEquals(pMock.starting,pMock.getState());pMock.takeOff();assertEquals(pMock.flying,pMock.getState());

WheelsOff/takeOff()

Exception/stop()

landing

startingq (p y g,p g ());pMock.land();assertEquals(pMock.landing,pMock.getState());pMock.halt();assertEquals(pMock.parking,pMock.getState());

PermitToLand/land()

flyingq (p p g,p g ());

public void testPath2() {..}}


Sebastian Richly

Folie

56 von 66

Validation

Model-Based Testing


Sebastian Richly

Folie

57 von 66

Validation

The UML Testing Profile UTP

The concepts of JUnit can be modeled in a class diagramThe OMG has standardized a UML profile (an extension of the UML p (metamodel) providing the concepts of JUnit

UTP is a collection of stereotypes (<<TestCase>>, <<TestSuite>>) and of tagged values (<<setUp>>, <<tearDown>>) that can mark up class diagrams

Tagged values are the same concept as C# attributes or Java Xdoclets

Test cases and suites can be specified while modellingCode of test cases can be generated from a CASE toolg


Sebastian Richly

Folie

58 von 66

Validation

UTP

Realized testing profile as:• UML 2.0 meta modelUML 2.0 meta model• MOF(Meta Object Facility) meta model

Mapping to existing test infrastructuresMapping to existing test infrastructures• Test Control Notation TTCN-3• JUnit

Usage in MDA Stack PIM

PSMPSM

Code


Sebastian Richly

Validation Folie

60 von 66

Eclipse-Based Test Platforms


Sebastian Richly

Folie

61 von 66

Validation

Eclipse Supports Many Test Platform Plugins

Hyades www.eclipse.org/test-and-performance Test Capture-Replay framework for web and other applications

Http requests can be recorded, generated into JUnit test case classes, afterwards replayedUses UTP to specify test cases

A Remove-Access-Controller mediates between Eclipse and the SUTTest data can be stored in data poolsLog-file analysis based on the Common-Base-Event format of IBM

Solex http proxy logger www.sf.net/projects/solex Scapa stress test www.scapatech.com HttpUnit, htmlUnit extensions of JUnit for test of web applications p , pp

httpunit.sf.nethtmlunit.sf.net


Sebastian Richly

Folie

62 von 66

Validation

TPTP

TPTP Platform Project• Covers the common infrastructure in the areas of user interface, EMF based Covers the common infrastructure in the areas of user interface, EMF based

data models, data collection and communications control, remote execution environments and extension points

TPTP Monitoring Tools Projectg j• Collects, analyzes, aggregates and visualizes data that can be captured in the

log and statistical models.

TPTP Testing Tools ProjectTPTP Testing Tools Project• Provides specializations of the platform for testing and extensible tools for

specific testing environments, initially 3 test environments: JUnit, manual and URL testing.g

TPTP Tracing and Profiling Tools Project• Extends the platform with specific data collection for Java and distributed

applications that populate the common trace mode, also viewers and analysis pp p p , yservices


Sebastian Richly

Validation Folie

63 von 66

What Have We Learned?

Separation of reviewer and producer is importantDefensive programming is goodp g g gTest-first development produces stable productsWithout regression tests, no qualityMock classes sim late classes nde test eali ing thei life c cle Mock classes simulate classes-under-test, realizing their life-cycle protocolTest tools, e.g., on the Eclipse platform, help to automate testing of

li i l b li iapplications, also web applications


Sebastian Richly

Folie

65 von 66

Validation

The End


Sebastian Richly

Folie

66 von 66

Validation

4. validation - tu dresdenst.inf.tu-dresden.de/files/teaching/ss09/stii09/04-validation.pdf ·...

Documents