testing team exercise have each team member contribute answers: –do you test your code? if no, why...

29
Testing Team exercise • Have each team member contribute answers: – Do you test your code? If no, why not? If yes: • When? • How? • How often? – What is your team’s test plan? You have one, right? :-)

Post on 21-Dec-2015

215 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

TestingTeam exercise

• Have each team member contribute answers:– Do you test your code? If no, why not? If

yes:• When?• How?• How often?

– What is your team’s test plan? You have one, right? :-)

Page 2: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Some TLAs(Three Letter Acronyms)

• SQA– Software Quality Assurance

• many different activities (including good communication, good design, etc)

• V&V– Verification and Validation

• Part of SQA• Verification: are we building the product right?• Validation: are we building the right product?

• Fault detection– Reviews: Walkthroughs and Inspections– Testing

• Debugging– what you do once you’ve established that there is a bug

Page 3: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Types of testing

• Unit testing• Integration testing• Functional testing• Acceptance testing• Pilot testing• Performance testing• Installation testing• Usability testing

Page 4: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Who does the testing?

• Developers

• Testers

• Customers

Page 5: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Unit tests

• Written by developers

• Tests individual components

• In TDD (another TLA) unit tests are written before code

• Automated test frameworks– JUnit– CPPUnit– etc.

Page 6: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Integration testing

• Testing of components when brought together

• Different strategies:– big bang: test all components together at once– bottom-up: test layer by layer, from bottom up– top-down: test layer by layer, from top (UI) down

• Unintegrated components are simulated by stubs during integration testing

Page 7: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Functional testing

• A whole-system test

• Also called “requirements testing”

• Uncovers differences between use-case model and observed system behavior

Page 8: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Acceptance testing

• Benchmark testing– testing against a specific set of test cases

• Competitor testing– evaluation wrt competitor product

• Shadow testing– evaluation wrt legacy system

Page 9: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Pilot testing

• Field testing– alpha test: system is tested by users in

development environment– beta test: system is tested by users in the

deployment environment

Page 10: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Performance testing

• Stress testing – simultaneous requests

• Volume – large amounts of data

• Security testing – is it secure?

• Timing tests – are timing constraints violated?

• Recover tests – can system recover from errors?

Page 11: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Installation testing

• Testing of system in deployment environment (after beta test)

Page 12: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Usability testing

• Is user interface acceptable to users?

Page 13: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Random testing

• Series of studies done by B. Miller and colleagues

• First study: Unix utilities (50-85 utilities, depending on OS version, 6 versions)

• Second study: Unix utilities and X-Windows applications

• Third study: Windows NT applications

Page 14: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Results

• How many utilities/applications crashed or hung on random input?

Page 15: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Results

• How many utilities/applications crashed or hung on random input?

• first study 25%-33%

• second study – utilities 40%– applications 25%

Page 16: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Window NT study

• Similar study for Windows NT, using software/utilities from different vendors (Microsoft, Adobe, Command Software Systems, Qualcomm, Aladdin, Ghostgum, Free Software Foundation, Sun, Netscape, Jasc, Van Dyke, MIT Kerberos Group, Nullsoft, Ipswitch, Metrowerks)

• 23 applications

Page 17: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Results

• How many utilities/applications crashed or hung on random input?

Page 18: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Results

• How many utilities/applications crashed or hung on random input?

• 21% of applications crashed (valid keyboard/mouse events)

• Additional 24% hung (valid keyboard/mouse events)

• 100% crashed or hung with random input streams

Page 19: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Blackbox vs. whitebox testing

• Blackbox testing considers only input-output behavior of a component.

• Blackbox testing tests functionality.• Whitebox testing considers the internal

structure of components.• Whitebox testing tests “structural and

dynamic aspects of the component”[Bruegge & Dutoit, “Object-Oriented Software

Engieering”, p. 448]

Page 20: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Equivalence testing

• Blackbox technique

• Partition inputs into equivalence classes.

• A representative from each equivalence class is used for each test.

• Assumption: all members of equivalence class will behave the same.

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 453]

Page 21: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Example• Calendar• Month equivalence classes

– months with 30 days– months with 31 days– months with 28 or 29 days– Invalid inputs: month<1, month>12

• Year equivalence classes– non-leap years– leap years– Invalid inputs: year < 0

• If method depends on both month and year, we have 2*3=6 equivalence classes.

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Page 22: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Example• Calendar• Month equivalence classes

– months with 30 days– months with 31 days– months with 28 or 29 days– Invalid inputs: month<1, month>12

• Year equivalence classes– non-leap years– leap years– Invalid inputs: year < 0

• If method depends on both month and year, we have 2*3=6 equivalence classes (My question: why not 4*3=12?)

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Page 23: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Boundary testing

• Test boundaries of equivalence classes• In example, February yields many boundary

cases. Leap years are years divisible by 4, unless divisible by 100, unless divisible by 400.– leap years divisible by 400 (e.g. Feb 2, 2000)– non-leap years divisible by 100 (e.g. Feb 2, 1900)

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 454]

Page 24: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Path testing• Whitebox testing• Ensure that all paths through the code are exercised• Start with a flow graph (a graph whose nodes are

executable blocks of code and whose edges represent flow of control)

• Test cases must cover each edge in graph• Minimum number of test cases required is the number of

independent paths in graph, which is # edges - # nodes + 2

• Can work off of activity diagram

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 456]

Page 25: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

State-based testing

• Whitebox testing

• Ensures that state transitions of system work as expected

• Works off of statechart diagram

[Bruegge & Dutoit, “Object-Oriented Software Engieering”, p. 459]

Page 26: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Unit Testing Example

In this exercise we will develop a simple class (RationalNumber) in a TDD fashion.

Page 27: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Properties

• A rational number is of the form n/d, where n is an integer, and d is an integer except zero.

• Rational numbers can be added, multiplied, tested for equality.

• Each rational number has an additive inverse (its negative) and a multiplicative inverse (its reciprocal).

Page 28: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Rational class

• Will provide public constructor:public Rational(int numerator, int denominator);

• Will provide public methods:public Rational add(Rational r);

public Rational negative();

public Rational multiply(Rational r);

public Rational reciprocal();

public boolean equals(Object obj);

Page 29: Testing Team exercise Have each team member contribute answers: –Do you test your code? If no, why not? If yes: When? How? How often? –What is your team’s

Exercise structure

• Break into 6 groups

• Each group identifies tests to perform on their operation (construction, addition, negative, multiplication, reciprocal, equals)

• I will write the tests, then we will write the code