cs498dmSoftware Testing

Darko Marinov

January 15, 2008

Teaching Staff

• Instructor: Darko Marinov– Email: marinov@cs.uiuc.edu– Office: SC 3116, 217-265-6117– Office hours: after classes or by appointment

• TA: Vilas Shekhar Bangalore Jagannath– Email: vbangal2@uiuc.edu– Office hours: by appointment

Course Overview

• Introduction to software testing– Systematic, organized approaches to testing– Based on models and coverage criteria– Testing is not (only) about finding “bugs”– Improve your testing (and development) skills– Not focused on research (unlike cs598dm)

• Five problem sets and a project– Centered around testing refactoring engines

Refactoring Engines: One Slide

• Next lecture will be on refactoring engines– Guest lecturer: Jeff Overbey

• You’ll learn a lot about refactoring engines in this class

• Refactorings are program transformations that change code but not its behavior

• Refactoring engines are tools that automate refactorings

Administrative Info

• Lectures: TR 3:30pm-4:45pm, 1304 SC

• Credit:– 3 undergraduate hours– 3 or 4 graduate hours (a larger project for 4)

• Prerequisites: recommended software engineering (cs427) and programming languages (cs225, cs421)– Consent of instructor (must if not senior)

Grading

• Points– Project (25%)– Problem sets (5*15%)

• Grades– A*(90%), B*(80%), C*(70%), D*(60%), F(<60%)– For more details, see the syllabus– The instructor may lower the point limits

Project

• Testing a part of refactoring engine– Eclipse– NetBeans

• Deliverables– Proposal (due in three weeks)– Progress report (around midterm)– Final report (by the grade submission deadline)– Bug reports (hopefully you’ll find some bugs)

Collaboration

• You must individually write solutions for the problem sets

• You can collaborate on everything else (unless explicitly stated not to collaborate!)– Discuss problem sets– Do projects in groups, preferably two or three

students

• Testing is a social activity– Communication matters

Course Communication

• Web site http://www.cs.uiuc.edu/class/cs498dm

• Wiki https://agora.cs.uiuc.edu/display/cs498dmsp08/Home

• Mailing list cs498dm@cs.uiuc.edu

• Instructor: Darko– Email: marinov@cs.uiuc.edu– Office: SC 3116, 217-265-6117– Office hours: after classes or by appointment

• TA: Vilas <vbangal2@uiuc.edu>

Signup Sheet

• Name

• Email address (NetID@uiuc.edu)

• Program/Year

• Interests: what would you like to learn about testing?

• Experience: what testing did you do?

• To do: please also sign up on Wiki

Textbook

• “Introduction to Software Testing”by Paul Ammann and Jeff OffuttCambridge University Press, Jan. 2008

• Books should be in the bookstore soon

• We won’t use book before the end of Jan.

• If necessary, you will get PDF files

• Book draft already used in cs{4,5}98dm

This Lecture: Introduction to “Bugs”

• Why look for bugs?

• What are bugs?

• Where they come from?

• How to detect them?

Some Costly “Bugs”

• NASA Mars space missions– Priority inversion (2004)– Different metric systems (1999)

• BMW airbag problems (1999)– Recall of 15,000+ cars

• Ariane 5 crash (1996)– Uncaught exception of numerical overflow– http://www.youtube.com/watch?v=kYUrqdUyEpI

• Your own favorite examples?

Some “Bugging” Bugs

• Two example bugs on my laptop– “Jumping” file after changing properties

• Put a read-only file on the desktop• Change properties: rename and make not read-only

– “Jumping” rectangle after move• Put up a slide with two rectangles• Move one within the textbox

• Your own favorite examples?

Economic Impact

• “The Economic Impact of Inadequate Infrastructure for Software Testing”NIST Report, May 2002

• $59.5B annual cost of inadequate software testing infrastructure

• $22.2B annual potential cost reduction from feasible infrastructure improvements

Estimates

• Extrapolated from two studies (5% of total)– Manufacturing: transportation equipment– Services: financial institutions

• Number of simplifying assumptions

• “…should be considered approximations”

• What is important to you?– Correctness, performance, functionality

Some Motivation for Testers

• An article from SD Times, a magazine for software development managers:“Improving Software Quality”by Lindsey Vereen (page 34/68)

• A slide from Debra Richardson, a professor at UC Irvine:“Analysis and Testing are Creative”(page 26/48)

Terminology

• Anomaly• Bug• Crash• Defect• Error• Failure, fault • G…• …

Dynamic vs. Static

• Incorrect (observed) behavior– Failure, fault

• Incorrect (unobserved) state– Error, latent error

• Incorrect lines of code– Fault, error

“Bugs” in IEEE 610.12-1990

• Fault– Incorrect lines of code

• Error– Faults cause incorrect (unobserved) state

• Failure– Errors cause incorrect (observed) behavior

• Not used consistently in literature!

Next Lecture

• Thursday, January 17, 3:30pm, 1304 SC– Topic: refactorings

• What they are?• Why you need them?• How to perform them?

– Guest lecturer: Jeff Overbey

• Assignments– Sign up on Wiki– Try out Eclipse and/or NetBeans