04/11/23 1

Contemporary Application Contemporary Application DevelopmentDevelopment

Are Testing Teams and Techniques keeping pace?

Pete CouperAMR & Associatespeter.couper@amr.co.nzOctober 2003

04/11/23 2

Question!Question!

How many of you have been in Testing Teams asked to transition from– Mainframe environment, predominantly

batch/file based interfaces, to– GUI based N-tier solutions with complex realtime

integration across bespoke and shrink wrapped systems including integration to legacy systems running in vendor supplied frameworks (eg application server or .NET)?

What process changes, training and tools were supplied to help you with this transition?

04/11/23 3

Session OutlineSession Outline

Contemporary Application Development Defined

Contemporary Application Development – Challenges for Testing

Some Contemporary Testing Techniques

A look at a Contemporary Testing Team

Session Summary

04/11/23 4

Contemporary Application Contemporary Application Development DefinedDevelopment Defined

Dictionary definition of Contemporary– Date: 1631

1 : happening, existing, living, or coming into being during the same period of time2 a : SIMULTANEOUS b : marked by characteristics of the present period : MODERN, CURRENT

Often Contemporary Application Development refers to :-– Methods, or– Tools

04/11/23 5

Contemporary ApplicationContemporary Application Development DefinedDevelopment Defined (cont.)(cont.)

What do I mean by Contemporary Application Development?– Method used & Structure of Teams– Tools – both Development and Runtime– n-Tier, distributed– Mix of bespoke, shrink wrapped, legacy– Complex integration – sync and async– Web Services – internal and/or external– Cross organisation business process

integration– Etc.

04/11/23 6

Contemporary ApplicationContemporary Application Development DefinedDevelopment Defined (cont.)(cont.)

Emphasis on “Speed to Market” & ROI

Less Large Projects

Increase in Buy/Build

Agile Software Development Methods (XP, SCRUM, FDD, DSDM, ASD etc.)

04/11/23 7

Contemporary ApplicationContemporary Application Development DefinedDevelopment Defined (cont.)(cont.)

Speed of Technology Change

Mix of Technology

Service Oriented Architecture (SOA)

Enterprise Application Integration (EAI)

04/11/23 8

Session OutlineSession Outline

Contemporary Application Development Defined

Contemporary Application Development – Challenges for Testing

Some Contemporary Testing Techniques

A look at a Contemporary Testing Team

Session Summary

04/11/23 9

Contemporary Application DevContemporary Application Dev.. – – Challenges for TestingChallenges for Testing

Need fully integrated test environment

Ability to test multiple delivery channels

Changing requirements

Ever Increasing Time Pressure

04/11/23 10

Contemporary Application DevContemporary Application Dev.. – – Challenges for TestingChallenges for Testing (cont.)(cont.)

Complex test planning

Lack of support for new packages

Faults hard to identify/define

Hard to match Prod. Environment

04/11/23 11

Contemporary Application DevContemporary Application Dev.. – – Challenges for TestingChallenges for Testing (cont.)(cont.)

Non-functional testing of solution

Environment stability

Testing on incomplete solutions

Access, availability, data & date sync. issues

04/11/23 12

Session OutlineSession Outline

Contemporary Application Development Defined

Contemporary Application Development – Challenges for Testing

Some Contemporary Testing Techniques

A look at a Contemporary Testing Team

Session Summary

04/11/23 13

Some Contemporary Testing Some Contemporary Testing TechniquesTechniques

Develop a Test Strategy that– Takes an Enterprise Level view of

testing– Covers all existing Systems– Defines “How to” for Projects– Includes vendor testing – for both

Business-as-usual and projects– Covers the Testing Teams involvement

throughout the SDLC

04/11/23 14

Some Contemporary Testing Some Contemporary Testing TechniquesTechniques (cont.)(cont.)

Test Planning is more complex– Multi system, fully integrated, data driven,

external interfaces, delivery channels, new packages, mix of technology, multiple vendors, incomplete solution, non-functional testing, etc.

Plan to Test early– Testers review functional and non-functional

requirements specifications prior to sign-off– Testers review Use Cases and define alternate

flows in Use Cases, again, prior to sign-off– Test iterations as they become available

04/11/23 15

Some Contemporary Testing Some Contemporary Testing TechniquesTechniques (cont.)(cont.)

Plan to allow Testers flexibility– Testers should be encouraged to follow

“hunches”, not simply stick to a rigid script Plan to include Non-functional testing

– Test the System “-abilities”(Reliability, Scalability, Recoverability, Availability,

Usability, Manageability, Compatibility, even Securability etc.)

Include a mix of testing techniques– Automated– Manual– Exploratory

04/11/23 16

Some Contemporary Testing Some Contemporary Testing TechniquesTechniques (cont.)(cont.)

Consider a Risk Based approach to testing– Define testing mission– Risk Assessment

Outside-In – “-abilities” Inside-Out

– Risk Prioritisation– Develop Risk Task List

Aim – “Find the fundamental faults fast”

04/11/23 17

Session OutlineSession Outline

Contemporary Application Development Defined

Contemporary Application Development – Challenges for Testing

Some Contemporary Testing Techniques

A look at a Contemporary Testing Team

Session Summary

04/11/23 18

Glenford Myers QuoteGlenford Myers Quote

The design of “good” system test cases requires more

creativity, intelligence and experience than required to

design the system.

04/11/23 19

Glenford Myers Quote - ModifiedGlenford Myers Quote - Modified

The design of “good” system test cases requires more

creativity, intelligence, technical appreciation and experience than required to design the

system.

04/11/23 20

A A Contemporary Testing TeamContemporary Testing Team (cont.)(cont.)

Test Team is expected to develop a comprehensive Test Plan

Faults need to be identified and reported to appropriate vendor

Ability to identify and prioritise risksNon-functional testing scripts need

to be developed and executed

04/11/23 21

A A Contemporary Testing TeamContemporary Testing Team (cont.)(cont.)

Test Teams need Testing ProfessionalsTesting Professionals need to

understand technologyBusiness resources added to the Test

Team need to be trained and mentoredConsider outsourcing technical testing

tasks e.g. Test Planning, Performance Testing, Reliability Testing

04/11/23 22

Session OutlineSession Outline

Contemporary Application Development Defined

Contemporary Application Development – Challenges for Testing

Some Contemporary Testing Techniques

A look at a Contemporary Testing Team

Session Summary

04/11/23 23

Session SummarySession Summary

Contemporary Application Development has increased the complexity of software solutions

This complexity directly impacts on testing

The effectiveness of testers will reduce if they do not change how they approach testing

04/11/23 24

Session Summary Session Summary (cont.)(cont.)

Comprehensive Test planning is critical when testing a complex software solution

It is vital the Testing Team contains the experience and technical skills required

Test Professionals are essential to ensure a quality outcome

Testing is a profession in its own right, it needs to be recognised as such

04/11/23 25

Session Summary Session Summary (cont.)(cont.)

We have Contemporary Application Development

Is it time that it is complemented by

Contemporary Application Testing?

Well I think it is!

04/11/23 26

THE ENDTHE END

Questions?

04/11/23 27

ReferencesReferences

James Bach – Satisfice Inc. – www.satisfice.com www.Stickyminds.com “Project Survival Guide” & “Rapid Development” - Steve

McConnell Dr. Magdy Hanna – International Institute for Software

Testing – www.softdim.com/iist Glenford Myers – “The Art of Software Testing” 1979

To get a soft copy of this presentation go to:-www.amr.co.nz