Slide 1

Improving Software Testing by Observing Process -Ossi Taipale -Kari Smolander Lappeenranta University of Technology, Finland Presented by Albert Saryan and Karo Mazidzhyan Slide 2 Breakdown Introduction Related Research Research Process Analysis Results Process Improvement Propositions Conclusion Slide 3 Introduction The objective of this study was to understand how software testing is conducted by observation. From observations propose improvement to the testing process. Improvements by reducing development and testing costs, and improving quality. Slide 4 Software Costs and Quality Software Engineering strives to reduce development costs and improving quality. Software Process Improvements (SPI) are the means to reaching these goals. Commitment to SPIs by all from all organizational levels is key to success Quality can be tested into products or developed and built into products Slide 5 Software Costs and Quality Cont. External events such as deadlines affect software quality. The cost of software testing is high, therefore SPIs are necessary to reduce cost. Slide 6 Related Research Involvement of testing during development occurs when testers develop test for developers to analyze The complexity of testing increases as a function of the complexity of the systems under testing. Testing strategy defines the contents of testing. Slide 7 Related Research Cont. Communication and interaction between development and testing processes requires cooperation and coordination. The use of software components are increasing rapidly Design outsourcing and distributed development increase the use of components. Cost of Quality is Free, but being late with products may be more costly than fixing faults. Slide 8 Research Process This study consisted of Organizational Units (OU) which develop and test technical software for automation or telecommunication in Finland. Initial Sample included 26 OUs, from which 5 were used as case studies. Cases were chosen to show polar types Slide 9 Research Process Cont. Data for the research was collected by a series interviews. Each interview had a different theme in mind and possibly a different interviewee in mind. The interviews took place during five rounds, based on the theme. Slide 10 Research Process Cont. Interview Rounds 1. Development and Testing Managers were asked to define their testing process. 2. Managers of Testing were asked to define their testing process in depth. 3. Testers were interviewed. 4. Systems Analysts interviewed. Slide 11 Research Process Cont. Case Breakdowns Slide 12 Research Process Cont. Data Analysis Information gathered from these interviews were then categorized. Categories were then analyzed to see how they were connected. The categories were then used to identify factors which affected testing. Slide 13 Analysis Results Description of Cases: Case A - Developed and tested Manufacturing Execution Systems : Turnover 50% product, 50% service Services included systems integration and customization Testing against requirements was a challenge because customers had special in-house requirements standards Developers and testers worked physically close to each other Time allocated to testing was consistent, although over time it has been reduced Use of components low, hinders testing Slide 14 Analysis Results Cont. Case B - Tested in house products and provided testing services for external customers: Turnover 75% service, 25% product Majority of requirements specifications were based on standards. Delays in development allowed for fewer time allocated for testing Communication flexible, developers talked face-to- face with testers Use of components high, testability of components must be considered Slide 15 Analysis Results Cont. Case C Customized Software Development: Turnover 2/3 service, 1/3 product Testers were involved early, involved in development process Testers often had issues due to lack of advisement from developers Delays in development does not often result in reduction of testing time Developers and testers communicate face-to-face Use of components low Testing of software components seen as difficult b/c of different implementation. Slide 16 Analysis Results Cont. Case D - Electronics: Turnover 100% product High product orientation required high quality because recalls are very expensive Avoided testing of unfinished product Testing tasks clear and well documented Testing involved in development late but planning of testing automation provided information on upcoming tests Communication between developers and tester planned, formal and transparent Use of components high Components tested initially by suppliers then again at system testing. Slide 17 Analysis Results Cont. Case E Software Testing Services: Turnover 100% service Working as an external testing organization required the adaptation of the process of the customer Early involvement of testing was necessary for the testing company to increase the testability of the software Sometimes testing was involved late Budgets for testing affected testing time Communication was handle through a contact person, but was active and clear Use of components depends on customer As an external testing organization it was hard to receive information about customers purchased components. Slide 18 Analysis Results Cont. Slide 19 Cause and effect Slide 20 Analysis Result Cont. Cause and Effect Business Orientation Directly associated with use of components and testing schedules Business Model value adding process Purely Service Oriented System integration Customizing Consulting Customers directly affect development and testing process Purely Product Oriented Product Development Marketing Customers do not directly affect development and testing process Slide 21 Analysis Results Cont. Process Improvement Propositions Testing ought to be adjusted to business orientation Product oriented should adopt formal planned testing process Service oriented should adopt a flexible testing process Enhanced Testability of software Consider testability when selecting components Review testing process of suppliers Slide 22 Analysis Results Cont. Effective Communication and interaction between development and testing Early involvement of testing and planning of testing Use of risk based testing Slide 23 Conclusion Proposals by observing best practices using grounded theory Better documentation improved testability of software and components Efficient communication between development and testing improved quality When time is a major issue, risk based testing is the best solution Business orientation affects: Use of components Amount and quality of communication Allocated testing time and the planning of testing Slide 24