20121213 qa introduction smileryang
TRANSCRIPT
Smiler Yang
2012/12/13
Quality Assurance in Software Industry
Outline
• About me
• Roles in a project
• Why a software needs QA
• QA tasks
• Testing Methodology
• Examples
• Some stories in real world
• Happy and Unhappy
• Q & A
Roles in a project
• PM (Product Manager)
• JM (Project Manager)
• DM (Developer Manager)
• QM (QA Manager)
• Architecture
• RD
• QA
• UI Designer
• Technical Writer
Why a software needs QA?
• Did you test your program?
– I think it’s “Yes”
– But you might not consider everything…
So, we need---
• Different people
• Different knowledge
• Different environment
QA tasks
• Requirement Clarification
• Test Plan
• Test Spec
• Test Case
• Environment Preparation
• Functional Testing
• System Testing
• Exploratory Testing
• Bug Reporting
• Bug Verification
• Document Testing
• Test Automation
• Beta training
• Customer on-site
Testing Methodology
Testing Methodology (1/5)
• Black Box Testing – Perform the tests on the basis of functional properties and
requirements of a software
– Focus on the selected test data and the interpretation of test results
– Needs experience…
– Includes • Functional Test
• Error-handling Test
• Boundary Test
• Performance Test
• Stress Test
• Usability Test
• I18N Test (internationalization test)
• …
Testing Methodology (2/5)
• White Box Testing – Need to understand overall system architecture
– Inspect code of function implementation
– Focus on testing the paths of control flow and data flow • Go through all independent paths
• Exercise logical decisions
• Executed all loops with boundary condition checked
Testing Methodology (3/5)
• Test Plan
– Purpose of Test Plan
• Sync Testing Direction with JM/DM/RD
• To Give all testing directions/information to QA Engineer
– Information required in a test plan
• MRD/PRD/Functional Spec/High Level Design
– Schedule of each development process, Feature List, System Requirement, Product Team Information
• Configuration Compatibility Testing
– hardware compatibility / software (OS/Application) compatibility
• What is the user expecting to the product?
– How does Customer use the product?
– on what kind of the environment?
Testing Methodology (4/5)
• Define the Scope of testing of Features and Environments
– Doing and not doing
– Decide the priority of Features and Environment
• Define of Bug Severity
– Severity: Critical/Serious/Non-Critical/Enhancement/Suggestion
• Define Testing methods
• Resource Planning
– People
– Environment
• Preparation of the testing
– Knowledge: QA knowledge, product knowledge, …
– Environment: Software, Hardware,…
• Process for Testing/Management
– Bug Tracking System
– Checking method of the progress
– Bug Review Meeting
– …
Testing Methodology (5/5)
• Test Case
– What is Test Case?
• The set of input data, operation steps and expected result
• Operation steps includes conditions, configurations, environment information, …
– What is good Test Case?
• The purpose of the testing is to improve the quality by finding the errors.
• Test case should cover all requirements.
• Test case should take the wrong input/output and wrong operation into consideration.
• Test case should have clear description.
Examples
Example of Test Case Creation
• Use Case:
– Account Login in a web portal
• Test cases??
– Hint: • Functional Test
• Error-handling Test
• Boundary Test
• Performance Test
• Stress Test
• Usability Test
• I18N Test
Example of Bug Reporting
• Issue: User failed to login the web portal
• How to describe a problem?
• The information of a bug should include:
– Title
– Environment
– Version/build number
– OS/Platform
– Priority
– Severity
– Defect Type
– Frequency
– Description
– Steps
– Expected results
– …
Some stories in real world
QA and JM
<Before the milestone day>
• JM: Any issues of the candidate build?
• QA: So far we found #358 is the problem for…, and #359 is the issue when user…., and…..
• JM: Have RD found the root cause for these issues?
• ….
• …
• Then “3M” may have a meeting to review the issues.
QA and RD
< After QA submit a bug to RD>
• RD: This problem didn’t appear in my machine.
• QA: It did happen in my testing environment!
• …
• ..
• .
QA and Customer
<One day…>
• Customer: The service is not available after I used the system for only 2 days.
• QA: What did you do on the machine today? Do you think it’s normal yesterday?
• Customer: I don’t know. I just moved the machine from my office to server room.
• QA: Why did you move the machine?
• …
• ..
A day of a QA
• Read emails
• Clean environment and install new build.
• Testing, bug verification, …
• Find new bug and submit it to RD.
• Discussion for the current issues.
• Discussion for the new features.
• Write test plan / test spec / test case
• Clarify, reproduce issues and collect more debug log…
• Review document, write automation, …..
• Reading, study group, …
Happy and Unhappy
• Happy
– Improve the quality of the software
– Provide the solution that customers really need
– Learn new things
– …
• Unhappy
– Too many tasks
– “Why didn’t you find the bug before the release?”
– Schedule VS Quality
– …
Q & A
Thank you~