the challenges of quality assurance for a constantly changing product - erik hittenhausen, testronic
DESCRIPTION
Slides from Erik Hittenhausen's talk at November 2013's London Games Conference. Today, many of the most popular global franchises continually evolve rather than relying on a day one ship. The advance of digital games and DLC has meant that it is now more important than ever that quality of a product and subsequent updates remain consistentTRANSCRIPT
TestronicYour Partner in Quality
The Challenges of Quality Assurance for an Ever-Changing Product
Erik HittenhausenHead of Functionality & Compliance QA – Games (Testronic)
TitleDate Slide no.
• Traditional Retail
ONE chance to get it right
• New Business Models
Multiple opportunities to screw up
The Challenges
TitleDate Slide no.
Traditional Retail
A project with a defined scope, start and end date.
Updates and patches post-release do not have to be tied to strict deadlines
Quality upon release main benchmark
TitleDate Slide no.
New Business ModelsMobile and Free To Play
• An on-going project, with ever increasing scope
• To stay successful and competitive the product has to be updated at regular intervals
• Consistency of quality main benchmark
TitleDate Slide no.
Multiple releases means multiple risks
Risks involved in New Business Models
• Breaking live environment
• Bad patch leads to negative social media and press coverage
• Compliance: changing requirements of platform holders
• Compatibility: keeping up with the latest hardware
TitleDate Slide no.
How to deal with the challenges
• Build and retain a core QA team
• Establish and define a synchronous Release and QA cycle
• Develop and maintain an evolving test plan
• Break down the product
TitleDate Slide no.
Build and retain a core QA TeamMore stable resourcing of QA teams is key
• Retain your core team
• An experienced dedicated team helps to retain a working knowledge of the product allowing efficiency gains to keep up with the growing scope/content of the product
• Specialising people and embedding expertise (exploratory testers, playthrough wizards, compatibility, compliance)
• Capatilise on experience and efficiency by effective test planning and checklist creation
TitleDate Slide no.
Release and QA cycle
Establish and define a synchronous Release and QA cycle
• Example 2 week cycle: compliance passes
compatibility passes
one exhaustive pass of all features and content
continuous scripted and exploratory testing of features and content under going development
TitleDate Slide no.
Break down the test focus and optimise the QA cycle by identifying which content is affected by updates:
Planning / Legacy Content vs. New Content
• Stable Content Content already tested and unaffected by ongoing
development can be re-tested once throughout the QA cycle
• Changing Content Existing content which is being changed, updated or
expanded, understand what is affected
• New Content Content that is being added and may interact with stable
and/or changing content and must be tested continuously