testing for a great app and web experience | qualitest group

Testing for a Great App and Web Experience

16:00 GMT November 8th 2016

Webinar Presenters:

Aviram ShottenUK BM and Global Head of Partnerships and Alliance

Stephen Ellwood-WadeMobile Product Manager

2

About QualiTest Group

| World’s 2nd Largest Independent Testing Company

| QualiTest was recognized as the only visionary pure play QA and testing Service Company in the “ Magic Quadrant”

4

Pillars of Quality

Predictable Quality & Ux

Func

tion

al

Acce

ssib

ilit

y Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

5


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Cross Platform

Test Automation

Desktops

Phones

TabletsBrowsers

Operating Systems

| One solution – automation and devices management

| X-Platform

| Scalable

| Consider open source vs. ease of use

Cross-Platform Test Automation

6


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Managed Crowd Testing Local and Global Functional Testing Localisation

Testing and Quality Champion owns the delivery of crowd testing and meeting the KPI and SLA

Testing for content, context, geo-restrictions/ local integration, local addresses, language

Dedicated Crowd Test Lead

Exploratory Functional + Test execution hours

Ux/ Cx

Exploratory Test Cycles

Dedicated Test Case Execution Hours

Every testing cycle will be accompanied by a set of standard and specific set of Ux questions about the SUT, will be used for improvement and for benchmarking

Crowd Delivery

Internal Evangelist

7


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Accessibility| Works for people with

Accessibility needs| Deliverables:

| WCAG 2.0 A, AA and AAA Compliant| Automated Tools Testing| Professional manual functional by

dedicated accessibility experts| Utilize testers with genuine accessibility

needs| Focused on both Mobile and Web

Applications

8


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Localisation

| The challenge of making an English app a global app| Localisation process and Development process should work

together closely from early stages:• Localisation Prototyping – readiness for multilingual app• Coverage • Timeline• Key concerns and sector

| The QA of a localised product should include functional testing and specifically linguistic and cosmetic testing.

9


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

App Performance – Complete Vision

• Traditional load generation of Virtual (synthetic) users

• Supports the • Hybrid load approach

• Helps analysing the implications of networks quality on performance

• Injecting low VU load • Stats gathering from servers and google

analytics• Costumed metrics analysis• Immediate notification and scheduled

automated• reports 24/7 Grey-

box Monitoring

Network Quality

Virtualisation

Protocol Level

Performance

Functional Automation/

Crowd

10


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Security Testing| Agile’s quest for Speed drives us toward embedded

security testing – Deliver Secured SW, Fast | Automated (and Manual) security code scan| Automated Pen-Test (automated dynamic analysis) for

web-based applications and Mobile behavior analysis for Apps

| Manual “Ethical Hacking” Pen-Test

11


Func

tion

al

Acce

ssib

ility

Loca

lisat

ion

Perf

orm

ance

Secu

rity

Usa

bilit

y an

d U

x

Ux/Cx vs. Usability

12

User Focused Testing

Deployment Go live

Continuous Performance Monitoring

Pre-Test Activities

Create Baseline Expectations

End User ExperienceEarly Test and Report

Accessibility

13

Hybrid Load /Hybrid Performance

Ux Focus & Device Fragmentation

Continual improvement

Continual feedback

Section 508 and WCAG 2.0 A, AA and AAA focus

Efficient Baseline

Functional

Security

Maintenance Ux Surveys

Surveys &Trend monitoring

Feature Ux ValidationSuggestions for Improvement

About Racing Post“Racing Post aims to be most valued brand for regular horse and dog racing bettors in the UK and Ireland and their preferred partner for form study (online and offline) and preferred digital betting partner. ”

We have spent the last few years, and in particular the last six months, getting to know our customers better.

It has become clear that we cannot be all things to all people and in order to grow, we need to focus on those areas which will drive growth.

14

Racing Post and QualiTest

15

To achieve the goals set in our vision Racing Post make use of two flexible services that QualiTest provide:

| Crowd testing| Special test case hours

These can be applied in a number of ways to support our product development process:

| Hypothesis testing| New feature testing| Regression and Acceptance testing

General Crowd Testing

16

Racing Post leverages QualiTest as part of regression testing and as a fail safe mechanism. As part of each round of testing we can define the following:

| Scope of testing| Known bugs - out of scope| Raised bug detail requirementsGeneralSpecific (crashes for example)| Devices and OSs

Following a release we often schedule another round to cover smoke testing. Although this is done internally this can help especially if bugs have been raised through other channels.

Product Feasibility TestingIn order to deliver value, Racing Post have to ensure products that are delivered are of use to our customers. Using the dedicated test case hours, we use for feasibility testing to allow for Quantitative testing.

Example Hypothesis: If we apply a Nav-Bar instead of a drop down nav on our Android app engagement and conversion will increase in app.

17

Product Feasibility TestingA number of questions were posed to get a well rounded response around whether we should try changing our navigation. The response was excellent and we now have the piece of work lined up as a future A/B test.

18

UAT and the CrowdQualiTest and the Racing Post are specifically recruiting bettors in the UK & Ireland to be part of our test groups meaning Racing Post can involve the service in the UAT process. Specialised scripts can be created in order to target any new functionality for an upcoming release.

Below is a snippet of a test script covering the specifics of the Tricast/Forecast betting integration.

19

In short

20

QualiTest help link up our product development roadmap in a fewhelpful ways in order for Racing Post to be able to deliver qualityproducts.

| Product feasibility testing (quantitative)| UAT| Regression testing| Smoke testing

Trending| Feature Trending

| Individual tracking of features and functionality on demand

| Results broken down by feature/question

| Trend-line reporting

21

Realistic Usable Recommendations

| Suggested outputs may include:| Functional changes, layout changes| Device Fragmentation Cycles| Additional or updated internal / external test case scripts| Specific cycles focusing on Ux Wireframes | Non-traditional output may include: Additional focus required on Localisation/Accessibility

22

Benefits – Client Feedback

It helps you react to changes, quickly – player feedback, new devices etc.

We can test and learn before we go live

It helps build confidence in the products and systems we build for your consumers

It gives you the device and OS coverage you need during your testing

It saves you money as you don’t have to buy all devicesIt helps give real user

feedback during your delivery cycles

It allows for you to easily test out of hours and on weekends

It helps shorten and reduce the cost of your test cycles

23

www.QualiTestGroup.com

Thank YouEmail:

[email protected]

testing for a great app and web experience | qualitest group

Technology