learn key insights from the state of web application testing research report
TRANSCRIPT
![Page 1: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/1.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Learn Key Insights from The State
of Web Application Testing
Research ReportDiane Hagglund, Senior Research Analyst at Dimensional Research
Kristin Brennan, Vice President of Marketing at Sencha
Jon Jarboe, Technical Marketing Manager at Sencha
![Page 2: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/2.jpg)
The State of Web Application TestingA Survey of QA and Development Professionals
![Page 3: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/3.jpg)
Research Goal The goal of this survey was to capture data on current
trends in web application quality.
Goals and Methodology
Methodology An online survey was sent to global databases of web
technology professionals. Questions were asked on a
wide range of subjects including current quality, impact of
quality issues, quality processes, future of quality
initiatives, and more. The incentive for participant was a
copy of the final report.
Participants 1,011 individuals with responsibility for QA or
development participated in the survey, representing a
wide range of roles, geographies, and company size.
![Page 4: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/4.jpg)
Participant Demographics
Less than 10 employees
23%
10 - 100 employees
30%
100 - 1,000 employees
26%
More than 1,000 employees
21%
Company Size
North or South America
41%
Europe, Middle East, or Africa
36%
Asia Pacific 23%
Region
46%
47%
68%
0% 10% 20% 30% 40% 50% 60% 70% 80%
B2C (Consumer)
Internal (employee or partner use)
B2B (Business)
Type of Application End User
14%
34%
36%
40%
85%
0% 20% 40% 60% 80% 100%
Business stakeholder
Support
Vision and strategy
Quality assurance
Development
Responsibilities
![Page 5: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/5.jpg)
Web application quality is critical
How important is the
quality of the apps you
are responsible for?
Critically important 34%
Very important 50%
Important 15%
Not important 1%
![Page 6: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/6.jpg)
Quality problems can have dire consequences
What are the potential
business impacts of quality
problems with your apps?
1%
3%
30%
43%
51%
56%
59%
79%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Quality problems do not have a business impact
Other
Miss time to market advantage
Lost revenue
Impact employee efficiency
Incur costs to fix problems
Damage reputation
Impact customer satisfaction
![Page 7: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/7.jpg)
Quality issues are the norm – not an exception
In general, how frequently do
business-impacting quality
problems occur with the apps
you have responsibility for?
13% 15% 25% 34% 4% 7%2%
0% 20% 40% 60% 80% 100%
It is an ongoing problem
Weekly
Monthly
A few times a year
Yearly
Less frequently than yearly
Never
![Page 8: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/8.jpg)
Good QA is hard
What challenges does your
team face in doing adequate
quality assurance to prevent
critical issues?
6%
7%
21%
22%
22%
40%
45%
55%
0% 10% 20% 30% 40% 50% 60%
We don’t face any QA challenges
Other
Too easy to ignore defects found in testing
Development and QA teams are siloed and don’t communicate well
Quality is not taken seriously
Testing is not integrated into the workflow
Aren’t able to sufficiently test across all browsers and devices needed
Schedule pressures don’t permit adequate testing
![Page 9: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/9.jpg)
When development schedules slip, testing suffers
Are planned testing
efforts ever compressed
because development
schedules slip?
Yes, always 30%
Sometimes, but not always
56%
No, never 14%
![Page 10: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/10.jpg)
Test automation practices lack maturity
How would you
characterize your team’s
current test automation
capabilities?
8% 17% 30% 25% 21%
0% 20% 40% 60% 80% 100%
We have a fully mature testautomation practice that hasautomated everything possible
We do a great job with testautomation, but we could do abit more
We have automated some ofthe most important tests buthave significant room to domore automationWe do minimal automation
We do not do any testautomation
![Page 11: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/11.jpg)
Hiring good test automation engineers is difficult
In your experience, which
type of professional is
harder to hire? Test automation engineer
61%
Software developer
39%
![Page 12: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/12.jpg)
Cultural issues may impact quality
If a frontline development
or QA professional
discovered a potentially
business-impacting issue
the day before a major
release, is there a process
to flag an urgent issue
and stop the launch?
Yes, we have a clear process
52%In theory we
have a process, but it’s not clear how it
actually works 30%
No, there is no process
18%
61%56%
48%
22%
30%34%
0%
10%
20%
30%
40%
50%
60%
70%
Executive Manager Individualcontributor
By Job Level
Yes, we have a clearprocess
In theory we have a process, but it’s not clear how it actually works
![Page 13: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/13.jpg)
Employees are rarely held responsible for issues
In the past two years,
what has been the result
of business-impacting
quality issues on the
careers of team members?
44%
6%
15%
15%
18%
32%
0% 10% 20% 30% 40% 50%
No team members have had their careersimpacted because of quality issues
Other
Bonuses were not paid
Individuals responsible lost their jobs
Individuals responsible were demoted or notadvanced
Negative performance reviews were given
![Page 14: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/14.jpg)
Compensation rarely tied to quality
Is your compensation
impacted based on the
quality of the applications
your team delivers (i.e.
bonus is paid or held
back)?
Yes 34%
No 66%
46%
33%
31%
54%
67%
69%
0% 20% 40% 60% 80% 100%
Executive
Manager
Individual contributor
By Job Level
Yes
No
![Page 15: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/15.jpg)
Organizations are investing in web app quality
How do you expect your
team’s testing investment
will change in the coming
year?
29% 44% 23% 4%
0% 20% 40% 60% 80% 100%
Increase significantly
Increase somewhat
No change
Decrease
![Page 16: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/16.jpg)
Many factors drive testing investment
Why will your team
increase investment in
testing in the coming
year?
n = will increase testing investment
29%
30%
35%
36%
38%
40%
41%
42%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Create competitive differentiation
Business stakeholders are emphasizing thevalue of higher quality
Reduce overall costs
Response to a business-impacting quality issue
Need to address customer satisfaction issues
Just part of our overall development investmentincrease
Need to improve time-to-market and bettermeet release schedules
End users are demanding higher quality
![Page 17: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/17.jpg)
Addressing Today’s Web Application Quality
Challenges
![Page 18: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/18.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Where Are You on the Test Automation Maturity Model?
![Page 19: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/19.jpg)
CONFIDENTIAL • Sencha Inc. ©2017
Test Automation Benefits
19
![Page 20: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/20.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Automated Testing Savings
![Page 21: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/21.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Automated Regression Testing Savings
![Page 22: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/22.jpg)
Invest in Tools to Help Deliver High-Quality Web Apps
Sencha Test: Unit and Functional Testing Ext JS Apps
• Complete, cohesive, integrated testing
platform for Ext JS apps – incorporates
years of experience
• Deepest integration into Ext JS for
faster test creation & highest accuracy
• Execute tests on multiple browsers
simultaneously
• Works seamlessly with existing tests
• WebDriver integration enables faster
testing of multi-page apps – mimics
user behavior
22
![Page 23: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/23.jpg)
Sencha Test Components
23
Test Studio
Test Authoring
Test Runner
Test Execution
Test CLI
Test Automation
Test Archiver
Results Reporting
And Archiving
![Page 24: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/24.jpg)
Test Authoring: Sencha Test Studio
• Write unit and end-to-end functional tests in
JavaScript (Jasmine)
• Create tests with built-in test editor or in a
separate IDE – store them in SCM
• Quickly write & execute tests while coding
• Leverage pre-authored tests for faster
results
• Advanced APIs improve testing efficiency
by minimizing the lines of code in a test
24
![Page 25: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/25.jpg)
Test Authoring: Targeting Elements
• Any testing tool must allow testers to select elements upon which to act
• Sencha Test helps you avoid brittle selectors and dynamic IDs
• These ”locator strategies” improve effectiveness and reduce maintenance
• Consistently target the right element through:
• Ext JS Component query
• DOM query (XPATH or element ID)
• A combination of the above
• Create powerful page objects that maintain locators in a central location for
consistency and easier maintenance
25
![Page 26: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/26.jpg)
Test Authoring: Event Recorder and Inspect
• Automatically records events while
interacting with the system under
test
• Even as you navigate multi-page
apps
• Generate tests in Chrome and
Microsoft Edge, execute on any
browser
Event Recorder
26
• Helps you easily locate components
on the app under test
• Choose the best locator from a list
of all possible options
• Automatically insert API code into
your test or page object, using that
locator
Inspect
![Page 27: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/27.jpg)
Test Authoring: External Libraries
• Leverage external libraries in test code
• Connect to databases
• Communicate with web services
• Consolidate common code
• Use like you would in any other JavaScript
code (e.g. require)
• Works for local and remote (WebDriver)
scenarios
27
NoSQL
Web
Services
![Page 28: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/28.jpg)
Test Execution: Test Runner
28
• Run selected tests on any or all browsers,
local machine or browser farm
• Dramatically reduce testing time by
executing tests simultaneously on
multiple browsers through integration with
leading browser farms such as Sauce
Labs and BrowserStack
![Page 29: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/29.jpg)
Application Lifecycle Management: JIRA
• Reduce the effort of tracking test failures
and defects in JIRA
• Manage tests, results and defects in Sencha
Studio
• Failed tests may be marked as defects,
easily exported to specified JIRA project
• Sencha Studio tracks status of bugs to
minimize duplicated reports
29
Sencha Test
![Page 30: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/30.jpg)
Test Automation: Sencha Test CLI (Command Line Interface)
• Maximize the efficiency of your testing
through automated test runs
• Launch tests within the CI system as soon as
application changes and tests are checked
into the source control repository
• Out of the box integration with TeamCity and
Jenkins
30
code
testreview
![Page 31: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/31.jpg)
Test Execution: Test Archiver
31
• Review results from automated and
manual test runs
• Summary level results & drill into the
detail of failed tests
• Out of the box integration with Istanbul
enables users to identify and rectify
code coverage gaps
• View % coverage at the Statements,
Branches, Functions and Lines levels
![Page 32: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/32.jpg)
Visual Screen Comparison: Test Archiver
32
• Minimize the time spent
verifying images on hundreds
of screens
• Review images that are
captured from previous test
runs and compare to each
subsequent test run
• Specify screen sizes to
improve accuracy of results
![Page 33: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/33.jpg)
Sencha Test System Overview
33
Sencha Test Studio
Sencha Test Archiver
Sencha Test CLI
Browser Farm
CI Server
Local Workspace
Local Browsers
App Server
PROXY
PROXY
MANAGES
USES
USES
USES
STORES
RETRIEVES
![Page 34: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/34.jpg)
Demo
![Page 35: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/35.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Long-term Sustainment
• 1+ Months
Test Development
• 1-6 Months
Test Infrastructure
Setup
• 1-3 Weeks
Initial POC
• 1-3 Weeks
Assessment
• 1-2 Weeks
35
Sencha Test Automation
![Page 36: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/36.jpg)
© 2017 Sencha Inc. • CONFIDENTIAL •
Benefits of Sencha Test Services
• Reduction in cycle time (dev or maintenance) for release to production
• Increase in code quality that can correlate to increased customer satisfaction,
competitive differentiation, reduction in support costs
• Implementing a Test Strategy that supports the business goals that aids in the
management team running the business. Supporting their business metrics
• Increased use of automated testing for reducing cost and cycle time
• Ability to support the range of device and OS combinations necessary in today’s
world
• Extend the useful life of the application – protect the large investment in developing
the app in the first place
36
![Page 37: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/37.jpg)
Final Recommendation: Drive Accountability
• What gets measured/rewarded gets done
• Ensure entire team is held accountable for app quality
• Quality is not just the job of the QA team
37
![Page 38: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/38.jpg)
Q&A
• Try Sencha Test for Free: https://www.sencha.com/products/test/evaluate/
• Contact us for help with your test strategy and automation needs:
https://www.sencha.com/company/contact/
38
![Page 39: Learn Key Insights from The State of Web Application Testing Research Report](https://reader035.vdocuments.site/reader035/viewer/2022062905/5a66ed307f8b9a91298b4ae9/html5/thumbnails/39.jpg)
Thank You!