testing best practices
Post on 09-May-2015
865 Views
Preview:
DESCRIPTION
TRANSCRIPT
Testing Best Practices
Kevin WhinneryPlatform EvangelistAppcelerator Inc.@kevinwhinnery
kwhinnery@appcelerator.com
• The Challenge of Testing Rich Client Applications
• Levels of Testing• Titanium Testing Tools• The Future of Testing
Titanium Apps• Q&A
Testing Best Practices
The Challenge of Rich Client Testing
A Few Challenges:
• Relevant tests?• Tooling?• Automation?• Production
environment?• Human
interaction?
Rich Client Testing Challenges
Testing rich client applications
remains a challenge today due to
the visual and human aspects of
client applications.
• Model tier testing is highly doable, and test friendly
• UI testing is hard to do in a meaningful way:• Does code work in a long-
running scenario?• If I programmatically build a
UI structure, how close is that to the real UI structure at runtime?
• I can test object properties, but reliably ensuring visual integrity is hard.
How relevant are our tests?
• In JS, client side test tools are somewhat limited in functionality and scope
• What is available is more geared toward simple applications• …and typically tied to the
browser, in the case of JavaScript
What tools do we use?
• Scripting user interaction is difficult, especially in native mobile apps
• Requires multi-platform environment
• Requires some sort of connection to a running simulator or device, which must also be scripted
How do we automate tests?
• A simulator only goes so far• Need to test on a variety of
devices• Mobile is a hostile and volatile
environment:• Data connectivity?• Network speeds?• Software/hardware
combinations (iOS has this pain now too)
• Background processes?• Geographic locations?
How close are we to prod conditions?
• Does our app “look right” with different data sets?
• Are our buttons responsive enough, or do loading screens take too long?
• Are there operations which are unintuitive, or error prone?
• Are the properties of the UI we are changing and animating actually reflected in the UI how we expect?
How is our “look and feel”?
These problems can all be overcome, but testing can become a classic struggle between…
Time
Returns (Quality)
And Resources
Consider:
• How long will I support this app?
• How useful is the test automation I am adding – how true to life are my test results?
• Automation versus human testing
Diminishing Returns
Test automation can become
extremely expensive. Frameworks
need to be maintained, infrastructure
needs to be created, and time needs
to be spent writing, scripting, or
recording tests.
Make sure the investment is
justified.
Levels of Testing
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Depending on the scope, shelf life,
and team size for every application,
only certain levels of testing may be
needed.
You probably already know these by
some name, but let’s establish a
common vocabulary.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Ad hoc testing by a single tester or
group of testers may be sufficient for
small applications with a short shelf
life.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Programmatic unit testing can
provide a baseline comfort level with
code in a project, giving the
developer knowledge that public
APIs have not regressed and are
working as expected.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Continuous integration can make
use of an existing suite of automated
unit tests, and ensure that all test
suites are passing as new code is
committed to the project.
Continuous integration becomes
especially useful as a team
increases in size and activity.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Automated functional testing tests
actual application interactions, and
inspects the state of an application
at known points.
This type of testing is sometimes
referred to as “record and play”
testing.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Levels of Testing
Crowdsourcing deploys an
application to real humans around
the world, who can test applications
on different data networks, devices,
and locations to provide real-world
feedback on application functionality.
Can also be employed to provide
feedback on look and feel and other
human elements.
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Where We Are Today
Appcelerator (and our community)
have developed test automation
tools to provide a baseline level of
comfort with application functionality.
The testability of your application
can be greatly enhanced by
employing modular design.
Demo: Testable Titanium apps with behave.js
Types of Testing:
• Ad hoc• Automated Unit
Testing• Continuous
Integration• Automated
Functional Testing• Crowdsourcing
Where We’ll Be Tomorrow
Appcelerator, the community, and
our commercial partners are
collaborating on testing solutions
across the board.
Visit with SOASTA and uTest in the
hallways!
• Test automation is a great tool for ensuring quality
• For rich client apps, useful tests have always been a challenge
• Choose the level of testing appropriate for the project
• Join us in reducing barriers to testing and building awesome tools
To Sum Up
Downloads/Resources:
• http://bit.ly/• codestrong_src• ti_test_slides
To Sum Up
top related