automated testing: saving time and money
TRANSCRIPT
Automated TestingSaving Time & MoneyBlayn ParkinsonE-Learning & training Support AssistantUniversity of York (UK)
ABOUT ME
Blayn ParkinsonE-Learning & training Support AssistantUniversity of York (UK)[email protected]
I have used Blackboard for 3 years.
ABOUT MY INSTITUTION
University of York
15,330 Students
Using Blackboard since 2005.
Central e-Learning team 6 members (some departments have their own provision)
WHAT WE ARE GOING TO LEARN TODAY
• The benefits of using automated testing strategies.
• Limits to automated testing
• Suggestions on future collaboration in this area
OUR CHALLENGE
Thorough testing cycle each (software not deployed without testing).3 months: April – June (One window a year to achieve this).
• Lack of agility in responding to new releases, due to the testing time that we need to devote to a release.
• Needed to lock in to a release at a particular time.• Spread the load across the team in terms of manual testing.
OUR CHALLENGE
Functional testing • is necessary• is time consuming• can be ambiguous
Thorough testing cycle each (software not deployed without testing).3 months: April – June (One window a year to achieve this).
• Lack of agility in responding to new releases, due to the testing time that we need to devote to a release.
• Needed to lock in to a release at a particular time.• Spread the load across the team in terms of manual testing.
OUR CHALLENGE
Functional testing • is necessary• is time consuming• can be ambiguous
Automated testing is• consistent• easily repeatable• reusable
Thorough testing cycle each (software not deployed without testing).3 months: April – June (One window a year to achieve this).
• Lack of agility in responding to new releases, due to the testing time that we need to devote to a release.
• Needed to lock in to a release at a particular time.• Spread the load across the team in terms of manual testing.
ORGANISING TESTS
RECORDING SCRIPTS
Tests
Command
Target (Locator)
Value
Speed
EXAMPLE SCRIPT
2_LoginTestStudentAccount
open https://vle.york.ac.uk/webapps/login/?action=relogin
type id=user_id studtesta
type id=password ********
clickAndWait css=input[name="login"]
TEST MANAGER
TEST MANAGER
CONSTRUCTING GOOD TESTS
▪ A good test– Mirror user behaviour– Independent– Report accurately
▪ HTML present?▪ Text present?▪ Screenshot
SOURCE CONTROL
TEST SCRIPT (A1.3) ADD BLOG POST (ROLE STUDENT)
BLOCK 1 Login as test student
BLOCK 2 Navigate to the test course site
BLOCK 3 Navigate the left hand menu to access the blog tool
NEW SCRIPT Enter the blog and click the Add New Entry button
BLOCK 4 Add content to the content editor
BLOCK 5 Click the submit button
BLOCK 6 Take a screenshot for human inspection/verification
NEW SCRIPT Verify text on the page is present
BLOCK 7 Logout of the VLE
RUNNING THE PROCESS
Pass
Pass pending check
Fail
Test Script IDExecute
Test Suite
Test Results Web Service
Test Manager Results
TEST CASES & RUNS
Case = Browser/Platform combination
Run = Results against a particular case
TEST RESULTS
PROS AND CONS
Saved time Maintenance burden
Reusable Human intervention
TAKING THIS FORWARD
• Automate more• Maintenance• Continuous testing• Load testing• Blackboard client assisted testing?
OUR RESULTS
• 50% of tests automated (from 2013 testing cycle)
• 2014 Upgrade locking into SP16
• How much maintenance is needed, BB switch from iframe’s to HTML5 will impact on this.
• Can we extend the current % of automated tests
• Review this summer of complete process
DO THIS NEXT
Think about how the adoption of automated testing processes might benefit your institution.
Jointly (with Blackboard) could we come up with a solution to better help the testing processes within our own institutions.
THANK YOU!
Blayn ParkinsonE-Learning & training Support AssistantUniversity of York (UK)[email protected]