optimost behave bootcamp: take your testing program from invisible to invincible
TRANSCRIPT
![Page 1: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/1.jpg)
Take Your Testing Program From Invisible To InvinciblePractical CRO tactics to boost your testing program’s success rate, visibility, and resources
1 Optimost.com
Uri Kogan | Zoltan Liu | David Gaskins
April 24, 2016
![Page 2: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/2.jpg)
Optimost.com
Agenda
• Introduction
• Plan and focus on the right part of your site using data
• Prioritize the order of tests you run
• Develop smart hypotheses
• Continuously Optimize your tests
• Build a culture of testing in your organization
2
![Page 3: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/3.jpg)
What would your testing program look like if you could have it all?
![Page 4: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/4.jpg)
Optimost.com
You’d Run a Ton of Tests and Make a Killing
<0%
0% -10%
10% -25%
25% -50%
50% -100%
100% -200%
200% - 300%300% - 400% 400% - 500%
$0 -$250 K
$250 K-$1M
$1 M -$5 M
$5 M -$10 M
$10 M -$25 M
$25 M -$50 M
$50 M -$75 M
$75 M -$100 M
Data from an Optimost client that ran ~200 tests in 1 year
Revenue gains of all testsLift gains of all tests
![Page 5: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/5.jpg)
Optimost.com5
If you’re doing hundreds of tests a year & making hundreds of millions of
dollars for your business, CONGRATULATIONS!
If not, this sessionIS FOR YOU
![Page 6: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/6.jpg)
Optimost.com
Highest Paid Person’s Opinion
HIPPOs
They are responsible for the death of more meaningful tests each year than all other corporate roles combined.
![Page 7: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/7.jpg)
7
“My wife was on the website last night and found the navigation confusing. We should fix this.”
An actual HIPPO quote:
– A Former CEO (yes, really)
![Page 8: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/8.jpg)
Optimost.com
RandomActs ofTestingThey definitely cause substantial ROI losses, especially in developing business.
RAT Syndrome
A
or
B
CED
![Page 9: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/9.jpg)
Optimost.com
What
9
gets in
yourway?
![Page 10: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/10.jpg)
Optimost.com
Plan and Prioritize
![Page 11: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/11.jpg)
Optimost.com
Overall Performance by Device Type: Desktop Leads
13
18,542,100Sessions
20%
Reservation Rate
$350Average Order Value
7,364,981Sessions
7%
Reservation Rate
$189Average Order Value
5,158,651Sessions
12%
Reservation Rate
$260Average Order Value
![Page 12: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/12.jpg)
Optimost.com
But Users Have Different Priorities by Device
14
Reservation
Deals
Login
Reservation
Login
Deals
Reservation
Login
Deals
1
2
3
![Page 13: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/13.jpg)
Optimost.com
Prioritize – Optimost’s SELECT Prioritization Framework
Based on the proposed experiment, what is the likelihood it will provide positive results?
Identifies the level of effort it will take to implement/execute the proposed experiment.
Estimated time to complete the proposed experiment.
Monthly revenue from 1% lift
Up to 3 custom attributes, such as customer satisfaction
SUCCESS POTENTIAL EASE OF
DEVELOPMENT LENGTH TO RUN CUSTOM TRAFFIC VALUE
![Page 14: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/14.jpg)
Optimost.com
Smart Hypotheses
![Page 15: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/15.jpg)
Optimost.com
Smart hypotheses
17
What does a Smart Hypothesis include? An easy to use format1
•Clearly defined testing objective: States how the test will likely generate lift, based on the specific testing goal or Key Performance Indicator (KPI)
•Strong rationale: Justifies why the intended element is being tested
•Specific target audience: Determines who the target audience is and builds off this visitor knowledge to inform the testing concept
•Knowledgeable approach: Uses insights garnered from other tests or best practices to build on the current test
Because we observed[data] and [feedback],
we believe that [doing]
for [people]
will result in [outcome].
Sources: Craig Sullivan, twitter.com/optimiseordie/status/633408692201852929, and Mark Fassbender, www.linkedin.com/in/flashbender
![Page 16: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/16.jpg)
Optimost.com
Smart hypotheses: Example
• Test objectiveDetermine if reversing the order of the “Price Matrix” columns would increase the average order value without hurting conversions
• RationaleThe testing team believed viewers weren’t even reviewing the premium options. Would putting the premium coverage level on the left prompt customers to see all the options and select the premium options more frequently?
• Knowledgeable approachThe test was influenced by the work of Robert Cialdini. He found that presenting the highest price on the left worked well.
![Page 17: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/17.jpg)
Optimost.com
Smart hypotheses: Example
Because we observed [customers weren’t interacting with all premium coverage levels available],
we believe that [moving the coverage order so it reads from premium to basic]
for [all users]
will result in [higher income and better customer satisfaction scores].
![Page 18: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/18.jpg)
Optimost.com
Which one boosted conversions? Which one boosted revenue?
A
B
![Page 19: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/19.jpg)
Optimost.com
Which one boosted conversions? Which one boosted revenue?
A
B
Result: Customers chose premium coverage 5.4% more frequently & conversions rose 13%.
![Page 20: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/20.jpg)
Optimost.com
What’s your smart hypothesis?
Because we observed [data] and [feedback],we believe that [doing]for [people]will result in [outcome].
Bring your questions to our expert 1:1s
![Page 21: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/21.jpg)
Continuous Optimization
![Page 22: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/22.jpg)
Optimost.com
Test 1 – Message & CTA
433%INCREASE IN CLICKS
![Page 23: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/23.jpg)
Optimost.com
Test 2 – Value Prop
4%INCREASE IN CLICKS
![Page 24: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/24.jpg)
Optimost.com
Test 3 - MVT
71%INCREASE IN CLICKS
![Page 25: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/25.jpg)
Optimost.com
Test 4 – Geo-targeting
+30.71%Geo-targeted images in San Francisco
+17.83%Geo-targeted images in New York increased
31%INCREASE IN CLICKS
![Page 26: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/26.jpg)
Build a Culture of Testing
![Page 27: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/27.jpg)
Optimost.com
Excite and Involve your Entire Organization!
![Page 28: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/28.jpg)
Optimost.com
Wrong:“This test achieved a 9% boost in adds to cart.”
Better: “Every 1% boost in adds to cart equals $10 million in annual revenue. This test, annualized, could bring in $90 million in annual revenue”
Get Buy In!
Every
1%BOOST
In adds to cart
$10MILLION
Annual revenue
$90MILLION
Annual revenue
Use terms that matter to your leaders and help finance and your CEO understand. Help them understand why you test in terms of their KPIs.
![Page 29: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/29.jpg)
Want to learn more?Meet Jim
optimost.com/tour
![Page 30: Optimost BEHAVE Bootcamp: Take Your Testing Program From Invisible To Invincible](https://reader031.vdocuments.site/reader031/viewer/2022030314/5899bc271a28aba11e8b73d3/html5/thumbnails/30.jpg)
Thank You!
linkedin.com/in/kogan
@UriDXM
linkedin.com/in/zoltanliu linkedin.com/in/david-gaskins-0b08761