![Page 1: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/1.jpg)
© 2012 Webtrends, All Rights Reserved. #optimizeplan @joestanhope @bob_garcia
Three-Step Framework for Overcoming Challenges of Site Optimization
June 19th, 2012
![Page 2: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/2.jpg)
© 2012 Webtrends, All Rights Reserved. |2#optimizeplan @joestanhope @bob_garcia
“
It’s all about relevance
Relevance means understanding
a consumer’s digital behavior
across channels and then being
able to use that intelligence to
optimize digital marketing.
“
![Page 3: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/3.jpg)
© 2012 Webtrends, All Rights Reserved. |3#optimizeplan @joestanhope @bob_garcia
Webtrends Focus
Enabling digital marketers to deliver real-time relevance to their customers
![Page 4: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/4.jpg)
© 2012 Webtrends, All Rights Reserved. |4#optimizeplan @joestanhope @bob_garcia
what?who?
how?
![Page 5: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/5.jpg)
© 2012 Webtrends, All Rights Reserved. |5#optimizeplan @joestanhope @bob_garcia
Where Does Optimization Fit?
Analytics
Identify and quantify opportunities thru insight
Optimization
Test, segment and target to improve performance
![Page 6: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/6.jpg)
© 2012 Webtrends, All Rights Reserved. |6#optimizeplan @joestanhope @bob_garcia
Site Optimization Includes …
![Page 7: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/7.jpg)
© 2012 Webtrends, All Rights Reserved. |7#optimizeplan @joestanhope @bob_garcia
Optimization Team
Optimization Consultant
• Strategy and roadmap • Analytics evaluation• Optimization project lead• Campaign performance / KPIs
Web Developer
• Build out/development• Implementation• QA
Designer
• Creative strategy• Asset generation• Usability
QA
• Verify against plan• Multiple Browsers• Conversion Checks
![Page 8: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/8.jpg)
© 2012 Webtrends, All Rights Reserved. |8#optimizeplan @joestanhope @bob_garcia
Joe Stanhope (Guest Speaker)+1 617.613.8929
@joestanhope
www.forrester.com
blogs.forrester.com/joseph_stanhope
PresentersForrester + Webtrends
Bob Garcia+ 1 877.932.8736
@bob_garcia
www.webtrends.com
blogs.webtrends.com
![Page 9: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/9.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited9 © 2009 Forrester Research, Inc. Reproduction Prohibited
A Complete Approach To Site Optimization: Anticipating Challenges and Harnessing Opportunities To Build An Online Testing Program
Joe Stanhope, Senior Analyst, Forrester Research
June 19, 2012
![Page 10: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/10.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited10
Website traffic is a valuable commodity. To make the most of this
asset, ensure that your marketing and customer experience efforts will
resonate with visitors and drive measurable benefits.
#optimizeplan
@bob_garcia
@joestanhope
![Page 11: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/11.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited11
What is Online Testing & Why Do You Need It?
Online Testing Maturity: The Scale Conundrum
Success Factor 1: Process
Success Factor 2: Organization
Agenda
#optimizeplan
@bob_garcia
@joestanhope
![Page 12: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/12.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited12Image source: Ursus Wehrli, The Art of Clean Up
![Page 13: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/13.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited13Image source: Ursus Wehrli, The Art of Clean Up
![Page 14: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/14.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited14
What is online testing?
Online testing defined: A statistical methodology that allows site owners to
test the impact of specific Web site components on target metrics
The primary experimental techniques of online testing are:
– A/B testing - Test multiple variations of a site
element against one another
– Multivariate testing - Test multiple variations
of multiple site elements against one
another simultaneously
#optimizeplan
@bob_garcia
@joestanhope
![Page 15: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/15.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited15Image source: www.carbodydesign.com
![Page 16: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/16.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited16
How is online testing useful?
Meet marketing or campaign goals (e.g. conversion, sales,
click-throughs, etc.)
#optimizeplan
@bob_garcia
@joestanhope
![Page 17: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/17.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited17
How is online testing useful?
Meet marketing or campaign goals (e.g. conversion, sales,
click-throughs, etc.)
Improve customer experience
#optimizeplan
@bob_garcia
@joestanhope
![Page 18: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/18.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited18
How is online testing useful?
Meet marketing or campaign goals (e.g. conversion, sales,
click-throughs, etc.)
Improve customer experience
Validate segmentation strategies
#optimizeplan
@bob_garcia
@joestanhope
![Page 19: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/19.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited19
How is online testing useful?
Meet marketing or campaign goals (e.g. conversion, sales,
click-throughs, etc.)
Improve customer experience
Validate segmentation strategies
Statistically based insight into what works and what doesn’t
– Understanding of:
– Factor importance
– Factor interaction
#optimizeplan
@bob_garcia
@joestanhope
![Page 20: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/20.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited20
Testing addresses a variety of challenges
But users test primarily to enhance revenue and usability
3%
23%
39%
42%
42%
58%
70%
Other, please specify
Marketing integration with other online channels (suchas search, email, and offsite display advertising)
Content relevance (such ascontent presentedonsite, searchresults, or on landing pages)
Visitor segmentation and targeting
Overall customer engagement and satisfaction
Siterevenue generation (such as via offers,promotions and cross sell/up sell)
Site usability (such as site design, navigationand multistep processes)
“Whichtopicsrepresent the three greatest challenges thatyou seektoresolve withyour onlinetesting program?”
Base: 71respondents who use an online testing platform(multiple responses accepted)
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 21: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/21.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited21
Testing works for sites of all sizes
10%
14%
20%
11%
18%
13%
6%
7%
Upto 50,000 unique visitors
Upto 250,000 unique visitors
Upto 1 million unique visitors
Upto 2.5 million unique visitors
Upto 5 million unique visitors
Upto 10 million unique visitors
Upto 50 million unique visitors
Over 50 million unique visitors
Base: 71respondents who use an online testing platform(percentages may not total 100 because of rounding)
“Onaverage, what is the monthlyvolume of unique visitors on theweb propertieson whichtesting is performed?”
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 22: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/22.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited22
Ultimately, online testing is effective
3%
6%
9%
10%
20%
20%
21%
24%
28%
39%
73%
No benefits
Other, please specify
Increased customer satisfaction (surveycontrolledgroup of customers)
Increased formcompletion (form submittals/visitsthat start form)
Increased site based searchconversion (ordersattributedto searchers/visitsto searchresults page)
Increasedregistration (registrations/visits)
Improved customer segmentation and targeting
Increasedaverage ordervalue (online revenue/orders)
Increased landing page conversion(orders/visitsto landing page)
Increasedconversion (orders/visits)
Base: 71respondents who use an online testing platform(multiple responses accepted)
“What are thetop three benefitsyourcompany hasrealized through A/B and/or multivariatetesting?”
Increased off-site display ad clickthough(clicks/ad impressions)
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 23: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/23.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited23
Ultimately, online testing is effective
3%
6%
9%
10%
20%
20%
21%
24%
28%
39%
73%
No benefits
Other, please specify
Increased customer satisfaction (surveycontrolledgroup of customers)
Increased formcompletion (form submittals/visitsthat start form)
Increased site based searchconversion (ordersattributedto searchers/visitsto searchresults page)
Increasedregistration (registrations/visits)
Improved customer segmentation and targeting
Increasedaverage ordervalue (online revenue/orders)
Increased landing page conversion(orders/visitsto landing page)
Increasedconversion (orders/visits)
Base: 71respondents who use an online testing platform(multiple responses accepted)
“What are thetop three benefitsyourcompany hasrealized through A/B and/or multivariatetesting?”
Increased off-site display ad clickthough(clicks/ad impressions)
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 24: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/24.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited2424Entire contents © 2010 Forrester Research, Inc. All rights reserved.
Source: Insurance Institute for Highway Safety
![Page 25: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/25.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited2525Entire contents © 2010 Forrester Research, Inc. All rights reserved.
Source: Insurance Institute for Highway Safety
Online Testing Maturity: The Scale
Conundrum
![Page 26: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/26.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited26
Online Testing Program Longevity Is Diverse But Skews YoungFebruary 2012 “The State Of Online Testing 2012”
#optimizeplan
@bob_garcia
@joestanhope
![Page 27: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/27.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited27
Few Online Testing Users Run Large Numbers Of ExperimentsFebruary 2012 “The State Of Online Testing 2012”
#optimizeplan
@bob_garcia
@joestanhope
![Page 28: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/28.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited28
“Onaverage, approximately how many A/B and/or multivariatetestsdoyou runconcurrently onyourweb properties?
4%
39%
32%
7%
3%
3%
1%
0%
10%
1-2
3-5
6-8
9-11
12-14
15-17
18-19
20 or more
0
Base: 71respondents who use an online testing platform(percentages may not total 100 because of rounding)
Concurrency is low
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 29: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/29.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited29
1%
20%
20%
21%
27%
41%
42%
47%
48%
51%
73%
Other
Breadth of available online testing techniques
Strength of vendor’sreference customers
Availability of advanced or proprietaryfeatures andfunctionality
Vendor’sreputation (e.g.known as a leader in this field)
The online testing application is a module within a largersuite of analytics and/or marketing tools
Test deployment method (i.e.type of site tags, etc.)
Vendor’s abilityto servicemy business needs
Total cost of ownership
Ease of use and automation
Vendor’s thought leadership
User interface
Professional services, managed services,and training capabilities
Standard features and functionality (such asreporting,segmentation and targeting)
Ease of implementation and ongoing management
Base: 71respondents who use an online testing platform(multiple responses accepted)
“Which of thefollowing are the five most important factors whenselecting an onlinetesting solution?”
Experience in our industry
17%
17%
14%
13%
11%
10%
Product support capabilities
Selection criteria reflects maturity and support needs
Steady
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 30: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/30.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited30
1%
20%
20%
21%
27%
41%
42%
47%
48%
51%
73%
Other
Breadth of available online testing techniques
Strength of vendor’sreference customers
Availability of advanced or proprietaryfeatures andfunctionality
Vendor’sreputation (e.g.known as a leader in this field)
The online testing application is a module within a largersuite of analytics and/or marketing tools
Test deployment method (i.e.type of site tags, etc.)
Vendor’s abilityto servicemy business needs
Total cost of ownership
Ease of use and automation
Vendor’s thought leadership
User interface
Professional services, managed services,and training capabilities
Standard features and functionality (such asreporting,segmentation and targeting)
Ease of implementation and ongoing management
Base: 71respondents who use an online testing platform(multiple responses accepted)
“Which of thefollowing are the five most important factors whenselecting an onlinetesting solution?”
Experience in our industry
17%
17%
14%
13%
11%
10%
Product support capabilities
Selection criteria reflects maturity and support needs
Steady
On the Rise
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 31: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/31.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited31
Components of an online testing program
Each element is crucial
#optimizeplan
@bob_garcia
@joestanhope
![Page 32: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/32.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited32
Components of an online testing program
Culture of measurement and innovation
– Online testing is an iterative program, not a discrete test
– Online testing is an experiment, not a pass/fail exercise
Each element is crucial
#optimizeplan
@bob_garcia
@joestanhope
![Page 33: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/33.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited33
Components of an online testing program
Culture of measurement and innovation
– Online testing is an iterative program, not a discrete test
– Online testing is an experiment, not a pass/fail exercise
Process
– Plan, design, execute, analyze, validate, repeat
– Marketing input
– Program goals
Each element is crucial
#optimizeplan
@bob_garcia
@joestanhope
![Page 34: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/34.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited34
Components of an online testing program
Culture of measurement and innovation
– Online testing is an iterative program, not a discrete test
– Online testing is an experiment, not a pass/fail exercise
Process
– Plan, design, execute, analyze, validate, repeat
– Marketing input
– Program goals
Organization
– Defined ownership for the program is critical
– Skills to support the process
Each element is crucial
#optimizeplan
@bob_garcia
@joestanhope
![Page 35: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/35.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited35
Components of an online testing program
Culture of measurement and innovation
– Online testing is an iterative program, not a discrete test
– Online testing is an experiment, not a pass/fail exercise
Process
– Plan, design, execute, analyze, validate, repeat
– Marketing input
– Program goals
Organization
– Defined ownership for the program is critical
– Skills to support the process
Technology
– Site management
– Analytics
Each element is crucial
#optimizeplan
@bob_garcia
@joestanhope
![Page 36: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/36.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited36
![Page 37: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/37.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited37
Success Factor 1: Process
![Page 38: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/38.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited38
14%
25%
28%
32%
Conducting tests is predominantly an ad hoc process
Testing followsformalized processesfor some tasksbut is mostly ad hoc
Testing follows a highly formalized process throughoutthe development, execution, and analysis of experiments
Testing followsformalized processesfor most tasks,but some tasks are ad hoc
Base: 71respondents who use an online testing platform(percentages may not total 100 because of rounding)
“To what degree doesyour firm adhereto a structured processtoconduct onlinetesting?”
The majority of firms struggle with processes
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 39: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/39.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited39
The elements of a testing plan
#optimizeplan
@bob_garcia
@joestanhope
![Page 40: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/40.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited40
The process is critical
#optimizeplan
@bob_garcia
@joestanhope
![Page 41: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/41.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited41
Online Testing Processes
Who are the stakeholders?
How do we ideate experiments?
What is the criteria for prioritizing experiments?
Can we get support from existing process management expertise such as
our PMO?
How does the team stay informed?
What information populates our online testing knowledgebase?
It’s important to consider:
#optimizeplan
@bob_garcia
@joestanhope
![Page 42: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/42.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited42Image source: Hostile Work Environment (http://hostilework.blogspot.com/)
![Page 43: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/43.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited43
Success Factor 2: Organization
Image source: Hostile Work Environment (http://hostilework.blogspot.com/)
![Page 44: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/44.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited44
3%
14%
14%
14%
16%
17%
18%
18%
18%
30%
34%
Usability of thetesting interface
Other
Abilityto segment/target test participants
Lack of corporate buy-in
Modifyingweb pagesfortesting
Establishing testing scenarios (i.e.concept, areasto test)
Acting upon testing results
Securing ITresourcesto deploy and manage testcodeon thewebsite
Demonstrating ROI
Interpreting test results
Developing testcontent (such as creative elementsand/or promotionsto serve astestvariations)
Ability of testing application to accommodate desiredtesting techniques and designs
Attaining enough site trafficfor statistically significant tests
Lack of staffto manage the testing program and utilizetesting applications
Prioritizing testing initiatives
Base: 71respondents who use an online testing platform(multiple responses accepted)
“What challenges doyouexperience in operatingyourcompany'sA/B and/or multivariatetesting application?
No challenges
13%
10%
7%
6%
6%
4%
Securing funding for testing initiatives
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 45: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/45.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited45
1%
6%
7%
10%
20%
24%
32%
IT
Dedicated onlinetestingor site optimizationgroup
Other
Business intelligence orcorporate analyticsgroup
Web analyticsgroup
eCommerce
Marketing
Base: 71respondents who use an online testing platform
“What department inyour firm isresponsiblefor onlinetesting?”
Online testing is part of the business
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 46: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/46.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited46
The Ownership Model SpectrumAugust 2010 “Organizing For Site Optimization”
#optimizeplan
@bob_garcia
@joestanhope
![Page 47: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/47.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited47
3%
11%
14%
17%
20%
35%
Other, please describe
A single full time resource
Acombination of part and full time resources
A single part timeresource
Multiple full time resources
Multiple part timeresources
Base: 71respondents who use an online testing platform
“How is the onlinetesting program staffedatyour firm?”
Online testing is a team effort
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 48: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/48.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited48
1%
1%
6%
20%
35%
37%
Allow external agencyto manage the entire process
Other, please specify
Allowvendorto manage the entire process
Work with an external agency orconsultancy
Manage entirely in house
Work with our vendor’s professional servicesgroup
Base: 71respondents who use an online testing platform
“What isyour preferred method of managing A/B and/or multivariatetesting onyourwebsite(s)?”
Most firms leverage third party resources
Source: Q4 2011 Global online Testing User Online Survey
#optimizeplan
@bob_garcia
@joestanhope
![Page 49: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/49.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited49
How can a third party service provider help?
Services to support all stages of online testing
#optimizeplan
@bob_garcia
@joestanhope
![Page 50: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/50.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited50
How can a third party service provider help?
Technical services
– Implementation
– Training
– Integrations
Services to support all stages of online testing
#optimizeplan
@bob_garcia
@joestanhope
![Page 51: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/51.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited51
How can a third party service provider help?
Technical services
– Implementation
– Training
– Integrations
Program support
– Test ideation and prioritization
– Test design
– Creative services
– Experiment analysis
– Strategic advisory
Services to support all stages of online testing
#optimizeplan
@bob_garcia
@joestanhope
![Page 52: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/52.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited52
How can a third party service provider help?
Technical services
– Implementation
– Training
– Integrations
Program support
– Test ideation and prioritization
– Test design
– Creative services
– Experiment analysis
– Strategic advisory
Managed services
Services to support all stages of online testing
#optimizeplan
@bob_garcia
@joestanhope
![Page 53: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/53.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited53
Third Party Services
Which components of the testing program would benefit from external
support?
How much help do you need to support a robust online testing program?
How much control do you want?
Do we want to become self-sufficient (eventually)?
What is the opportunity cost of delayed testing versus the expense of
services?
It’s important to consider:
#optimizeplan
@bob_garcia
@joestanhope
![Page 54: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/54.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited54
Make the business case for testingProve the concept with a modest, well chosen pilotJustify investment to grow and overcome challenges
Start with the basicsWork the maturity curve from A/B to Multivariate testingBuild up cadence, complexity, and concurrency over time
Online testing programs require ownershipA named resource must own the online testing program, even if in a
part-time capacity
Online testing is a people businessRealistically assess your current organization and skills relative to
managing an effective online testing program
![Page 55: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/55.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited55
Make the business case for testingProve the concept with a modest, well chosen pilotJustify investment to grow and overcome challenges
Start with the basicsWork the maturity curve from A/B to Multivariate testingBuild up cadence, complexity, and concurrency over time
Online testing programs require ownershipA named resource must own the online testing program, even if in a
part-time capacity
Online testing is a people businessRealistically assess your current organization and skills relative to
managing an effective online testing program
![Page 56: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/56.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited56
Make the business case for testingProve the concept with a modest, well chosen pilotJustify investment to grow and overcome challenges
Start with the basicsWork the maturity curve from A/B to Multivariate testingBuild up cadence, complexity, and concurrency over time
Online testing programs require ownershipA named resource must own the online testing program, even if in a
part-time capacity
Online testing is a people businessRealistically assess your current organization and skills relative to
managing an effective online testing program
![Page 57: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/57.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited57
Make the business case for testingProve the concept with a modest, well chosen pilotJustify investment to grow and overcome challenges
Start with the basicsWork the maturity curve from A/B to Multivariate testingBuild up cadence, complexity, and concurrency over time
Online testing programs require ownershipA named resource must own the online testing program, even if in a part-time capacity
Online testing is a people businessRealistically assess your current organization and skills relative to
managing an effective online testing program
![Page 58: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/58.jpg)
© 2012 Forrester Research, Inc. Reproduction Prohibited58
Make the business case for testingProve the concept with a modest, well chosen pilotJustify investment to grow and overcome challenges
Start with the basicsWork the maturity curve from A/B to Multivariate testingBuild up cadence, complexity, and concurrency over time
Online testing programs require ownershipA named resource must own the online testing program, even if in a part-time capacity
Online testing is a people businessRealistically assess your current organization and skills relative to managing an effective online testing program
![Page 59: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/59.jpg)
© 2012 Webtrends, All Rights Reserved. #optimizeplan @joestanhope @bob_garcia
Summary + Close
![Page 60: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/60.jpg)
© 2012 Webtrends, All Rights Reserved. |60#optimizeplan @joestanhope @bob_garcia
Summary - Why It Matters?
1. Make More Money – You are likely leaving money on the table
We often deliver 20-50% lift translating to thousands and millions of new found revenue.
2. Customer Experience – Customers own your future
A business that listens and relates will win. What are your competitors doing?
3. Eliminate Risk – You’re taking big and unnecessary risks
Not only does testing make money, but it helps you avoid making costly mistakes.
![Page 61: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/61.jpg)
© 2012 Webtrends, All Rights Reserved. |61#optimizeplan @joestanhope @bob_garcia
British fashion drives
e-commerce relevance
Business Challenge
Reduce cart abandonment and encourage repeat
website visitors for new merchandise launches,
both on the UK and USA sites.
Solution
Webtrends Optimize for testing and targeting
during both sales events and non-sales time
periods, and for testing website components for
best responses.
![Page 62: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/62.jpg)
© 2012 Webtrends, All Rights Reserved. |62#optimizeplan @joestanhope @bob_garcia
Gearing Emails for
One-to-One Merchandising
Business Challenge
Find more effective ways to act on rich customer
data and provide dynamic, one-to-one offers based
on individual views, searches and cart activity
Solution
Webtrends Analytics identifies promotional
opportunities for products that generate a high
number of views, searches or purchases.
![Page 63: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/63.jpg)
© 2012 Webtrends, All Rights Reserved. #optimizeplan @joestanhope @bob_garcia
Great Forrester Quotes
![Page 64: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/64.jpg)
© 2012 Webtrends, All Rights Reserved. |64#optimizeplan @joestanhope @bob_garcia
Realities …
“Testing Programs Require Investment to Achieve Scale.”
“… experience, process, and resource constraints throttle their efforts to scale up online testing program.”
“… firms must commit to investing in online testing programs to achieve scale and long-term success.”
“… testing delivers measureable benefits across a range of success metrics”
Forrester Research, Inc., The State of Online Testing 2011, February 2012
![Page 65: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/65.jpg)
© 2012 Webtrends, All Rights Reserved. #optimizeplan @joestanhope @bob_garcia
Q&A
#OptimizePlan | @joestanhope | @bob_garcia
![Page 66: Three-Step Framework for Overcoming Challenges of Site Optimization](https://reader034.vdocuments.site/reader034/viewer/2022042821/55d51507bb61eb616b8b45fa/html5/thumbnails/66.jpg)
© 2012 Webtrends, All Rights Reserved. |66#optimizeplan @joestanhope @bob_garcia
Joe Stanhope (Guest Speaker)+1 617.613.8929
@joestanhope
www.forrester.com
blogs.forrester.com/joseph_stanhope
Thank YouForrester + Webtrends
Bob Garcia+ 1 877.932.8736
@bob_garcia
www.webtrends.com
blogs.webtrends.com