how to properly split test your wordpress website
TRANSCRIPT
How to properly split-test your
WordPress website
IGOR KOLOSOV
WORDCAMP VIENNA, APRIL 2015
What is split (A/B) testing?
o Traffic is split randomly and evenly between existing web page (control version) and different variations of the same page
(Widerfunnel.com definition)
oA vs. B vs. n-th variation
oAdditional note: multiarmed bandit as a test type
@IGORKO_HUB ON #WCVIE, APRIL 2015
What is multivariate testing?
o Simultaneously testing numerous variations of the page by combining web page elements (title, images, colours..)
o A1 B1 vs. A2 B1 vs. A1 B2 vs. A2 B2 etc.
@IGORKO_HUB ON #WCVIE, APRIL 2015
Why split/multivariate test?
o Data beats opinionso Test what people really do and not what they say they would do
(in usability tests)o Insights gained can help your other (marketing) effortso Valid Conversion rate optimization doesn’t exist without valid
testing
@IGORKO_HUB ON #WCVIE, APRIL 2015
Optimise or dieo Stepwise redesign (one big change every couple of years) o Gains missed between launches
vs. o Continuous design (and usefulness) improvementso Requires more resources
o Website requirements changing fast so #2 is a better option in the longer runo Establish process and the ROI won’t be a problem
Thanks Craig Sullivan (@optimiseordie) for the inspiration for the title
@IGORKO_HUB ON #WCVIE, APRIL 2015
Before testingo Quantitative analysis (answers what and how)o Web analyticso Behavior tracking and analysis (heatmaps, session recording)
o Qualitative analysis (answers why)o User feedbacko Polls, interviews
o Define sample size and test lengtho Use calculators for two-sided t-testo The only way for the test to be statistically valid is to stick to this
@IGORKO_HUB ON #WCVIE, APRIL 2015
Before testing (continued)o Seasonality or important dateso Christmas sales, Cyber Monday, etc.
o Align with business strategy/goals o determine KPIso make a testing roadmap
o Business cycles/waves lengtho How many visits and days before the sale is made?o Test for minimum one purchase cycle and/or two business cycles
@IGORKO_HUB ON #WCVIE, APRIL 2015
Before testing (continued)o Technical checko WP plugins compatibility (also consider removing unnecessary ones)o Server resourceso Browsers, devices, screen resolutionso Website performance (page load time)
@IGORKO_HUB ON #WCVIE, APRIL 2015
Before testing (resource list)
@IGORKO_HUB ON #WCVIE, APRIL 2015
Before testing (resource list continued)
@IGORKO_HUB ON #WCVIE, APRIL 2015
What to test?o Button? Background colour? Photo size or content of the photo? Headline?
Call-to-action on the button?
o Align with your goals first
o Don’t take other websites’ “good case practices“ for granted, but look for patterns on highly optimized websites (Airbnb, Amazon, Booking.com…)
@IGORKO_HUB ON #WCVIE, APRIL 2015
How to test on WordPress?o Page A vs page B (different URLs)
o Dynamic changes on page, which creates version A, B, C…
o Multivariate test – dynamic changes of elements within page
o Split test page template
o Split test website template
@IGORKO_HUB ON #WCVIE, APRIL 2015
Wordpress testing (resource list)
@IGORKO_HUB ON #WCVIE, APRIL 2015
General testing pitfalls and myths
o Stopping test as 95% significance is reachedo Sample size, business cycles, traffic volume is what determines test length
o Test results for one experiment are valid forevero Market changes so as your visitor habits, traffic sources, promotion calls-to-action; re-do
one single test from time to time (rule of thumb: once a year)
o Test without conversion rate lift is failureo Track and measure all the data and pull out conclusions: you will know what not to do any
more in the same way
@IGORKO_HUB ON #WCVIE, APRIL 2015
WordPress website testing problemso Often testing cannot run properly because of caching plugin setupo Solution: try different caching plugin or turn of caching completely (use CDN
instead); sometimes excluding certain scripts from minification is enough
o Too many (unnecessary or poorly optimized?) plugins causing webpage to load slow; combined with testing services becomes a problemo Solution: turn off plugins that you don’t need
o Testing plugin not playing nice with some other plugino Solution: try alternative plugins if possible; try other testing service
@IGORKO_HUB ON #WCVIE, APRIL 2015
Other (resource list)
@IGORKO_HUB ON #WCVIE, APRIL 2015
About me & contact info Web Analytics and Online business growth independent consultant, a psychologist working in the field of internet marketing. Passionate about analysing website visitors’ behaviour, eliminating frustrations and improving user’s online experience. Believes that randomness is just a name for an undiscovered pattern.
Certified for Google Analytics and Adwords Search, Google Partner as an agency.With 7+ years and a broad experience in CRO and web analytics, worked with more than 70 clients as a freelance consultant. Former Customer Acquisition and Web analytics guy at ManageWP.
o [email protected] igorkolosov.como http://optimize.rso Twitter: @igorko_hubo LinkedIn: http://rs.linkedin.com/in/igorkolosovo WP org: https://profiles.wordpress.org/igorkol
@IGORKO_HUB ON #WCVIE, APRIL 2015