How to properly split test your WordPress website

Download How to properly split test your WordPress website

Post on 08-Aug-2015




6 download


  1. 1. How to properly split-test your WordPress website IGOR KOLOSOV WORDCAMP VIENNA, APRIL 2015
  2. 2. What is split (A/B) testing? o Traffic is split randomly and evenly between existing web page (control version) and different variations of the same page ( definition) oA vs. B vs. n-th variation oAdditional note: multiarmed bandit as a test type @IGORKO_HUB ON #WCVIE, APRIL 2015
  3. 3. What is multivariate testing? o Simultaneously testing numerous variations of the page by combining web page elements (title, images, colours..) o A1 B1 vs. A2 B1 vs. A1 B2 vs. A2 B2 etc. @IGORKO_HUB ON #WCVIE, APRIL 2015
  4. 4. Why split/multivariate test? o Data beats opinions o Test what people really do and not what they say they would do (in usability tests) o Insights gained can help your other (marketing) efforts o Valid Conversion rate optimization doesnt exist without valid testing @IGORKO_HUB ON #WCVIE, APRIL 2015
  5. 5. Optimise or die o Stepwise redesign (one big change every couple of years) o Gains missed between launches vs. o Continuous design (and usefulness) improvements o Requires more resources o Website requirements changing fast so #2 is a better option in the longer run o Establish process and the ROI wont be a problem Thanks Craig Sullivan (@optimiseordie) for the inspiration for the title @IGORKO_HUB ON #WCVIE, APRIL 2015
  6. 6. Before testing o Quantitative analysis (answers what and how) o Web analytics o Behavior tracking and analysis (heatmaps, session recording) o Qualitative analysis (answers why) o User feedback o Polls, interviews o Define sample size and test length o Use calculators for two-sided t-test o The only way for the test to be statistically valid is to stick to this @IGORKO_HUB ON #WCVIE, APRIL 2015
  7. 7. Before testing (continued) o Seasonality or important dates o Christmas sales, Cyber Monday, etc. o Align with business strategy/goals o determine KPIs o make a testing roadmap o Business cycles/waves length o How many visits and days before the sale is made? o Test for minimum one purchase cycle and/or two business cycles @IGORKO_HUB ON #WCVIE, APRIL 2015
  8. 8. Before testing (continued) o Technical check o WP plugins compatibility (also consider removing unnecessary ones) o Server resources o Browsers, devices, screen resolutions o Website performance (page load time) @IGORKO_HUB ON #WCVIE, APRIL 2015
  9. 9. Before testing (resource list) @IGORKO_HUB ON #WCVIE, APRIL 2015
  10. 10. Before testing (resource list continued) @IGORKO_HUB ON #WCVIE, APRIL 2015
  11. 11. What to test? o Button? Background colour? Photo size or content of the photo? Headline? Call-to-action on the button? o Align with your goals first o Dont take other websites good case practices for granted, but look for patterns on highly optimized websites (Airbnb, Amazon, @IGORKO_HUB ON #WCVIE, APRIL 2015
  12. 12. How to test on WordPress? o Page A vs page B (different URLs) o Dynamic changes on page, which creates version A, B, C o Multivariate test dynamic changes of elements within page o Split test page template o Split test website template @IGORKO_HUB ON #WCVIE, APRIL 2015
  13. 13. Wordpress testing (resource list) @IGORKO_HUB ON #WCVIE, APRIL 2015
  14. 14. General testing pitfalls and myths o Stopping test as 95% significance is reached o Sample size, business cycles, traffic volume is what determines test length o Test results for one experiment are valid forever o Market changes so as your visitor habits, traffic sources, promotion calls-to-action; re-do one single test from time to time (rule of thumb: once a year) o Test without conversion rate lift is failure o Track and measure all the data and pull out conclusions: you will know what not to do any more in the same way @IGORKO_HUB ON #WCVIE, APRIL 2015
  15. 15. WordPress website testing problems o Often testing cannot run properly because of caching plugin setup o Solution: try different caching plugin or turn of caching completely (use CDN instead); sometimes excluding certain scripts from minification is enough o Too many (unnecessary or poorly optimized?) plugins causing webpage to load slow; combined with testing services becomes a problem o Solution: turn off plugins that you dont need o Testing plugin not playing nice with some other plugin o Solution: try alternative plugins if possible; try other testing service @IGORKO_HUB ON #WCVIE, APRIL 2015
  16. 16. Other (resource list) @IGORKO_HUB ON #WCVIE, APRIL 2015
  17. 17. About me & contact info Web Analytics and Online business growth independent consultant, a psychologist working in the field of internet marketing. Passionate about analysing website visitors behaviour, eliminating frustrations and improving users online experience. Believes that randomness is just a name for an undiscovered pattern. Certified for Google Analytics and Adwords Search, Google Partner as an agency. With 7+ years and a broad experience in CRO and web analytics, worked with more than 70 clients as a freelance consultant. Former Customer Acquisition and Web analytics guy at ManageWP. o o o o Twitter: @igorko_hub o LinkedIn: o WP org: @IGORKO_HUB ON #WCVIE, APRIL 2015