a/b testing for ecommerce: features of convert experiments

Download A/B Testing for Ecommerce: Features of Convert Experiments

Post on 12-Aug-2015

467 views

Category:

Retail

0 download

Embed Size (px)

TRANSCRIPT

  1. 1. A/B Testing for Ecommerce Features of Convert Experiments Brought to you by:
  2. 2. Dennis van der Heijden Founder & CEO of Convert.com @Convert #ConvertAcademy
  3. 3. Prepare as Agency for Ecommerce Clients We like to help you prepare for new ecommerce clients coming to your agency. We want you to be awesome and always make you look great in front of your clients. Prepare well this webinar helps.. Purpose of Webinar
  4. 4. Convert.com Agency and Ecommerce Focus
  5. 5. How A/B Testing Tools Work Most of what I share you can use with: Convert Optimizely VWO Monetate Adobe and other Javascript web-based tools. Disclaimer: Im super happy you are testing with Convert or our colleagues, that means you are in the top 5% of world's marketers. Now its hard to not mention differences between the tools forgive me, I just think Convert Experiments is a better fit for many CRO agencies. www.convert.com
  6. 6. Packed Presentation, Lets Get Started Myths and Half-Truths About Async/Sync, Tag Managers and Speed 1. Install all codes at once: o Tracking code & Locations URL and header o Revenue tracker o Mapping of CMS o Goal / Event JS o Security documentation o AJAX refresh codes o Information on Tag Managers 2. Precheck for Advanced Installs o Non structured URLs o AJAX load of content and goals o Testing behind login o Tag Manager Fans o Subdomains and cross domains o Offsite revenue tracking 3. Inner workings of A/B testing tools o Async and sync codes o Avoid seeing the blink, what hack do A/B testing tools use? www.convert.com
  7. 7. Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Tracking Code & Locations URL Send the tracking code + Checklist of URLs: 1. Homepage 2. Category Page 3. Product Page 4. Cart Page 5. Checkout 6. Revenue / Thank You Page
  8. 8. Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Yes, In Header Please -tag More in this later, but specify in email to client Check each page for presence of snippet: http://robhammond.co/tools/seo-crawler http://www.screamingfrog.co.uk/
  9. 9. Script in Header Being the first to load allows A/B testing tool to manipulate the content following and make the changes without blinking. It is blocking and yes it is hurting site-speed, later on why that hardly matters... www.convert.com
  10. 10. Convert Tracks GA Automatically GA Universal GA Classic Manual Tracking Script Default One Time Revenue Event http://j.mp/1FcldYq Optional Multiple Revenue Events Ask support@convert.com Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Revenue Tracker
  11. 11. Basic Script Advanced Script: Mapping Page Type Category ID Category Name Product SKU Product Name Product Price Customer ID Custom Field 1 (text) Custom Field 2 (text) Custom Field 3 (number) Custom Field 4 (number) Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Mapping of CMS http://j.mp/1JJgmWt
  12. 12. Segmentation or Goal Events: Pop-ups / Hovers Ajax Goals Multi-step Checkout Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Goal / Event JS
  13. 13. Ask Support for latest version: Technical Documentation System Implementation Guide (PDF) Tackle every IT department question on Day 1 Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Security documentation
  14. 14. Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com AJAX refresh codes 1. Experiment Done When Page Loads 2. Ajax Loads After 3. Trigger Experiment Search & Replace Again http://j.mp/1FNrds2
  15. 15. Get one big Pack in the Hands of Clients Nothing is more frustrating for a client getting 20 separate emails with install details! www.convert.com Information on Tag Managers Tealium Awesome Adobe Great Rest not recommended http://j.mp/1C25Amh Loading Async or on DOM-ready THAT'S TOO LATE -> BLINKING
  16. 16. Precheck for Advanced Installs
  17. 17. Dont Blame the Tool Prepare Well
  18. 18. Precheck the Client Website for A/B Testing Warning signs you should monitor before quotation The following site signals will make the implementation harder: Non structured URLs AJAX load of content and goals Testing behind login Tag Manager Fans Subdomains and cross domains Offsite revenue tracking www.convert.com
  19. 19. Non Structured URLs SEO friendly URLs a good idea. But hard for some tools www.convert.com Structured Ecommerce URLs shop.com/product/jeans-1234.php shop.com?prod=1234&name=jeans Unstructured Ecommerce URLs shop.com/jeans-blue-awesome shop.com/special-jeans-june
  20. 20. AJAX Load of Content and Goals Test or Monitor Ajax, Hovers or Pop-ups? Testing Tools Prefer Refresh www.convert.com
  21. 21. Subdomains and Cross Domains Convert Experiments Auto-Reconnect - Add to Settings and Done www.convert.com
  22. 22. Testing Behind Login, No Problem Have Two Tabs/Sessions Open + Script Installed? Convert Experiments has Cross Session Setup www.convert.com
  23. 23. Tag Manager Fans? Tealium & Adobe :-) - Others :-( So -> Load Direct From www.convert.com
  24. 24. Offsite Goal & Revenue Tracking Pass Experiment ID and Variation ID to Backend and Send Webhook www.convert.com
  25. 25. The Inner Workings of A/B Testing Tools psst, please dont tell...
  26. 26. The Dirty Secrets and Misdirections
  27. 27. Understanding the terminology is super important when selecting a new A/B testing tool. There is a reason smart agencies go deep on sync/async parts of the script and how tools hide their true technology from you? Let's talk about: Do you really understand sync and async? Rethink your current A/B testing tools impact... How do we trick your eyes and make SmartInsert possible? Dont Copy Low Quality A/B Testing Articles How do we really affect site speed and why to some tools blink? www.convert.com
  28. 28. Understand How Page Loads Thanx New Relic, for image www.convert.com
  29. 29. A/B Testing Tools Technology How does Convert Patent Pending SmartInsert Work? www.convert.com Click to add text. Click to add text. Click to add text. Click to add text. 0.30 Sec. Nothing... 0.35 Sec. Convert Starts Hide Body for 1 Sec. Check Every 0.05 Sec. Unhide Body 1.50 Sec. Convert Long Done 0.30 Sec. DOMloading 0.35 Sec. DOMcontent Loaded Start 1.50 Sec. DOMcontent Loaded End 1.80 Sec. DOMcomplete Fast Website + Convert Experiments
  30. 30. A/B Testing Tools Technology How other A/B testing Tools Work example #1 www.convert.com Click to add text. Click to add text. Click to add text. Click to add text. 0.30 Sec. Nothing 0.35 Sec. Tool Starts Makes All Changes Visible Changes Swapping / Blinking 1.50 Sec. Tool #1 Done 0.30 Sec. DOMloading 0.35 Sec. DOMcontent Loaded Start 1.50 Sec. DOMcontent Loaded End 1.80 Sec. DOMcomplete Fast Website + Other A/B Testing Tool #1
  31. 31. A/B Testing Tools Technology How other A/B testing Tools Work example #2 www.convert.com Click to add text. Click to add text. Click to add text. Click to add text. 0.30 Sec. Nothing... 0.35 Sec. Nothing 1.50 Sec. A/B Testing Tool Loads Hide Body for 1 Sec. Make All Changes. 1.80 Sec. Body Still Hidden 2.50 Sec. + 0.7 Sec. White Body Unhide 0.30 Sec. DOMloading 0.35 Sec. DOMcontent Loaded Start 1.50 Sec. DOMcontent Loaded End 1.80 Sec. DOMcomplete Fast Website + Other A/B Testing Tool #2
  32. 32. Synchronous and Asynchronous Scripts
  33. 33. Calling #BS on 35kb Code Size
  34. 34. Comparing Sync and Async So ehh so how much load time does a 35kb script take in seconds? Does it matter how big the initial script is? www.convert.com Client Small Script http://cdn.fast.com CDN.Fast.com 35kb jQuery X 280kb Geo-Check 120ms New Image Variation 400kb 715kb + 120ms
  35. 35. Comparing Sync and Async Now what is it really? www.convert.com 100ms 200ms 400ms In a Sync-World Convert Experiments 100ms 200ms In a Async-World Convert Experiments 400ms Total Time = ~700ms Total Time = ~450ms
  36. 36. Convert Experiments Script What do we really do and when? www.convert.com 100ms 100ms 400ms Sync and Async = + 100ms extra 200ms 200ms 100ms Async Goals & Cookie SettingSmartInsert Sync
  37. 37. Competitor #2 Async Script Others might do it different and why that matters www.convert.com 100ms 100ms 400ms Sync and Async 200ms 200ms 100ms Async Goals & Cookie Setting Async Script Variations = Blinking* *When changes are loaded in areas A or B A B C
  38. 38. Q&A Dennis van der Heijden Founder & CEO of Convert.com @Convert #ConvertAcademy dennis@convert.com
  39. 39. Convert Experiments Authentic A/B Testing Software http://www.convert.com Brought to you by: