a/b testing for ecommerce: features of convert experiments

A/B Testing for Ecommerce

Features of Convert Experiments

Brought to you by:

Dennis van der HeijdenFounder & CEO of Convert.com

@Convert #ConvertAcademy

Prepare as Agency for Ecommerce Clients

We like to help you prepare for new ecommerce clients coming to your agency. We want you to be awesome and

always make you look great in front of your clients. Prepare well … this webinar helps..

Purpose of Webinar

Convert.com Agency and Ecommerce

Focus

How A/B Testing Tools WorkMost of what I share you can use with:● Convert● Optimizely● VWO● Monetate● Adobe● … and other Javascript web-based tools.

Disclaimer: I’m super happy you are testing with Convert or our colleagues, that means you are in the top 5% of world's marketers. Now it’s hard to not mention differences between the tools… forgive me, I just think Convert Experiments is a better fit for many CRO agencies.

www.convert.com

Packed Presentation, Let’s Get StartedMyths and Half-Truths About Async/Sync, Tag Managers and Speed

1. Install all codes at once:o Tracking code & Locations

URL and headero Revenue trackero Mapping of CMSo Goal / Event JSo Security documentationo AJAX refresh codeso Information on Tag

Managers

2. Precheck for Advanced Installso Non structured URLso AJAX load of content and

goalso Testing behind logino Tag Manager Fanso Subdomains and cross

domainso Offsite revenue tracking

3. Inner workings of A/B testing toolso Async and sync codeso Avoid seeing the blink, what

hack do A/B testing tools use?

www.convert.com

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Tracking Code & Locations URL

Send the tracking code+ Checklist of URL’s:

1. Homepage2. Category Page3. Product Page4. Cart Page5. Checkout6. Revenue / Thank You Page

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Yes, In Header Please </title>-tag

More in this later, but specifyin email to client…

Check each page for presenceof snippet:● http://robhammond.co/tools/seo-crawler ● http://www.screamingfrog.co.uk/

Script in Header

Being the first to load allows A/B testing tool to manipulate the content following and make the changes without blinking.

It is blocking and yes it is hurting site-speed, later on why that hardly matters...

www.convert.com

Convert Tracks GA Automatically● GA Universal● GA Classic

Manual Tracking Script● Default One Time Revenue Event

http://j.mp/1FcldYq ● Optional Multiple Revenue Events

Ask support@convert.com

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Revenue Tracker

● Basic Script● Advanced Script: Mapping

○ Page Type○ Category ID○ Category Name○ Product SKU○ Product Name○ Product Price○ Customer ID○ Custom Field 1 (text)○ Custom Field 2 (text)○ Custom Field 3 (number)○ Custom Field 4 (number)

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Mapping of CMS http://j.mp/1JJgmWt

Segmentation or Goal Events:

● Pop-ups / Hovers● Ajax Goals● Multi-step Checkout

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Goal / Event JS

Ask Support for latest version:

Technical Documentation System Implementation Guide(PDF)

Tackle every IT department question on Day 1

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Security documentation

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

AJAX refresh codes

1. Experiment DoneWhen Page Loads

2. Ajax Loads After3. Trigger Experiment Search

& Replace Again http://j.mp/1FNrds2

Get one big Pack in the Hands of ClientsNothing is more frustrating for a client getting 20 separate emails with install details!

www.convert.com

Information on Tag Managers

● Tealium Awesome● Adobe Great● Rest… not recommended

http://j.mp/1C25Amh

Loading Async or on DOM-readyTHAT'S TOO LATE -> BLINKING

Precheck for Advanced Installs

Don’t Blame the Tool…

… Prepare Well

Precheck the Client Website for A/B TestingWarning signs you should monitor… before quotation …

The following site signals will make the implementation harder:● Non structured URLs● AJAX load of content and goals● Testing behind login● Tag Manager Fans● Subdomains and cross domains● Offsite revenue tracking

www.convert.com

Non Structured URLsSEO friendly URL’s a good idea. But hard for some tools

www.convert.com

Structured Ecommerce URL’s● shop.com/product/jeans-1234.php● shop.com?prod=1234&name=jeans

Unstructured Ecommerce URL’s● shop.com/jeans-blue-awesome● shop.com/special-jeans-june

AJAX Load of Content and GoalsTest or Monitor Ajax, Hovers or Pop-ups? Testing Tools Prefer

Refresh

www.convert.com

Subdomains and Cross DomainsConvert Experiments Auto-Reconnect - Add to Settings and Done

www.convert.com

Agency Benefit

Testing Behind Login, No ProblemHave Two Tabs/Sessions Open + Script Installed?

Convert Experiments has Cross Session Setup

www.convert.com

Agency Benefit

Tag Manager Fans?Tealium & Adobe :-) - Others :-( … So -> Load Direct From

<Header/>

www.convert.com

Offsite Goal & Revenue TrackingPass Experiment ID and Variation ID to Backend and Send Webhook

www.convert.com

Agency Benefit

The Inner Workings of A/B Testing Tools

psst, please don’t tell...

“The Dirty Secrets and Misdirections”

Understanding the terminology is super important when selecting a new A/B testing tool. There is a reason smart agencies go deep on sync/async parts of the script and how tools hide their true technology from you?

Let's talk about:● Do you really understand sync and async?● Rethink your current A/B testing tools impact...● How do we trick your eyes and make SmartInsert© possible?

Don’t Copy Low Quality A/B Testing ArticlesHow do we really affect site speed and why to some tools “blink”?

www.convert.com

Understand How Page LoadsThanx New Relic, for image

www.convert.com

A/B Testing Tools TechnologyHow does Convert Patent Pending SmartInsert© Work?

www.convert.com

● Click to add text.○ Click to add text.

■ Click to add text.● Click to add text.

● 0.30 Sec.Nothing...

● 0.35 Sec.Convert Starts

○ Hide Body for 1 Sec.

○ Check Every 0.05 Sec.

○ Unhide Body

● 1.50 Sec.Convert Long Done

● 0.30 Sec.DOMloading

● 0.35 Sec.DOMcontent Loaded Start

● 1.50 Sec.DOMcontent Loaded End

● 1.80 Sec.DOMcomplete

Fast Website + Convert Experiments

A/B Testing Tools TechnologyHow other A/B testing Tools Work… example #1

www.convert.com

● Click to add text.○ Click to add text.

■ Click to add text.● Click to add text.

● 0.30 Sec.Nothing…

● 0.35 Sec.Tool Starts

○ Makes All Changes

○ Visible Changes Swapping / Blinking

● 1.50 Sec.Tool #1 Done

● 0.30 Sec.DOMloading

● 0.35 Sec.DOMcontent Loaded Start

● 1.50 Sec.DOMcontent Loaded End

● 1.80 Sec.DOMcomplete

Fast Website + Other A/B Testing Tool #1

A/B Testing Tools TechnologyHow other A/B testing Tools Work… example #2

www.convert.com

● Click to add text.○ Click to add text.

■ Click to add text.● Click to add text.

● 0.30 Sec.Nothing...

● 0.35 Sec.Nothing…

● 1.50 Sec.A/B Testing Tool Loads○ Hide Body for 1 Sec.○ Make All Changes.

● 1.80 Sec.Body Still Hidden

● 2.50 Sec. + 0.7 Sec. WhiteBody Unhide

● 0.30 Sec.DOMloading

● 0.35 Sec.DOMcontent Loaded Start

● 1.50 Sec.DOMcontent Loaded End

● 1.80 Sec.DOMcomplete

Fast Website + Other A/B Testing Tool #2

Synchronous and Asynchronous Scripts

Calling #BS on 35kb Code Size

Comparing Sync and AsyncSo ehh … so how much load time does a 35kb script take in seconds?

Does it matter how big the initial script is?

www.convert.com

Client Small Script http://cdn.fast.com

CDN.Fast.com35kb

jQuery X280kb

Geo-Check120ms

New Image Variation400kb

715kb+

120ms

Comparing Sync and AsyncNow what is it really?

www.convert.com

100ms 200ms 400ms

In a Sync-World ≠ Convert Experiments

100ms

200ms

In a Async-World ≠ Convert Experiments

400ms

Total Time = ~700ms

Total Time = ~450ms

Convert Experiments ScriptWhat do we really do and when?

www.convert.com

100ms 100ms 400ms

Sync and Async = + 100ms extra

200ms

200ms

100ms

Async Goals & Cookie SettingSmartInsert© Sync

Competitor #2 Async ScriptOthers might do it different… and why that matters

www.convert.com

100ms

100ms

400ms

Sync and Async

200ms

200ms 100ms

Async Goals & Cookie Setting

Async Script Variations = Blinking*

*When changes are loaded in areas A or B

A

B

C

Q&ADennis van der HeijdenFounder & CEO of Convert.com

@Convert #ConvertAcademydennis@convert.com

Convert ExperimentsAuthentic A/B Testing Software

http://www.convert.com

Brought to you by: