website testing & optimization framework & results

Sumo Logic Framework & Journey of Website Testing/Optimization

Saad Hameed & Jithesh AmbalapadMarketing, Operations, Analytics & technology team

LinkedIn: http://j.mp/SaadHameedTwitter: @SaadVirtus https://twitter.com/SaadVirtus

LinkedIn: https://www.linkedin.com/in/apjithesh

○Improve Navigation (IA)

○Align Messaging

○Improve Bounce Rate

Website Journey Phase 1: Create Baseline of Web Visitor Engagement

○ Build taxonomy & foundational tools to measure & optimize

• Google Tag Manager

• Google Analytics

• User Testing

• Optimizely

• CrazyEgg

○ Website Redesign & Continuous Improvement

• Major revamp of our website UI/UX

• Updated Content/Messaging

The How of Improving Baseline Engagement for Website

Sumologic.com Site Pre-Phase 1 (Aug 2015)

https://web.archive.org/web/20160405124254/https://www.sumologic.com/

Sumologic.com Post-Phase 1 Refreshes:

May 2016

https://web.archive.org/web/20150814044927/https://www.sumologic.com/

• Results

○ Continued refreshes improved Bounce Rates from 65% to 40%

○ Improved free conversions + 10%

○ Improved Alexa Ranking (15% Improvement)

Nov 2015 May 2016

Results From The Phase 1 Refresh

○Ongoing major initiative to optimize for engagement and conversion

○Started with detailing a testing framework

○Framework and testing is geared towards

• Improving Conversion

• Improving Engagement

Journey Phase 2: Web Optimization to Improve Conversions

Improve Conversions & Engagement: • Platinum (Free/Trial, Contact Us, Click to chat)• Gold (Platinum + Gated content resources)• Silver (Gold + Non-Gated content resources)

Define the different elements that we would need to test and have a categorization (A/B Testing taxonomy)

Detailing out test stories, execute the tests, measure, & communicate

Why?Why to test

What?What do we need

to test

How?Process to test

The Why, What, and How of Web Optimization & Testing

Information Architecture Content

● Components○ Main Navigation○ Utility Navigation○ Footer

● Attributes○ Size○ Position○ Verbiage

Images Content Modules Call to ActionsOffers

● Components○ Spotlight○ Logos○ Icons○ Product

● Attributes○ Size○ Position○ Image types

(Human /Product)

● Components○ Company Pitch ○ Product Pitch○ Product Validators○ Product Demos ○ Product Benefit ○ Customer Testimonial○ Customer Logos ○ Pricing

● Attributes○ Tone○ Voice○ Size○ Position

● Components○ Gated○ Non-Gated

● Attributes○ Title○ Type (Blog, Podcast,

video, PDF, etc.)○ Perceived Value of

offer ○ Relevance to phase

in buyer lifecycle

● Components○ Free CTA○ Video CTA○ Download

● Attributes○ Size○ Position ○ Action/Verbiage○ Color ○ Font○ Image

Web Optimization & Testing Framework

Information Architecture

● Components○ Main Navigation○ Utility Navigation○ Footer

● Attributes○ Size○ Position○ Verbiage

Size Position VerbiageMain Nav Large or Small Change Position Nav Item VerbiageUtility Nav Large or Small Top or Next to Main Item VerbiageFooter Fat footer or Thin Footer Left or Center Item Verbiage

How to Leverage the Framework? Example:

• Definition and detailing of “What” becomes the essence of testing ideas

• Map the ideas based on Impact to business & Ease of implementation

• Prioritize and execute ideas • Document & take action on Results

Creating a Funnel for Web Optimization & Testing Experiments

Ease of Implementation

Bus

ines

s Im

pact

● New Offers● Nav position● Related content

position

● Offers A/B testing with Audience data

● Contact us position

● Footer Link sizes● Media download

positioning

● Positioning of Share button

● Link out positioning for external references

● Position of Mobile nav

● Enable/Disable link to product demo

● Elevator pitch message

● Hide/enable related links

● Multivariate Testing with customer quotes

● Spotlight size

● Demo Multi page● CTA positioning● Nav element

positioning

● Free CTA● Nav text verbiage● Login position

High

Medium

Low

Low (Hard)

Medium High (easy)

THIS IS A SAMPLE

NOT ACTUAL IDEAS

How to Prioritize the Web Optimization & Testing Experiments?

Example of Web Experiment Test Click and Conversion rate when button verbiage is changed

Example of Web Experiment

Web Experiment Results

The How (Process) of Web Optimization & Testing