conversion 2017 - guido jansen (euroflorist)
TRANSCRIPT
How Euroflorist is preparing for AI:Changes to our optimization strategies.
euroflorist.com
How can we make the next step?
How can we maximize output(in terms of learnings and conversion uplift)
with fixed resources?
CRO is hard!
Optimizing for“the average user”
Who is (s)he anyway?
Need more resources!(but who doesn’t…?)
Too many variables impacting conversion!● Customer persona’s
● Countries
● Budgets
● Customer history
● Regulations
● Colors
● Flower seasons
● Product types
● Holiday peaks
● Time of day/week
● Purchasing power
● Culture
● Devices
● Social media platforms
● Acquisition channels
● Symbols
● Step in Customer Journey
● Tech adoption level
● Accessoires
● Multi-channel funnels
● Etiquette
● Consumer vs Corporate
● Platform
● Different payment systems
● Brand awareness/perception
● Different meaning of flowers
● Logistic situation
● Aesthetics
● Traditions
● Social context
● Weather
● Competition campaigns
● Consumer state-of-mind
● ...
And to make it even worse...
These variables keep changing
all the ffing time!
Let’s take a step back...
Why does Euroflorist exist?
We help people to make others
happy, noticed or remembered
Who are we?
Who is Euroflorist?
● We sell flowers & others gifts
● We’ve been doing this since 1947
● We do B2C & B2B, online and offline
● We are a Swedish company
● Our Dutch office is located in Amsterdam
Euroflorist in numbers (1/2)
● > 21 years in e-commerce● > 60% revenue from online● > 200 employees: 50% in CS, 35
working in online● > 2M bouquets sent every year● 19 sites in 11 countries and 8 languages
Euroflorist in numbers (2/2)
● 10M+ users on our sites in 2017● Sites with up to 20% conversion rate● Market share ranging from 5-45%● 8+ score on Trustpilot, NPS > 50● Growing revenue, teams and taking
market share
Guido Jansen
● Chief Psychology Officer @ Euroflorist
● UX/CX Team Lead
● Full Customer Journey optimization
[email protected] | @guido
What are we doing now (with CRO)?
Customer journey teams
● Branding/Marketing
● Traffic
● Landing pages
● PDP
● Product
● Checkout
● Post-order experience
● Delivery
● Customer Support
● E-mail / Retention
1 Business owner + 1 UX/CX team member
Live Testing
Complete/Analysis
Finished report
Design
Development
Q&A
Pending approval
Approved
“Act” phase:
Idea
Hypotheses
Full testing plan
How can we make the next step?
How are others doing this?
“[Booking.com’s] utilization of A/B testing … drives higher conversion across it’s entire platform, resulting in conversion levels 2-3x the industry average”
Source: Evercore Equity Research
Sadly, when you test more, the % of tests with an uplift will go down.
When A/B testing a fully optimized site, the % of tests with an uplift is low(er).
Euroflorist: 1 in 5 tests with uplift
Booking.com: 1 in 10 tests with uplift
Booking.com Source: Stuart Frisby, Booking.com - Conversions@Google 2017
How to get more tests with an uplift?
Euroflorist: 10-20 concurrent test
Booking.com: 1000 concurrent tests
Booking.com Source: Stuart Frisby, Booking.com - Conversions@Google 2017
Euroflorist: 1 optimization team
Booking.com: 75 product teams
Booking.com Source: Stuart Frisby, Booking.com - Conversions@Google 2017
Of course, we are not Booking.com...
...but we do know a thing (or 2) about experiments...
...so let’s experiment on the way we experiment!
Let’s start looking for a solution...
Evolutionary AI?Tell me more!
Possible applications
Different applications of AI in E-commerce
● Catalog optimization
● Personalization
● RTB / Advertising
● Chatbots
● Virtual personal shopper
● Improved optimization process
● ...
We tried AI catalog optimization
Previous experience:
● “Black box”
● No (perceived) control for E-commerce managers
● No proven uplift in CR
● Hard to do for (emotional) consumers buying for
someone else?
How could AI help the optimization process?
● Increase conversion?
● Speed up the optimization cycle?
● Give you better insights?
● More tests with same team capacity?
● ...
Sentient
Sentient Ascend
Tooling:
Implementation partner:
$143Min total funding
9 YEARSplatform & technology development
35 PATENTSsubmitted to date; 11 patents issued
100+ EMPLOYEESHQ in San Francisco with offices in San Jose and Hong Kong
Founded by the team that developed the core technology behind Siri
Sentient Ascend
DEEP LEARNING BAYESIAN
EVOLUTIONARY ALGORITHMS NEUROEVOLUTION
Tested in multiple industries
FINANCE HEALTH CARE
INSURANCE
CYBERSECURITY
AGRICULTURE
But does it work for E-commerce…?
The promise...
MORE CAPABLE• Test your entire funnel in a
single experiment
ADAPTIVE• Evolves with your audience• Always-on optimization
GREATER IMPACT• More tests = better results• Focus on marketing, not test
administration
FASTER AND MORE EFFICIENT• Tests dozens of ideas• Test 1,000,000s of
combinations• Get to results faster
TIME-SAVING• Fully automated• Eliminates tedious test
administration• Handles the data science
“Massive MVT + evolution”
Massive MVT
Evolution: A Proven Efficient Design System
Evolutionary Algorithms: AI Based on Natural Selection
FITNESSWhat makes one candidate (website, trading strategy, animal) more viable than another?
COMBINATIONBy combining successful candidates, you can achieve greater fitness, and also prune out bad attributes.
MUTATIONAn element of randomness helps discover new successful combinations and ensures you find the best solution—not just a good one.
How the “evolution” part works...
+ 0 - + - + 0 + +
How the “evolution” part works...
+ 0 - + - + 0 + +
How the “evolution” part works...
How the “evolution” part works...
How the “evolution” part works...
How the “evolution” part works...
Always-on continuous optimization
Multipage Funnel Optimization
Home PDP Cart
Current brands experimenting with Sentient.ai ßeta
Sentient E-commerce ßeta @ Euroflorist
Countries selected for trial
The Netherlands
Norway Sweden
Workshop: Jan 19th
Variants based on researchStill set a strong hypothesis
Garbage in, Garbage out
8 elements on PDP, 2 variants each
● Header - less content
● Image position - left to right
● Add progress bar - feed forward
● Social sentence (under CTA, Hobson’s choice +1)
● Product info - full width / less prominent
● Price partitioning
● USP Bar content - different content
● USP Bar position - to the top
28 = 256 variants
(1st) baseline active: end of Feb
Live: March 20th
Test still ongoing...
Current status NL: 1st gen. | 3,3%./17,2% lift
Current status SE: 2nd gen. | 2,9%./7,0% lift
Current status NO: 2nd gen. | 4,7%./12,9% lift
Final thoughts
Regular A/B/MvT testing
Pro:
● More control
● Stricter hypothesis testing
● ...
Con:
● Slow
● Less variants
● Needs a lot of traffic
● Limited test capacity
● ...
Testing with AI
Pro:
● “Time per tested variant” much shorter
● End up with “uncommon” variation (that somehow convert better)
● Requires less traffic● Higher profit● ...
Con:
● More extensive setup process● Black box (?)● Don’t know exactly how fast
generations will follow...● Garbage in, Garbage out● ...
“I just followed instructions”
Will Sentient deliver on their promise...?
MORE CAPABLE• Test your entire funnel in a
single experiment
ADAPTIVE• Evolves with your audience• Always-on optimization
GREATER IMPACT• More tests = better results• Focus on marketing, not test
administration
FASTER AND MORE EFFICIENT• Tests dozens of ideas• Test 1,000,000s of
combinations• Get to results faster
TIME-SAVING• Fully automated• Eliminates tedious test
administration• Handles the data science
Presentation of results
● Meet Magento 2017 - May 10th, Utrecht
● Growth Marketing Conference - May 12th Amsterdam
● Elite Camp 2017 - Tallinn, Estonia
Or follow me @guido
Will this make the CRO team obsolete?
We’re hiring!
Optimization team:
● Web Analysts● UX designers● Growth hackers● Interns● AI/CKI background● (Cognitive/Behavioral) Psychologists
Other online:
● SEA Specialist● Marketing & communications
specialist● CRM specialist● Social Media Manager● Interns● ...
Are you smart & curious? Looking for a growing international company where you can experiment & learn a lot? Join Euroflorist!
euroflorist.com
