ux strat usa, emily leahy, "measuring return on experience (rox) for ux strategy"
TRANSCRIPT
![Page 1: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/1.jpg)
InterContinental Hotels Group
Measuring Digital Return on ExperienceWhat We’ve Learned On Our Journey So Far
Emily LeahyManager, User Experience Strategy
![Page 2: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/2.jpg)
2
Problem
Large companies often make huge investments in large-scale redesigns where we change everything, making it hard to tell what worked, what didn’t, and we miss the opportunity to learn.
Time
Succ
ess
Redesign 1
Redesign for Responsive, Unique Brand Experiences
2012
2015
![Page 3: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/3.jpg)
“…the only poster where the family was shown safely outside.”
Smoke Detectors Save Lives
Always a Strategist
![Page 4: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/4.jpg)
4
What the VP thinks I do What my boss thinks I do What the rest of the UX team thinks I do
What I think I do What I actually doWhat Product Managers think I do
Emily Leahy
Manager, UX Strategy
![Page 5: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/5.jpg)
5
• Formally founded in 2003• Globally Based in Denham, UK• World’s largest hotel company (by
rooms)• 12 brands in over 100 countries
with over 4,942 hotels and more than 474,290 rooms
• 13 major dot coms• $1,858m total revenue
About IHG
UX Strategy Delegation
![Page 6: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/6.jpg)
6
And how do we do that at a company like IHG?
Time
Succ
ess
20122015
Redesign 1
Redesign for Responsive, UBE
Design updates in a year
KPI
Lift Home
Search Results Hotel
Details
Rooms & Rates
Payment
Problem
How do we get from here to there?
![Page 7: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/7.jpg)
7
Problem
One small exampleBefore
After
ResultPoints and Cash $
![Page 8: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/8.jpg)
8
Problem
One small exampleBefore
After
Result
Quick, change it back!
Points and Cash $
![Page 9: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/9.jpg)
9
Quiz Time!
![Page 10: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/10.jpg)
10
A.Change it back and hope that solves the problem without any unintended consequences.
Let’s try hope… again.
Quiz Time!
![Page 11: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/11.jpg)
11
B.Dig our heels in and point back to all the usability testing and analytics that supported the research-based design decision.
Hey, we used, “The Force!”
Quiz Time!
![Page 12: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/12.jpg)
12
C.Test the hypothesis, “Removing the rate preference dropdown decreased Points and Cash revenue,” before proceeding.
Search your feelings, you know it to be true
Quiz Time!
![Page 13: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/13.jpg)
The Digital Return on Experience (ROX) program supports IHG’s Web Channel performance objectives by routinely conducting the practices required to design, implement, and manage digital customer experiences in a disciplined way.
![Page 14: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/14.jpg)
Performance Practices
People & Processes
Performance
What we do with itDigital ROX
Program structure
![Page 15: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/15.jpg)
15
Tools help us focus on IHG’s business metrics and that “automate the mundane” and help us spot opportunities faster
Measurement
![Page 16: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/16.jpg)
Performance
What matters to the business?
Web Channel Metrics
• Web Channel Revenue
• Loyalty Contribution
• Ancillary Revenue
• Brand Perception
• Digital Self Service
And of course, digital guest experience…
![Page 17: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/17.jpg)
Performance
Our measurement universe
Scorecards Ad Hoc Analytics VoC Reports
+
Web Channel Checkout Revenue Revenue Contribution Average Daily RateRevenue Per Available RoomBooking Conversion Rate/FunnelWeb Rewards Club Enrollments Credit Card Sign UpsBrand Perception GapSocial Brand/Web SentimentVisits vs. 2+ Page Visits Property page viewsCustomer escalations
+ =
Digital ROX KPIs, by Brand & Region
Global , Multi- Brand Data
![Page 18: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/18.jpg)
Demand'KPI' Prior'Period'Performance' Current'Month' Trend' Score'Calcula8on' Score' Dimensional'Score' Health'Score'
A" Homepage"to"Room"Rate"Conversion" 4%" 5%" 6%" (5%74%)/(6%74%)*"100" 50"
Demand"Performance"Score"*"Dimension"Weight""78.5%x25%"
Demand"Dimensional"Score""+""Conversion"Dimensional"Score""+""SaLsfacLon"Dimensional"Score""+""Revenue"Dimensional"Score""24.11%"+"18.75%"+"19.58%"+"8.9%"
B" Homepage"to"Rewards"Club"Enroll"Conversion" 4%" 7%" 6%" (7%74%)/(6%74%)*"100" 150"
C" Search"Results"to"Room"Rate"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100"
D" Search"Results"to"Rewards"Club"Conversion" 4%" 5%" 6%" (5%74%/6%74%)"*"100" 50"
E" Hotel"Details"to"Room"Rate"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100"
F" Hotel"Details"to"Rewards"Club"Conversion" 4%*" 5%*" 4%*" (5%/4%)"*"100" 125"
G" Site"Exits"to"Chase"Enrollment"over"Site"Visits" 4%**" 4%**" 4%**" (1/1)*"100" 100"
Demand'Performance'Score' A+B+C+D+E+F+G/(#"KPIs"*"100)" 96.43%' 24.11%'
Conversion'KPI''
H" Room"Rate"to"Booking"ConfirmaLon"Conversion" 4%" 5%" 6%" (5%74%/6%74%)"*"100" 50" Conversion"Performance"Score""*"Dimension"Weight"I" Rewards"Club"Submission"Conversion" 4%" 6%" 6%" (6%74%/6%74%)"*"100" 100"
Conversion'Performance'Score' H+I/(#"KPIs"*"100)" 75%' 18.75%'
Sa8sfac8on'KPI''
J" Rate"Experience" 7" 8" 9" (877/977)"*"100" 50"
SaLsfacLon"Performance"Score"*"Dimension"Weight"
K" Purpose"of"Visit"CompleLon" 8" 10" 10" (1077/1077)"*100" 100"
L" Discovery" 6" 5" 6" (576/676)"*"100" 750"
M Relevancy" 7" 7.7" 7.5" (7.777/7.577)"*"100" 140"
N" Convenience" 6" 7.3" 7" (7.376/776)"*"100" 130"
O" Bobom"Line" 8" 8" 8.5" (878/8.578)"*"100" 0"
Sa8sfac8on'Performance'Score' (J+K+L+M+N+O)/(#"KPIs"*"100)" 78.33%' 19.58%'
Revenue'KPI''
P" Web"Channel"Checkout"Revenue" $75M" $80M" $90M" ($80M7$75M/$90M7$75M)"*"100" 33"Revenue"Performance"Score"*"Dimension"Weight"
Q" Number"of"Bookings" 120K" 100K" 150K" (100k7120K/150K7120K)"*"100" 767"
R" Revenue"per"Booking" $280" $520" $450" (5207280/4507280)"*"100" 141"
Revenue'Performance'Score' (P+Q+R)/(#"KPIs"*"100)" 35.6%' 8.9%' 71.34%'
Performance
The fancy math
![Page 19: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/19.jpg)
19
Performance
Where we’re going
![Page 20: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/20.jpg)
20
Performance
What we do with it
StrategyHypotheses Optimization Plans
Stakeholder Input Competitor Analysis Industry Trends
User Session Replays
3rd Party Research
Call Center Data
Vendor Vision / Recommendations
Usability Testing Findings
Site Analytics
Online Chat Log A/B Testing Small Enhancements
Winners
Okay, let’s get to work…
![Page 21: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/21.jpg)
21
Performance
What we do with it
Where should we start?
StrategyHypotheses Optimization Plans
Stakeholder Input Competitor Analysis Industry Trends
User Session Replays
3rd Party Research
Call Center Data
Vendor Vision / Recommendations
ROX Scorecard Metrics Usability Testing
Findings
Site Analytics
Online Chat Log A/B Testing Small Enhancements
Winners
![Page 22: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/22.jpg)
22
Performance
What this helped us do• Optimization plan for membership enrollment form
• Optimization plan for login
• A/B tests for booking conversion
• Strategy for key ancillary revenue projects
• Strategy for IHG.com
• And the list grows daily
![Page 23: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/23.jpg)
23
• Optimization plan for membership enrollment form
• Optimization plan for login
• A/B tests for booking conversion
• Strategy for key ancillary revenue projects
• Strategy for IHG.com
• And the list grows daily
Performance
What this helped us doIncreases Loyalty Contribution
Enables Digital Self Service
Increases Channel Revenue
Grows Ancillary Revenue
Improves Brand Perception
Builds a Sustainable Program
… But you said “routine” and “disciplined”
![Page 24: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/24.jpg)
24
Routines that keep us focused on our vision and achieving results that build buy-in.Practices
![Page 25: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/25.jpg)
Practices
How do we get it all done?
5 weeks of website analysis of booking funnel lead to:• 4 Small Enhancements• 8 Online Test Ideas evolve to enhancements• 5+% Lift in funnel entry • 8-15% lift in each step of funnel• 24% increase in funnel conversion• Leads to $8.6MM incremental/month once deployed to all
traffic• +114,000 new bookings and +$51.6M Incremental
Revenue (6 mths after implementation)
12 Quick Wins*
Page Quick Win Approach Priority
All New CTA Button Verbiage, Color and Visual Treatment Test First 1
All Image/Content Refresh focus on benefits and brand Test First 1
Search Results Explain sister hotels in search results Test First 1
Room Rate More visibility of refining search options Test First 1
Room Rate Sense of Urgency Messaging Test First 1
Room Rate Minimize confusion and create differentiation of Best Flexible Rate/Deals by moving to bottom of page Test First 1
Guest Info Move Credit Card offer to bottom of page Test First 1
Guest Info Remove Learn More link Implement 1
All Targeted content for returning visitors Implement 2
Room Rate Default to Currency Selector to current country currency Implement 2
Guest Info Make Hotel Stay Summary easier to find and review Test First 2
Guest Info Edit Hotel Stay Summary On Page Implement 3
*Quick'wins'apply'to'the'current'state'of'the'desktop'website'for'Crowne'Plaza'
![Page 26: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/26.jpg)
Practices
What to do when - high level
Heuristic Assessments & Design Exploration/Concepts
The$quarterly$and$monthly$feedback$loop$will$7e$into$an$orchestra7on$workflow$and$set$cadence$of$research,$analysis,$planning$and$execu7on$processes$
Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul
Digital Health Scorecard
!!Staybridge,!Candlewood,!IC,!CP! !!Holiday!Inn!Family!Brands! !!IHG,!Indigo,!Hualuxe,!!!!Kimpton,!EVEN!Brand Site Experience Research
1:1 Usability Tests (Deep Dive)
Small Enhancements
Small Enhancements (AEM)
Online Testing
Quarterly!Brand!Summit!
!!Staybridge,!Candlewood,!IC,!CP! !!Holiday!Inn!Family!Brands!
Monthly!Performance!Review!Quick Win Analysis
& Design Explorations
![Page 27: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/27.jpg)
Practices
What to do when - details26#Usability#Studies#Per#Year#• Remote&Global&Studies&with&Deep&Dive&1:1&Modera9on&&5#Brand#Summit’s#Per#Year#• Summit&1:&Candlewood,&Staybridge,&IC,&Crowne&Plaza&• Summit&2:&Holiday&Inn&Family&Brands&• Summit&3:&IHG,&Hualuxe,&Indigo,&Even&• Summit&4:&Candlewood,&Staybridge,&IC,&Crowne&Plaza&• Summit&5:&Holiday&Inn&Family&Brands&&12#Performance#Reviews#Per#Year#• 2&(4&to&5&Week)&Quick&Wins&Analysis&Per&Brand&
45#@#57#Online#Tests#Per#Year##• (3&Test&Running&Concurrently&(6&–&9&Week&Test&Cycles)&&24#Small#Enhancements#Releases#Per#Year#• 24&(2&Week&Sprints)&–&383&Story&Points&(3&Developers)&• 12&(4&Week&Sprint)&–&420&Story&Points&(3&Developers)&&24#AEM#Small#Enhancements#Release#Per#Year#• 24&(2&Week&Sprints)&–&256&Story&Points&(2&Developers)&• 12&(4&Week&Sprint)&–&280&Story&Points&(2&Developers)&&
A#set#orchestraHon#calendar#that#extends#12#months#out#will#drive#increase#producHvity#
![Page 28: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/28.jpg)
28
Increase bandwidth in key areas including design, research, and project management while promoting changes that make us more efficient and effective.
People & Processes
![Page 29: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/29.jpg)
People & Processes
Where we want to be
DXD#Director#
Performance*Strategy* Performance*Design*Studio*
Op,miza,on#Lead#(Mgr)#
Op,miza,on#Analyst# Interac,ve##Project#Manager#(Mgr)##Studio#Manager#(Mgr)# +2#
New#
Small*Enhancements*&*Online*Tes:ng* Brand*Campaigns* GSM*&*Brand*Priori:es*
#####Hybrid#x3# #####Copywriter#x2#
#####Designer#x5#
####UEA#+#Hybrid#x6#
#####Designer/AD#x7#
#####Content#########Str/Sp#x5#
#####Copywriter#x1#
DXD*PEOPLE*COUNT*CHANGE*38****!****44*
CX#Strategy#Lead#(Mgr)#
Experience#Researcher#
+#Research#Vendor#
+2#New#
#####Hybrid#x2# +2#New#
SLT#Sponsor#DXD*
########Experience#Lead#(Mgr)#–#Ryan,#John,#Mike,#Kim#
Ideal UX Team
![Page 30: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/30.jpg)
30
People & Processes
Many hands make light workArea Allies
Measurement Analytics Team, Brand Teams, Product Managers, Testing & Optimization
Quarterly Brand Summits Product Management, Testing & Optimization, Mobile, Ent.Content Strategy
Monthly Performance Reviews Product Management, Testing & Optimization, Small Enhancements
![Page 31: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/31.jpg)
31
People & Processes
From our perspective
Design Documentation Ticketing Post-Launch Ticketing
Design Direction Design Prototype
Ticketing
Post-Launch Optimization
Launch
Core Documentation
Design Support
Currently
Proposed
High-level design process
![Page 32: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/32.jpg)
32
People & Process
From the organization’s perspective
Idea% Project%Charter% Crea.ve%Brief%Crea.ve%Concepts%
AEM%Specs%%&%AX%Wires%
Wireframes%&%Annota.ons%
Visual%Design%
Solu.on%Needs%Document%
Sprint%0%%…%
Defini.on% Solu.on%Design%
Implementa.on%
DXD% DXD% DXD%
C.Ops%
W%D%
GT%
DXD%P%M%Biz%Stakeholder%
Prototype%Tes.ng%
DXD%
Sprint%1% Sprint%X%
![Page 33: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/33.jpg)
Automate the mundane
Results Orientation
Build the ROX “Brand”
Performance Practices
People & Processes
![Page 34: UX STRAT USA, Emily Leahy, "Measuring Return on Experience (RoX) for UX Strategy"](https://reader030.vdocuments.site/reader030/viewer/2022021423/588746941a28ab5a628b590b/html5/thumbnails/34.jpg)
34
What we’ve learned
A journey we are on1. You’ll always be a rebel, but the old
ways exist for a reason
2. Be clear and repeat often why you’re asking for change (sometimes you’re even reminding yourself)
3. Use challenging moments as a way to show why a different approach is necessary
4. Establish routines as early as possible
5. Find allies within your organization