which would you choose?

Which would you choose?Many on-line sites offer customers an option to review a product or service to give other potential customers an idea of what to expect.Most use a star rating system as a simple way of giving an overview together with an optional comment.5 stars usually represents the best and 1 star represents the worst.What sites and products have you seen which use this system for feedback?

Which would you choose?The feedback others leave can help when making a decision about buying an item or booking a hotel or planning a visit to an attraction.

On the next slide there are brief details about two similar hotels. Which would you choose?

Which would you choose?Hotel’s own website details

Surf and Sand Sandy ShoesSandy Shoes is a beach front hotel offering a warm welcome to guests.Rooms are comfortable and breakfasts are plentiful.

Official 3 rosette rating.

User reviewsBased on 12 reviews

Surf and Sand is a renowned 3 rosette hotel with a top reputation for hospitality. Guests will find everything they need in their rooms, many of them have stunning sea views. Breakfast is served in the sunny ‘Wave Watcher’ room.User reviews Based on 12 reviews

Which would you choose?It doesn’t seem as if there’s much to choose between the two.

Have a closer look at the summary of the reviews on the next slide and then decide.

The numbers indicate how many reviewers have given each rating.

Which would you choose?Customer Reviews

5

3

1

2

1

Surf and Sand Sandy Shoes

4

6

2

0

0

Which would you choose?Does the more detailed information help?

Both hotels seem to have a customer rating of 4.

Do you think that is correct?

What summary rating would you give each hotel?

How do you think the customer rating is calculated?

How the star rating is calculatedOn many sites, the star rating shown is simply the mean of all reviews posted; the summary rating is often visually rounded to the nearest half star.Using this method, should the hotels both have 4 stars?Stars Number of ratings

5 5

4 3

3 1

2 2

1 1

Stars Number of ratings5 4

4 6

3 2

2 0

1 0

How the star rating is calculatedSurf and Sand

Stars Number of ratings

Calculation

5 5

4 3

3 1

2 2

1 1

Totals



Calculation

5 5

4 3

3 1

2 2

1 1

Totals 12



Calculation

5 5 5 x 5 = 25

4 3

3 1

2 2

1 1

Totals 12

What does the calculation

represent?



Calculation

5 5 5 x 5 = 25

4 3 4 x 3 = 12

3 1 3 x 1 = 3

2 2

1 1

Totals 12



Calculation

5 5 5 x 5 = 25

4 3 4 x 3 = 12

3 1 3 x 1 = 3

2 2 2 x 2 = 4

1 1 1 x 1 = 1

Totals 12



Calculation

5 5 5 x 5 = 25

4 3 4 x 3 = 12

3 1 3 x 1 = 3

2 2 2 x 2 = 4

1 1 1 x 1 = 1

Totals 12 45



Calculation

5 5 5 x 5 = 25

4 3 4 x 3 = 12

3 1 3 x 1 = 3

2 2 2 x 2 = 4

1 1 1 x 1 = 1

Totals 12 45

45÷ 12 = 3.75

How the star rating is calculatedSandy Shoes


Calculation

5 4 ?

4 7 ?

3 1 ?

2 0 ?

1 0 ?

Totals 12 ?

?? ÷ ?? = ??

How the star rating is calculatedSandy Shoes


Calculation

5 4 5 x 4 = 20

4 7 4 x 6 = 24

3 1 3 x 2 = 6

2 0 2 x 0 = 0

1 0 1 x 0 = 0

Totals 12 50

50 ÷ 12 = 4.17

How the star rating is calculatedThe Lighthouse

A third hotel in the area also has an average customer rating of 4.What might the breakdown of ratings be?Can you find 3 alternatives?


5 ?

4 ?

3 ?

2 ?

1 ?

Totals 12

How much difference does each review make?

Details of another two hotels are given on the following slide.

Which one would you choose and why?

Which would you choose?Hotel’s own website details

Mountain Vista Lakeside House Lakeside house provides the comfort of your own home in a five star setting. The spa is available for all guests to use throughout your stay. Our team of world class chefs will tantalise your taste buds.User reviewsBased on 6 reviews

Mountain Vista provides guests with a relaxing retreat set amongst the lakes and forests in the foothills of the mountains. Experience our superb cuisine or relax in our luxurious spa facilities.User reviews Based on 120 reviews


The breakdown of reviews is given on the next slide.

Do these present a different picture?

Would you change your decision or not?

Why?

Which would you choose?Customer Reviews

61

18

3

27

11

Mountain Vista Lakeside House

2

2

1

0

1


If each hotel were to receive one more 5 review (as shown below) what happens to their average ratings? Mountain Vista Lakeside House

Stars Number of ratings5 624 183 32 271 11

Stars Number of ratings5 34 23 12 01 1


Sometimes hotels and business go through a ‘bad patch’ when they might have issues with staff or the building etc. Reviews at these times can be lower than normal.

When the hotel returns to normal, it can be difficult to counteract the bad reviews with newer good reviews.

How much difference does each review make

61

18

3

27

11

Mountain VistaHow many more 5 reviews would it take for the average customer rating to reach the next level of 4.5?

Is this the best way?Many websites do use this approach to calculate average ratings and then present the items in the order of their average rating.Consider this extreme example for two similar televisions:

Which one has thehighest average rating?

Which would you be more likely to buy?


5 994 1321


5 14321

What would your advice be?When looking at ratings on review sites, what should people be looking at and why?

What’s important?

Teacher notes: Star RatingThis activity is designed to encourage students to look more closely at common statistics they are presented with in everyday life.

Students are required to calculate the mean of data presented in a frequency table and to consider the impact of a single value within large and small data sets.

The realistic context makes the activity accessible to a wide range of students.

» Students should have the opportunity to discuss this with a partner or in a small group

» Students should calculate

Teacher notes: Star RatingMany sites use customer feedback to present a rating for a product or service.

These ratings are subject to many of the issues that arise when using the mean in statistics, namely:• The same mean value can be obtained through a set of data with

little variability and a set of data with wide variability.• One value in a small data set has a bigger impact on the mean than

a similar value in a larger data set.

Teacher notes: Which would you choose?In this part of the activity, two hotels with seemingly similar profiles and

feedback are presented.The detailed breakdown of review grades draws attention to the fact that the same rating can be obtained in different ways.

Slide 8: checking the rating. Students may already have met this content or may be able to work out for themselves how to find the mean of data in a frequency table. If not, slides 9 to 17 give an overview.

Teacher notes: How the star rating is calculated

Slide 18: The Lighthouse.To obtain a rating of 4, the value of the average rating must be:3.75 ≤ rating <4.25.

This means that for 12 reviewers, the sum of their ratings should be: 45 ≤ sum < 51

There are many possibilities, including:

54 12321

5 14 103 121

5 44 43 421

5 94321 3

This one has the maximum number of 5s and minimum number of 1s

Teacher notes: How much difference does one review make?

Slides 19 - 22In the next part of the activity, two hotels with slightly different feedback are presented, but the averages are obtained from 120 reviews and 6 reviews respectively.Discussion could be about the relative ‘reliability’ of the values and about whether using a percentage rather than a number for each value in the breakdown would be helpful or not. Mountain Vista Lakeside House


% of reviews

5 61 51

4 18 15

3 3 2.5

2 27 22.5

1 11 9


% of reviews

5 2 33

4 2 33

3 1 17

2 0

1 1 17


Slide 23: what difference does one 5 rating make? Mountain Vista Lakeside House

Average rating = 456÷121 = 3.77 Average rating = 27÷7 = 3.86 (still rounds to 4) (now also rounds to 4)This should help to emphasise that adding the same value to both has a bigger impact in a smaller data set.

Star Freq Calculation5 62 5 x 62 = 310

4 18 4 x 18 = 72

3 3 3 x 3 = 9

2 27 2 x 27 = 54

1 11 1 x 11 = 11

Total 121 456

Star Freq Calculation5 3 5 x 3 = 15

4 2 4 x 2 = 8

3 1 3 x 1 = 3

2 0 2 x 0 = 0

1 1 1 x 1 = 1

Total 7 27


Slide 24 -25 How many more 5 reviews would it take for the average customer rating to reach the next level of 4.5?

This part of the activity demonstrates how much positive feedback is needed to balance out negative feedback.

This is much more challenging to consider as it requires students to set up and solve an algebraic equation, but it does demonstrate that they have fully understood work on averages from frequency tables.

If this content is beyond students, a simple trial and error approach to find an approximate (or exact) number, by adding 10 or 20 extra 5 reviews each time, will work.

Whichever approach is taken, students are likely to find the number required surprising.


Slide 24 How many more 5 reviews would it take for the average customer rating to reach the next level of 4.5?

Average = sum of reviews ÷ number of reviews

Average needs to be ≥4.25 in order to round to 4.5

4.25 = (451+5n) ÷ (120+n)4.25 x (120+n) = (451+5n) 510+4.25n = 451+5n59 = 0.75nn = 78.66679 more 5reviews are required (with no additional lower reviews).

Rating Freq Calculation5 61 5 x 61 = 3054 18 4 x 18 = 723 3 3 x 3 = 92 27 2 x 27 = 541 11 1 x 11 = 11Totals 120 451

Teacher notes: Is this the best way?Slide 26Two extreme cases are given such that a single 5 rating will be rated more highly than anything with many top ratings and a single other rating.Examples of this can be found on many sites – though usually less extreme cases. Visually a ‘half star’ rating is shown, but the reality is that the ratings are ordered according to their mean values.A real example is Televisions on Amazon - scrolling to the bottom of the first page, a television with a single 5 rating is before one with 681 ratings which average at 4.58.

For interest, teachers might like to look at Evan Miller’s website for an alternative approach.

http://www.amazon.co.uk/s/ref=sr_nr_p_n_size_browse-bin_3?rh=n:560798,n:560858,n:560864,k:televisions,p_n_size_browse-bin:161401031&sort=review-rank&keywords=televisions&ie=UTF8&qid=1401724746&rnid=161398031

http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

Teacher notes: What would your advice be?

Slide 27To sum up the lesson, students can be asked to work in pairs or groups to consider what advice they would give to people reading customer feedback and reviews. Responses can be invited to be shared with the class.

Their responses might include some of the following:• Look at the summary statistic, but be also look at the breakdown of

values• Take into account the number of reviews that contribute to the

average – a higher number of reviews is likely to be a more reliable value

• Read the comments and look at the dates to see what reviews have been like more recently.

Acknowledgements

• http://www.amazon.co.uk/s/ref=sr_nr_p_n_size_browse-bin_3?rh=n%3A560798%2Cn%3A560858%2Cn%3A560864%2Ck%3Atelevisions%2Cp_n_size_browse-bin%3A161401031&sort=review-rank&keywords=televisions&ie=UTF8&qid=1401724746&rnid=161398031

• http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

Accessed 2/6/14






http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

which would you choose?

Documents

star rating system

summary rating

customer rating

rosette rating

nearest half star

reviews surf

rosette hotel

similar hotels