V. Bellotti, B. Begole, et al.CHI 2008 Proceedings, pp. 1157-1166

Activity-Based Serendipitous Recommen-dations with the Magitti Mobile Leisure

Guide

Motivation & Introduction• Motivation

– Traditional city guide• “Time Out” in London and New York, and “Tokyo Walker” in Tokyo

– Location-based services• Search for local restaurants, movies, stores and so on

– Discovery of activities and venues in context-aware computing

• Magitti project– Sponsored by Dai Nippon Printing Co., Ltd. (DNP)

• DNP, one of Japan’s largest printing companies– Development of a service to replace printed city guides

• An activity-centered mobile leisure-time guide• Delivering timely and personally relevant recommendations about nearby

venues• Predicting future activity based on the user’s context and models of past be-

havior– Target

• People : 19~25 year-olds• Locations : Japanese cities that have so many venues

Understanding Leisure Time Priorities• Dearth of English literature on Japanese leisure time activi-

ties– Previous time-based survey → too coarse activity for specific recommen-

dation

• Field exercises for following questions– How do young Japanese spend their leisure time?– What resources do they use to support leisure time?– What needs exist for additional support by a new kind of media technol-

ogy?

• Methods for field exercises– Interviews and Mockups (IM)

• 20 semi-structured interviews with 16~33 year olds• 12 interviews with 19~25 year olds

– Online Survey• A survey on a market research web site to get statistical information• 699 responses from 19~25 year olds

– Focus Groups• 6~10 participants for each group• Presentation about a walkthrough of the Magitti mock-up and its functions• Gathering detailed feedback on the concept

Understanding Leisure Time Priorities• Methods for field exercises

– Mobile Phone Diaries (MPD)• Daily activities of 19~25 year olds• Two mobile phone diary studies

– First study → 12 people for one Sunday– Second study → 21 participants for a seven-day week

– Street Activity Sampling (SAS)• 367 short interviews with people in target age range

– Reporting three activities from their day– Choosing one as a focal activity– Classifying the activity into one of a number of pre-determined types

– Expert Interviews• Three experts on the youth market in the publishing industry• Information commonly published to inform and support their activities

– Informal observation• Observing young adults in popular Tokyo neighborhoods at leisure

Critical Findings from Field Exercises• How young people in Tokyo spend their leisure time ?

– Shopping > going out with friends > dining out > going on a date > do-ing sport

– Activity frequency in SAS interviews• Dining (31.8%), shopping (24.6%), browse/explore/look (7.5%)

– Dining and shopping are major activities that involve going out

• What resources are used to support leisure time ?– Friends and family, TV, Internet, and Magazines

• Online survey respondents → Internet• IM interviewees → Friends and family

– Information based on personal experiences of friends and family• Most trusted but not very extensive

• What needs exist for additional support ?– 58.8% of SAS interviewees : interest for more information to support fo-

cal activity– Requests for information

• Maps and venue locations (14.6%)• Customers’ and friends’ opinions (8.2%)• Prices (7.8%)• Store/venue contents (6.8%)

Design Requirements• Relaxation, Serendipity and Spontaneity

– Relaxation• Busy schedules, often with multiple occupations (e.g., student and part-time-

worker)– Serendipity

• Attraction to serendipitous information

• Avoidance of Information Overload– Reducing leisure information to only the most relevant

• Minimal size– Particular preference of the younger generations

• As small as possible in a pocket

• One-handed operation– Strong requirement for one-handed operation by interviewees

• Focusing on relaxation, serendipity, and spontaneity– Generating recommendations automatically using activity inference

Magitti Design• Magitti

– Context filtering to reduce overload of leisure time in dense urban areas– No requirement of explicit definition of a user’s profile or preferences– Inference of interests and activities from learned models

• Using data such as places visited, web browsing, and communications with friends

• Magitti’s three key features– Context Awareness

• Using current time, location, weather, store hours, and user patterns– Activity Awareness

• User’s inferred or specified activity based recommendation• Eating, Shopping, Seeing, Doing, or Reading

– Serendipitous, relaxing experience• Not necessary for profile, preferences, or queries• Activity inference for Magitti using context

Related Activity-Detection Research• Lamming and Newman’s activity-based information-retrieval

system– One early system related to Magitti– Presentation of information that was generated in contexts– Impossible to infer activity with effective accuracy

• Other activity detection approaches– Begole et al. : sensor-based availability detection– Inference of human activities from use of objects with RFID tags– Inference of human activities by using video and audio data analysis– Froehlich et al. : finding correlations between place preference and data

• Activity modeling research– Liao et al. using location-based sensing with Relational Markov Networks

• ‘AtHome’, ‘AtWork’, ‘Shopping’, ‘DiningOut’, and ‘Visiting’

• Previous works– Detection of a person’s current activity– Magitti guide system : predicting a person’s future activities

Related Mobile City Guide Applications• Location-based information recommendation system

– Similar in spirit to Magitti: location-aware tourist guides

• Some systems to recommend venues based on the user’s state

– No prediction of the user’s activities

• Cyberguide– A mobile tourist guide for the Georgia Tech campus– Awareness of its time, location, and history– Matching information on venues and special events to the data

• MobyRec– A context-aware mobile tourist recommender system

• Hotels, restaurants, etc. – Improvement of recommendations over time

Related Mobile City Guide Applications• GUIDE

– Providing tour routes and accesses ticket reservation services– Dynamically recomputing routes based on location and time– Targeting for touring unfamiliar areas

• COMPASS– A tourist guide service covering a wide range of venue types– Using profile and goal information entered by its user– Using location, speed, user profile, schedule, shopping list, and recent

visit– Filtering by the user’s stated goal and preferences

• CRUMPET– Providing tips, tour suggestions, maps and other information on a range

of tourist-related venues (restaurants, movies, shows, etc.)– Learning user preferences over time

Magitti : User Interface• Main Screen

– A scrollable list of up to 20 recommended items in Main Screen• Matching the user’s current situation and profile

– Automatic list update to show items relevant to new locations

• Detail Screen– Viewing Detail Screen by tapping each recommendation

• Initial texts of a description, a formal review, and user comments• Rating the item on a 5-star scale by a user

Magitti : User Interface• Partial map on the Main Screen

– Showing the four items currently visible in the list

• Minimal size and one-handed operation requirements– Large buttons on the screen to enable the user to operate Magitti with a

thumb– Marking menus on touch screens to operate the interface

• Menu buttons at the bottom of the Main Screen– Adjusting the recommendation list if needed– Five modes of user activity; Eat, Buy, See, Do, or Read– Recommendations from just one category – Bookmarking recommended items

Magitti : System Architecture• Client-server architecture

• Mobile client UI on a handheld device – Providing data for the Context Sensing Module

• Gathering data about user’s physical context and data con-text

– User’s physical context• GPS, time of day, user inputs, weather

– Data context • Content of emails sent/received, calendar, web pages and documents viewed,

applications used

Magitti : Activity Prediction Module• Data for probabilistic modeling

– Using data collected on Magitti’s target demographic in the fieldwork– Japanese Survey on Time Use and Leisure Activities

• Modeling the frequency of each mode by tracking user be-havior

– Visiting a retail store → Buy– Visiting a restaurant or café → Eat– Visiting theater or museum → See– Gym or park → Do– Reading of content on Magitti itself → Read

Magitti : Recommender System• Computation of the utility of each content item

– Combining results from a variety of recommendation models– After computing scores of all items, top results are allocated in the slot

• Computing score for an item in Magitti – Combination of Eight Model

Computing score for an item in Magitti • Collaborative filtering

– Computing similarities between users– Determining scores each item based on how other similar users rated it

• Stated Preferences– Scoring items according to how closely they match the user’s stated

preferences• Learned Preferences

– Learning from observed behavior rather than explicitly stated prefer-ences

• Content preference– Measuring the similarity of an item’s content to a user’s profile

• Distance– Items within a distance range (either entered or inferred from location

traces)• Reading

– Using a model of users from the fieldwork• Boredom Buster

– Reducing scores of items that have previously been seen• Future Plans

– Temporarily raising scores based on future plans derived from the Con-tent Analysis

Data Context Detection• Detecting the user’s physical context

– Calendar appointments, viewed documents, and messages to extract in-formation about the user’s plans

– Leisure activity plans with friends using mobile email and SMS

• Test for the potential usefulness of SMS– 10,000 SMS messages by students at the National University of Singa-

pore– 11% of the messages related to leisure activities

• Prototype Content Analysis module– Only Eat and See activity planning– Other activities planned for future work

Field Evaluation• 11 volunteers with Magitti in the Palo Alto, California area be-

tween one and four times each over several days– Participants, who were company employees not working on the project,

ranged in age from mid-20s to late-50s, and averaged 37

• Visiting a total of 60 places over 32 outings, averaging 1.9 places per outing.

– About half the outings (16) accompanied by a family member or friend

Supporting Serendipity• Try to find a new place(such as restaurant)

– Very successful at discovering new places– Over half new places (53%)

• Including 38% that they had never heard of• Including 15% they had heard of but never been to

– Places visited once or twice (25%)– Places visited many times (23%)

• People’s expression about finding new places– “Cool! I like that. I would never have found that place if it wasn't for this.” – “I think it makes life more interesting. It allows you to get out of your

daily routine, almost as if you’re going to a different city.”

• Magitti’s overall usefulness– 4.1 on a scale of 1-5 (5=very helpful)– Useful for residents and not just tourists or newcomers

Predicting User Activity• User activities in the experiment

– Visiting 30 places to Eat, 27 to Buy, and 3 to Do– Some most frequent activities

• Changing activity type : an average of 5.1 times per outing– Eat (1.8 times per outing), Buy (1.4), Do (0.7), See (0.5), and Read (0.1)– “Any” mode : an average of 0.7 times per outing

• Wrong inference : easy to switch to a different activity

Context-Aware Recommendations• Relevant and interesting recommendations

– Average rating of 3.8 (1=rarely, 5=almost always)• A little less than “usually”

– A person’s opinion• “Most of the time, the list contained a mix of useful and not so useful recom-

mendations”

• Several factors that affected people’s confidence in the sys-tem

– Omission• “the list did not represent what downtown has to offer”• Small omissions or inaccuracies reduced people’s trust

– Distance• People expect that the closest places would be at the top of the list• Poor recommendation if it required driving

– First Item• More weight on the first item recommended• Reasonable first item → good recommendations

– Guide vs. Recommender• Relatively less loss of confidence for recommendation of a closed place

– Some people : location information guide (closest place)– Other people : recommender (similar place)

Context-Aware Recommendations• Several factors that affected people’s confidence in the sys-

tem– Transparency

• Some users try to understand how Magitti decided which activities and venues to list

• A complex set of algorithms based on many factors – Location, time, preferences, similar users’ opinions, prior behavior

• Lack of transparency of the algorithm – sometimes confusing or even frustrating users

• Need for offering more cues to help users develop an appropriate user model

Issues & Conclusion• User Control

– Desire to have more control in managing the recommendation list– Ability to sort the items by factors such as rating, price, or distance– Ability to remove items from the list

• Social Use– Outings involved two or more people– Incorporation into a social setting

• Conclusion– Predicting the user’s current and future leisure activity– Modeling the user’s preferences, to filter and recommend relevant con-

tent– An interface with a novel one-handed, thumb based interaction