the perfect search engine is not enough jaime teevan †, christine alvarado †, mark s. ackerman...
TRANSCRIPT
![Page 1: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/1.jpg)
The Perfect Search Engine Is Not Enough
Jaime Teevan†, Christine Alvarado†, Mark S. Ackerman‡ and David R. Karger†
† MIT, CSAIL‡ University of Michigan
![Page 2: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/2.jpg)
Let Me Interview You!
Email:–What’s the last email you read? What did you do with it? –Have you gone back to an email you’ve read before?
Web:
Files:
–What’s the last Web page you visited? How did you get there?–Have you looked for anything on the Web?
–What’s the last file you looked at? How did you get to it?–Have you looked for a file?
![Page 3: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/3.jpg)
SearchOverview:Understanding
Introduction Related work Methodology What we learned
– How?– Why?– Who?– So what?
Directed
Introduction Related work Methodology What we learned
– How?– Why?– Who?– So what?
![Page 4: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/4.jpg)
Haystack:Personal Information Storage
Email Web pages
Files Calendar
Contacts
Haystack
![Page 5: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/5.jpg)
Directed Search in Haystack
What was that paper I read last week about
Information Retrieval?Haystack
![Page 6: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/6.jpg)
Directed Search in Haystack
Ah yes! Thank you.
Haystack
“Perfect Search Engine”
![Page 7: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/7.jpg)
Related Work
Directed search– Lab studies [Capra03, Maglio97]– Log analysis [Broder02, Spink01]
Observational studies [Malone83]
Information Seeking– Marchionini, O’Day and Jeffries, Bates, Belkin, …– Evolving information need
![Page 8: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/8.jpg)
Modified Diary Study
Subjects: 15 CS graduate students Ten interviews each (2/day x 5 days) Two question types
– Last email/file/Web page looked at– Last email/file/Web page looked for
Supplemented with direct observation and an hour-long semi-structured interview
![Page 9: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/9.jpg)
Directed SearchOverview:Understanding
Introduction Related work Methodology What we learned
– How?– Why?– Who?– So what?
![Page 10: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/10.jpg)
Directed Search Today
Target: Connie Monroe’s office number
Type into a search engine: “Connie Monroe, office number”
![Page 11: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/11.jpg)
What We Observed
Interviewer: Have you looked for anything on the Web today?Jim: I had to look for the office number of the Harvard professor.I: So how did you go about doing that?J: I went to the homepage of the Math department at Harvard
![Page 12: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/12.jpg)
What We Observed
I: So you went to the Math department, and then what did you do over there?J: It had a place where you can find people and I went to that page and they had a dropdown list of visiting faculty, and so I went to that link and I looked for her name and there it was.
![Page 13: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/13.jpg)
What We Observed
J: I knew that she had a very small Web page saying, “I’m here at Harvard. Here’s my contact information.”
![Page 14: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/14.jpg)
Strategies Looking for Information
Teleporting
Orienteering
![Page 15: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/15.jpg)
Why Do People Orienteer?
Easier than saying what you want You know where you are You know what you find
The tools don’t work
![Page 16: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/16.jpg)
Easier Than Saying What You Want
Describing the target is hard– Can’t– Prefer not to
Habit – “Whichever way I remember first.”
Search for source– E.g., Your last email search
![Page 17: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/17.jpg)
You Know Where You Are
Stay in known space– URL manipulation– Bookmarks– History
Backtracking– Following an information scent– Never end up at a dead end
![Page 18: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/18.jpg)
You Know What You Find
Context gives understanding of answer
“I was looking for a specific file. But even when I saw its name, I wouldn’t have known that that was the file I wanted until I saw all of the other names in the same directory…”
Understanding negative results
“I basically clicked on every single button until I was convinced… I don’t think that it exists…”
![Page 19: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/19.jpg)
Individual Search Behavior
Search behavior varied by individual Categorize based on email usage
– Filers– Pilers
People who pile information take small steps People who file information take big steps
![Page 20: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/20.jpg)
0 1 2 3 4 5 6 7 8 9
ABCDEFGHIJKLM
Keyword Search Other
How Individuals Search For Files
Filers
Pilers
Big steps
Small steps
![Page 21: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/21.jpg)
More to Learn from the Data
Differences in finding v. re-finding How organization relates to search Importance of type (email, files and Web) Looked at v. looked for
Keep in mind population
![Page 22: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/22.jpg)
Support orienteering
Applying What We Learned
Advantages to orienteering– Easier than saying what you want– You know where you are– You know what you find
Individual differences in step size
– Highlight source (e.g., flag sources with info) – Integrate tools used for steps– Support exhaustive search
– Allow for different step sizes
![Page 23: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/23.jpg)
More to Learn from the Data
Differences in finding v. re-finding How organization relates to search Importance of type (email, files and Web) Looked at v. looked for
Keep in mind population
![Page 24: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/24.jpg)
Structural Consistency Important
All must be the same to re-find the information!
![Page 25: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/25.jpg)
Supports orienteering for re-finding Allows access to new information
Preserve What User Remembers
![Page 26: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/26.jpg)
File or Pile Email
0
1
2
3
4
5
6
7
8
0 20 40 60 80 100
% found in Inbox
# o
f se
arch
es
Filer
Piler
![Page 27: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/27.jpg)
Searching Other Collections
Ah yes! Thank you.
![Page 28: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/28.jpg)
Keep Population in Mind
CS grad students not representative Very familiar with search tools
Would expect to see lots of tool use
![Page 29: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/29.jpg)
Orienteer to specific information
Relating How and What
People only keyword search 39% of the time What people look for related to how they look
Specific General Document
Other 47 19 41
Keyword 34 23 17
Surprise:
![Page 30: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/30.jpg)
Relating How and Corpus
Email and files: Almost never keyword searched Easy to associate information with document Web: Used keyword search much more often
Email Files Web
Other 59 42 19
Keyword 06 10 64
![Page 31: The Perfect Search Engine Is Not Enough Jaime Teevan †, Christine Alvarado †, Mark S. Ackerman ‡ and David R. Karger † † MIT, CSAIL ‡ University of Michigan](https://reader036.vdocuments.site/reader036/viewer/2022081516/56649de45503460f94adac24/html5/thumbnails/31.jpg)
Relating What and Corpus
Email Files Web
Specific 39 7 33
General 10 7 30
Document 08 35 14
Email searches were primarily for specific information File searches were primarily for documents Web searches were more evenly distributed