MyLifeBitsJim GemmellJim Gemmell

February, 2005February, 2005

ConclusionWe have entered an era of virtually We have entered an era of virtually

unlimited storage, enabling the lifetime unlimited storage, enabling the lifetime store store

To make the store useful we need To make the store useful we need annotation, typed links, and database annotation, typed links, and database featuresfeatures

More capture, more correlation – less work More capture, more correlation – less work by the userby the user

Collaborators

Chief inspiration & guinea pig: Gordon BellChief inspiration & guinea pig: Gordon Bell Software development lead: Roger LuederSoftware development lead: Roger Lueder

MSR Collaborators: Lyndsay Williams, Ken Wood, MSR Collaborators: Lyndsay Williams, Ken Wood, Kentaro Toyama, Ron Logan, Steve Drucker, Curtis Kentaro Toyama, Ron Logan, Steve Drucker, Curtis Wong, Mary Czerwinski, Brian MeyersWong, Mary Czerwinski, Brian Meyers

Interns: Josh Blumenstock, Evan Salomon, Aleks ArisInterns: Josh Blumenstock, Evan Salomon, Aleks Aris

Outline

What is MyLifeBitsWhat is MyLifeBits History/MotivationHistory/Motivation MyLifeBits system outlineMyLifeBits system outline DemoDemo Future workFuture work

MyLifeBits is:An experiment in lifetime storageAn experiment in lifetime storage

Digitizing Gordon Bell’s pastDigitizing Gordon Bell’s pastCapturing more of his futureCapturing more of his future

A software systemA software systemCaptureCaptureStorage & retrievalStorage & retrievalOrganization & annotationOrganization & annotation

Minimum requirement: fulfill Vannevar Minimum requirement: fulfill Vannevar Bush’s 1945 “Memex” visionBush’s 1945 “Memex” vision

MemexAs We May Think, Vannevar Bush, 1945

““A memex is a device in which an individual stores all A memex is a device in which an individual stores all his books, records, and communications, and which his books, records, and communications, and which is mechanized so that it may be consulted with is mechanized so that it may be consulted with exceeding speed and flexibility”exceeding speed and flexibility”

Full-text search, text & audio annotations, and Full-text search, text & audio annotations, and hyperlinkshyperlinks

I am data

The guinea pig Has now scanned virtually all:Has now scanned virtually all:

Books written (and read when possible)Books written (and read when possible) Personal documents (correspondence including Personal documents (correspondence including

memos and email, bills, legal documents, papers memos and email, bills, legal documents, papers written, …)written, …)

PhotosPhotos Posters, paintings, photo of things (artifacts, …Posters, paintings, photo of things (artifacts, …

medals, plaques)medals, plaques) Home movies and videosHome movies and videos CD collectionCD collection And, of course, all PC filesAnd, of course, all PC files

Now recording: phone, radio, TV (movies), Now recording: phone, radio, TV (movies), web pages… conversations and meetings to web pages… conversations and meetings to comecome

Paperless throughout 2002. 12” scanned, Paperless throughout 2002. 12” scanned, 12’ discarded12’ discarded..

Only 44 GB, incl. 10 wma, 14 SQL!!! Video: Only 44 GB, incl. 10 wma, 14 SQL!!! Video: o(100) + 500 movo(100) + 500 mov

The 1 TB Life

1TB gives you 65+ years of:1TB gives you 65+ years of: 100 email messages a day (5KB each)100 email messages a day (5KB each) 100 web pages day (50KB each)100 web pages day (50KB each) 5 scanned pages a day (100KB each)5 scanned pages a day (100KB each) 1 book every 10 days (1 MB each)1 book every 10 days (1 MB each) 10 photos per day (400 KB JPEG each)10 photos per day (400 KB JPEG each) 8 hours per day of sound - e.g. telephone,8 hours per day of sound - e.g. telephone,

voice annotations, and meeting recordings (8 Kb/s)voice annotations, and meeting recordings (8 Kb/s) 1 new music CD every 10 days (45 min each at 128 Kb/s)1 new music CD every 10 days (45 min each at 128 Kb/s)

It will take you 5 years to fill up your 80 GB driveIt will take you 5 years to fill up your 80 GB drive Want video? Buy more cheap drives (1 TB/year lets Want video? Buy more cheap drives (1 TB/year lets

you record 4 hours/day of 1.5 Mb/s video)you record 4 hours/day of 1.5 Mb/s video)

Trying to fill a terabyte in a year Gordon’s lifetime collection < 30 GB Gordon’s lifetime collection < 30 GB

(12 GB is music CDs)(12 GB is music CDs)

ItemItem Per TBPer TB Per dayPer day

Photo Photo (400 KB JPEG)(400 KB JPEG) 2.7M photos2.7M photos 7.3K photos7.3K photos

1 MB document1 MB document 1.0M docs1.0M docs 2.9K docs2.9K docs

128 kb/s audio128 kb/s audio 18.6K hours18.6K hours 51 hours51 hours

256 kb/s video256 kb/s video 9.3K hours9.3K hours 26 hours26 hours

1.5 Mb/s video1.5 Mb/s video 1.6K hours1.6K hours 4 hours4 hours

“yet if the user inserted 5000 pages of material a day it

would take him hundreds of years to fill the repository, so that he can be profligate and

enter material freely”-Vannevar Bush, 1945

So you’ve got it – now what do you do with it?

Can you find anything?Can you find anything?Can you organize that many objects?Can you organize that many objects?Once you find it will you know what it is?Once you find it will you know what it is?Once you’ve found it once, could you find Once you’ve found it once, could you find

it again?it again?

““A record if it is to be useful … must be A record if it is to be useful … must be continuously extended, it must be stored, continuously extended, it must be stored, and above all it must be consulted” and above all it must be consulted”

““The difficulty seems to be, not so much that The difficulty seems to be, not so much that we publish unduly … but rather that we publish unduly … but rather that publication has been extended far beyond publication has been extended far beyond our present ability to make real use of the our present ability to make real use of the record” record”

- Vannevar Bush- Vannevar Bush

MyLifeBits Software

MyLifeBits store

database

Voice Voice annotation annotation tooltool

Telephone Telephone capture toolcapture tool

TV capture TV capture tooltool

TV EPG TV EPG download tooldownload tool

Radio Radio capture capture & EPG& EPG

PocketPC PocketPC transfer transfer tooltool

PocketRadio PocketRadio playerplayer

Import filesImport files

MyLifeBits MyLifeBits ShellShell

Browser Browser tooltool

InternetInternet

IM captureIM capture

GPS import & GPS import & Map displayMap display

SenseCamSenseCam

Screen saverScreen saver

Text Text annotation annotation tooltool

MAPI MAPI interfaceinterface

Legacy Legacy email clientemail client

OutlookOutlookinterfaceinterface

files

Legacy Legacy applicationsapplications

VIBEVIBElogginglogging

Entities & Links

AnnotatesAnnotates

Caller in Phone CallCaller in Phone Call

Photo of EventPhoto of Event

TranscludesTranscludes

MyLifeBits Schema(simplified)

ImagesImages

MusicMusic

Phone callsPhone calls

ResourcesResources

Relation-Relation-shipsships

Relation-Relation-ship typesship types

Entity typesEntity types

Resource Resource entitiesentities

Event Event typestypes

Event logEvent log

EventsEvents

TasksTasks

PeoplePeople

NotesNotes

Email Email MessagesMessages

Saved searchesSaved searches

DEMO

Future work: new capture modes/devices

SenseCam

Deja View

Body Media

Quindi

Future work: Visualizations Don't give me a little card Don't give me a little card

image and say, "That's all image and say, "That's all you've got, because that's you've got, because that's what I thought you should what I thought you should want for your virtual want for your virtual shoebox." There have got shoebox." There have got to be multiple modalities to be multiple modalities and the designers have to and the designers have to be able to deal with that. be able to deal with that. … don't metaphor me in, … don't metaphor me in, don't give me only one don't give me only one way of looking at things.way of looking at things.

-Andy van Dam, Hypertext '87 Keynote -Andy van Dam, Hypertext '87 Keynote AddressAddress

Next MediaNext Media

Web ScoutWeb Scout

U. MarylandU. Maryland IN-SPIRE

Future work: UI

UI ImprovementsUI Improvements User studiesUser studies

Future work:Content analysis & Data Mining

“Creative thought and essentially repetitive thought are very different things. For the

latter there are, and may be, powerful mechanical aids” – Vannevar Bush

Is MyLifeBits just enough rope to hang yourself with?Is MyLifeBits just enough rope to hang yourself with? MyLifeBits must become MyPersonalAssistantMyLifeBits must become MyPersonalAssistant Content analysis and data miningContent analysis and data mining Doc similarity & “clean living” Doc similarity & “clean living” Document meta-data extractionDocument meta-data extraction

Future work: scaling

Just starting to hit performance problemsJust starting to hit performance problemsStress tests & design modificationsStress tests & design modifications

www.MyLifeBits.com

http://research.microsoft.com/CARPE2004http://research.microsoft.com/CARPE2004

BONUS SLIDES

Everything goes in a database

You need all the features of a databaseYou need all the features of a database(Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, (Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication)replication)

If you don’t use one, you will find yourself creating one!If you don’t use one, you will find yourself creating one! Files as blobs, also sync with file system for legacy appsFiles as blobs, also sync with file system for legacy apps

SQLSQL

CARPE ’04The First ACM Workshop on

Continuous Archival & Retrieval of Personal Experiences

CARPE ’04The First ACM Workshop on

Continuous Archival & Retrieval of Personal Experiences

October 15October 15thth 2004 2004Columbia University, New York, NY, USAColumbia University, New York, NY, USA

October 15October 15thth 2004 2004Columbia University, New York, NY, USAColumbia University, New York, NY, USA

Dear Appy, How committed are you? Signed, Lost and Forgotten Data

Dear Appy,Dear Appy,I'm having trouble with long-term commitment -- not on my end, I'm having trouble with long-term commitment -- not on my end, heaven knows, but from the apps that created me and with whom I heaven knows, but from the apps that created me and with whom I like to associate. Over time, these pesky apps evolve and they like to associate. Over time, these pesky apps evolve and they simply don't recognize the data that they once helped create! But, simply don't recognize the data that they once helped create! But, we data progeny -- and there are lots of us -- feel that as our we data progeny -- and there are lots of us -- feel that as our creators, these apps should be responsible for eternal support. creators, these apps should be responsible for eternal support. But the little problem with recognition isn't the worst of it – But the little problem with recognition isn't the worst of it – sometimes the apps even disappear altogether. I ask you, is it sometimes the apps even disappear altogether. I ask you, is it expecting too much for 20-something year old data like me to be expecting too much for 20-something year old data like me to be interpretable by my app (e.g. Acrobat, DB2, Draw, Eudora, Office, interpretable by my app (e.g. Acrobat, DB2, Draw, Eudora, Office, Quicken, or RealNetworks), or am I just associating with Quicken, or RealNetworks), or am I just associating with irresponsible apps? irresponsible apps? If things continue on their current path, it seems I will be completely If things continue on their current path, it seems I will be completely un-interpretable within 20 to 50 years! My apps will move to other un-interpretable within 20 to 50 years! My apps will move to other platforms, or evolve to be more Internet- or Next-Big-Thing-centric...platforms, or evolve to be more Internet- or Next-Big-Thing-centric...

By Gordon Bell http://research.microsoft.com/~gbell

A Storocratic Oath

1.1. Do no harm to datesDo no harm to dates(File creation, Photo taken)(File creation, Photo taken)

2.2. Do no harm to device created & Do no harm to device created & other meta-data.other meta-data.

• Camera data & location data are sacred.Camera data & location data are sacred.

3.3. Support & aid the creation of critical meta-Support & aid the creation of critical meta-data. data.

• When/how the user feels like itWhen/how the user feels like it• Auto-magically!Auto-magically!

4.4. Maintain user confidentialityMaintain user confidentiality

Classification wish list Download classifications rather than build themDownload classifications rather than build them Definitions & synonyms should help find what I wantDefinitions & synonyms should help find what I want Today it is too expensive to manually classify my Today it is too expensive to manually classify my

scanned paper. E.g. “right time” meta-data is critical!scanned paper. E.g. “right time” meta-data is critical! Next year I hope “the system” can classify papers Next year I hope “the system” can classify papers

and other documents e.g. billsand other documents e.g. bills In 10 years I expect all documents to appear In 10 years I expect all documents to appear

electronically & classified electronically & classified with a little help from mewith a little help from me

Personal Search is notProfessional or Web search

System sees every entry & accessSystem sees every entry & accessEverything, not just a professional life Everything, not just a professional life Limited to SIS, not an infinite amount, Limited to SIS, not an infinite amount,

covers a profession & personal lifecovers a profession & personal life

Web as seen by search engines

MyLifeBits

Knowledge breadth e.g. Dewey classification

Depth e.g. information item types & coverage

Professional user