Making Sense of Twitter Search

Gene Golovchinsky / FXPALMiles Efron / UIUC

100,000,000 people can’t be wrong

All the cool researchers are doing it

There is Science in everything

We can build something cool

Maybe we can build something useful!

How do we know we’ve succeeded?

Test collections (data sets) are one way to evaluate systems

The fire hose is not a test collection

Test collection to-do list

• Research questions

• Data collection methodology

• Metrics

• Data!

What are we trying to study?The network?

Or the person?

We’re interested in interactive search

What do people look for on Twitter?

• Trending topics

• Events

• Specific tweets

• Catching up

• Etc.

First Steps

First Steps

First Steps

Evaluation in Information Retrieval

• Need a collection

• Need tasks

• Need metrics

• Ground Truth helps

How do we sample the data?

• By time, by topic, randomly? …

How do we measure?

Credits

• Title slide– http://www.flickr.com/photos/matthamm/

3383916444/ Matt Hamm– http://www.flickr.com/photos/nixart/370553161/

nixArt• “10,000 people can’t be wrong”

– http://www.flickr.com/photos/luc/1824234195/ luc legay

• “All the cool researchers are doing it” – Gene Golovchinsky

• “There is science in everything”– Gene Golovchinsky

• “We can build something cool”– http://www.flickr.com/photos/donpezzano/

3230179951/ urban don• “Maybe we can build something useful!”

– http://www.flickr.com/photos/herzogbr/2274372747/ herzogbr

• “How do we know we’ve succeeded?”– http://www.flickr.com/photos/

14696088@N04/1606128341/ rstrawser• “Test collections (data sets) are one way to

evaluate systems”– http://www.flickr.com/photos/_after8_/

4052028795/ Régis Gaidot• “The fire hose is not a test collection”

– http://www.flickr.com/photos/magh/2565471904/ Magh

• “Test collection to-do list”– http://www.flickr.com/photos/ario/3362388566/

ario_• “What are we trying to study? The network?”

– http://www.flickr.com/photos/meg/3533025291 Meg Pickard

• “Or the person?”– Gene Golovchinsky

• We are interested in interactive search– http://www.flickr.com/photos/dullhunk/

3389581452/ dullhunk• What do people look for on Twitter?

– http://www.flickr.com/photos/kevglobal/3170506186/ kevglobal

• First steps (3)– Gene Golovchinsky

• Evaluation in Information Retrieval– http://www.flickr.com/photos/ozyman/

155517365/ Ozyman• How do we sample the data?

– http://www.flickr.com/photos/red_devil/18540700/ SeenyaRita

– http://www.flickr.com/photos/texasrobo/4185851361/ texasrobo

• How do we measure?– http://www.flickr.com/photos/jurvetson/

17945646/ jurvetson