using web archives to enrich the live web experience through storytelling

Using Web Archives to Enrich the Live Web Experience

Through Storytelling

Yasmin AlNoamany Computer Science Department

Old Dominion University, Norfolk, VA [email protected]

Using Web Archives to Enrich the Live Web Experience Through Storytelling Yasmin AlNoamany

Once upon a time

Every story is made up of a sequence of events

2

Events are exemplified through corresponding web pages from the live web and web archives, (semi-)automatically discovered, arranged in a narrative structure ordered by time, and replayed through an appropriate visualization interface.


Do you recall the relevant dates, people, and web pages to replay the story of the Egyptian Revolution?

3

The story of the Egyptian Revolution


MOTIVATION

• There are three information needs:

– Overview

– Recency

– Replaying the story

4


5


6


7


8


Replaying the story

9


Replaying the story

Not yet addressed

10


Here's my hand-crafted story about the Egyptian

Revolution...

11


12

02/19/2010


13

06/10/2010


14

06/25/2010


15

07/23/2010


16

http://web.archive.org/web/20110707060758/http://www.thedailybeast.com/articles/2011/01/22/we-are-all-khaled-said-will-the-revolution-come-to-egypt.html

01/22/2011

http://web.archive.org/web/20110707060758/http:/www.thedailybeast.com/articles/2011/01/22/we-are-all-khaled-said-will-the-revolution-come-to-egypt.html























Using Web Archives to Enrich the Live Web Experience Through Storytelling Yasmin AlNoamany 17

01/25/2011


s

18

01/28/2011


19

01/28/2011


20

01/28/2011


21

01/29/2011


22

01/31/2011


23

01/31/2011


24

02/01/2011


25

02/02/2011


26

02/10/2011


27

02/10/2011


28

02/11/2011


Replaying the past can be more compelling more than a

summary

29


What about other stories?

30

Hurricane Katrina Boston Marathon bombings


Current Capabilities

31


Google provides summary and recency of information

32


Adding date does not solve the problem

33


How do we know to include this page?

34


The current status is not relevant to the purpose of the page

35


News websites are hard to search

36



37



38



39


Egyptian Revolution on Storify


Bookmarking, not preserving!


Archive-It collections

42


Seed URIs are gathered by people

43


44

Seed URIs are gathered by people


People gather ad hoc seed URIs

45


46


47


48


49


Research Question

Based on the terminology introduced earlier:

How do we define the time frame of a story, the individual events that make up a story, identify, evaluate, and select from candidate (archived) web pages to support the events, and visualize the result?

50


Our Preliminary Work

51


JCDL 2012: Visualizations for Archive-It Collections

52

• Archive-It collections are list view

– Hard to explore


JCDL 2012: Visualizations for Archive-It Collections

53

• Archive-It collections are list view

– Hard to explore

• We introduced a variety of visualizations for understanding the collections


JCDL 2013: How do people access web archives?

• We identified four major web archive access patterns: Dip, Slide, Dive, Skim

• The median length of the patterns is 3, which is very short! – We can not rely on web archive logs to create

seed URIs for stories

• People do not know about web archives – Robots outnumber humans 10:1 in terms of

sessions

54


TPDL 2013: Who and What Links to the Internet Archive

• Reaching web archives is not easy – 82% of human sessions connect to the Wayback Machine

via referrals

• External referrers link to the web archives because they can't find the web pages on the live web – 83% of all the pages that have links from outside the

archive do not currently exist on the live web

• People come to web archives because they can’t find the web pages on the live web – 65% of the requested archived pages no longer exist on

the live web

55


Dissertation Main Steps

56


How do we automatically construct this?

var mementos = [{ …

"Title": "Egypt braces for anti-government protests",

"Image": "im001.png",

"Snippet": "Egypt's authoritarian government is bracing itself …",

"Memento-Datetime": "25 January 2011 2:47 a.m.EST“

"URI": " http://web.archive.org/web/20110125024787/http://www.cnn.com/ ... “

}, {

"Title": "Will Egypt follow Tunisia's lead?",


"Snippet": " Demonstrators protest in central Cairo, Egypt, on Tuesday …",

"Memento-Datetime": "25 January 2011 3:00 p.m.EST"

"URI": " http://web.archive.org/web/20110125030042/http://www.cnn.com/ ... "

}, {

"Title": "Obama to Mubarak: Deliver on vows",


"Snippet": "President Obama spoke with Egypt's president moments after …",

"Memento-Datetime": "28 January 2011 8:56 p.m.EST“

"URI": "http://web.archive.org/web/20112801085658/http://www.cnn.com/...”

“

}, {

"Title": "Without police, Cairo 'like Wild West",


"Snippet": " With local police effectively no longer on the ground in …",

"Memento-Datetime": "29 January 2011 3:00 p.m.EST“

"URI": "http://web.archive.org/web/20110129030000/http://www.cnn.com/ ... “}

… ];

57


Usage Scenario

58

I want to replay the story of Egyptian Revolution, but I

don’t remember its date


Lori has her search box for replaying the story

59


Step 1: Calculating the datetime of the story dynamically

60

• Problem – People remember the name of the story, but not the date – Some stories have no specific time. History is not static!

• Methods: – Use the time in Wikipedia infoboxes Hoffart et al. 2012 (may be not

applicable for all stories) – Extract temporal expressions from unstructured text using time

and event recognition algorithms Kanhabua et al. 2012

– Look up in news sites, such as wikinews – Investigate different techniques for finding the time range of the

story

• Evaluation – Use gold standard dataset of stories that have specified

datetimes


Step 2: Get the URI seeds for the story

• Problem – Specifying the top K related pages automatically

• Methods – Use the list of references on Wikipedia page and Check the older versions of the

page for URIs – Use Google’s API to search for the term of the story – Query the archives full text search (such as Archive-It and UK Web Archive) – Use Social media web sites (such as storify, twitter, and topsy) – Determine information retrieval techniques for finding relevant links from the

seeds.

• Evaluation – Test the aboutness of the web pages

• Check the anchor text and its relation to the content. Furthermore, examine the existence of the URI in the live web and its aboutness by looking to the page content in the web archives

– Check the coverage of Google search against the coverage of the links from social media and from the web archives

– Contrast the cost with the quality (Google may have bias, but it may be faster)

61


Step 2: Different sources give us different URI seeds

62


Step 3: Determine datetimes of web pages

• Problem – There are many notions of time for a web page (creation

date, modification date, archiving date, etc.) – Hard to know the creation date

• Methods – Compute and assign those notions of time to each web

page • Estimating the date of a web page by looking at the

pages that link to it • Carbon dating SalahEldeen et al. 2013

• Evaluation – Use gold standard dataset of articles that have clear

timestamps to be extracted

63


Step 3: This page was archived two months after the start of the Revolution

64


Step 4: Suppose we got 10,000 related pages for each event of the story

65

I’ll add here many copies from bbc, nytimes, foxnews


Step 4: Choose the best candidates for each event of the story

• Problem – Specify the best high quality pages that express the

story

• Methods – Eliminate the duplicates – There are multiple dimensions of quality:

• Web-based structural quality, such as page rank • Quality of replaying the archived page

• Evaluation – Choose small number of candidates for each event – Show all candidates and ask Mechanical Turkers if the

candidates are good or not

66


Step 4: Example of duplication

67


Step 4: The same event from two different news sites

68


Step 4: Quality of archived pages

69


Step 5: Visualize the results Interactive Timeline

70

Replaying Story of Egyptian Revolution


Step 5: Visualize the results Slideshow

• Different View

71


Step 5: Visualize the results using Storify

72


Step 5: Visualize the results

73

• Problem – From the chosen pages, how to create a human-readable

story and visualize it?

• Methods – Provide different interactive visualizations that enable

exploring the story easily, such as • Timeline Visualization • Slideshow Visualization

– Provide the user with the ability to modify the story and specify the start and end dates

• Evaluation – Solicit feedback from humanities researchers


Step 6: Make the created story accessible for others

• Problem

– People do not know about the created story

– Discovering links between stories

• Method

– Allow sharing of the story with others

– Allow feedback and updating of the story by others

74


Dissertation Timeline

75

Year1 Year 2 Year 3 Year 4 Year 5

Background Research

Course Work

Qualifying Exam

Proposal writing

Candidacy

Framework Implementation

Calculate the datetime of the story dynamically

Get the seeds of URIs

Determine the datetimes of web pages

Choose the best candidates for each event

Visualize the results

Evaluate the framework

Make the created story accessible for others

Dissertation Writing

Ph.D. Defense

Completed To be completed Not started

Oct. 2012 Nov. 2013 Dec. 2015


Extra Slides

76


User Access patterns in Web archives

77


Everybody Dips, Humans Dive, Robots Skim

78

Robots (34,203 sessions) Humans (3,431 sessions)


Prior Work User Access Patterns in Web Archives

79


86% of the links to mementos

80


Using Wikipedia infobox for detecting the time of the event

81


The Last modified date is months away from archiving date

82

http://web.archive.org/web/20110707060758/http://www.thedailybeast.com/articles/2011/01/22/we-are-all-khaled-said-will-the-revolution-come-to-egypt.html






















