archive-it collection on “occupy movement 2011/2012” archiving web content

Archive-It collection on “Occupy Movement

2011/2012”Archiving Web Content

Archive-It

• Web archiving service first deployed at the Internet Archive in 2006

• In 2007, started to collect “at risk” web content on spontaneous events that occur in the US and the world.

• Web content needs to be documented and archived for historical and cultural purposes

• Curators use Archive-It to add websites, metadata and set up automated crawls

• Digital Collections are all publicly available

“Occupy Movement 2011/2012” collection

• Collection is publicly available at:http://archive-it.org/collections/2950

• Organized into Website Groups: – Blogs, International, News Sites and Articles, Other Sites,

Social Media

Collection was created Nov 30, 2011:• Web sites selections were just filtering in.

• Working with “Activist Archivists”, groups from NYU, OWS, and other individuals.

• Named the collection “Occupy Movement”, to include content from around the world.

• Staff at the the Internet Archive created a blog post to generate visibility and seed submissions for the collection.

“Occupy Movement 2011/2012” collection

Current Crawling Activity on “Occupy Movement 2011/2012”

• Have included 770 websites to be crawled• Captured 17 million documents• Archived 637 gigabytes of data• Crawling daily, weekly, and monthly

Managing the “Occupy Movement 2011/2012” Collection

• Seed submissions:– bulk and single website submission from curators of

content and other individuals– scraped and included websites from community

generated lists (e.g. “We All Occupy”, “Occupy Feeds”)• Monitor and check crawls:– looking for crawler traps– adding crawling rules to capture content where

needed

Global Content

“Occupy Clermont-Ferrand”France

•http://wayback.archive-it.org/2950/20120210032957/http://www.occupyclermont.org/

“Mi smo 99%”Serbia

•http://wayback.archive-it.org/2950/20120210032944/http://occupyserbia.org/

Unique Content

“Occupy Writers”Static websites with unique content

that may not be maintainedhttp://wayback.archive-it.org/2950/20111217041530/http://occupywriters.com/

News & Blogs

News Articles:Article about arrest of Occupy

Protestorshttp://wayback.archive-it.org/2950/20120105015434/http://www.theatlanticwire.com/national/2012/01/occupy-livestream-operators-will-be-homeless-after-they-get-out-jail/46989

/

Special Interest Groups:Article about destruction of OWS

“People’s Library”http://wayback.archive-it.org/2950/20111218041012/http://mhpbooks.com/44284/ala-calls-nypd-destruction-of-ows-peoples-library-unacceptable/

Images & Video

Photo Albums of Events:“Occupy Long Beach October

18 2011”http://wayback.archive-it.org/2950/20120221032852/http://occupylb.org/photos/occupy-long-beach-october-18-2011/

“25 best Occupy photos of 2011”http://wayback.archive-it.org/2950/20120107024836/http://news.nationalpost.com/2011/12/31/25-best-occupy-photos-of-2011-2/

Citizen Video:Pepper spray victim in

Birminghamhttp://wayback.archive-it.org/2950/20120327043950/http://www.occupyalbany.org/wp-content/uploads/2011/12/treating-sprayed-protester.3gp

Thank you!The Archive-It Teamwww.archiveit.org

graham@archive.org