personal born-digital in what environments do we...

3/6/20

1

ArchivingtheNon-Organiza8onalBorn-Digital:TheChallengesPosedby

MaterialfromIndividuals,Communi8es,&Events

HowardBesserMovingImageArchiving&Preserva8on

NewYorkUniversityhQp://besser.tsoa.nyu.edu/howard/Talks/hQp://www.nyu.edu/8sch/preserva8on/

Besser-Berkeley3/6/2020 1

ArchivingtheNon-Organiza8onalBorn-Digital:TheChallengesPosedbyMaterialfromIndividuals,Communi8es,&Events

•  Background&TheProblemofPersonal,Community,&Event-basedDigitalArchiving(forbothtextandimage)

•  Thepar8cularproblemofMessaging•  ThePDAConferences•  MyProjectsthatprovidedkeyconcepts&Challenges

–  InterPARES–  PreservingDigitalPublicTelevision–  Ac8vistArchivists&theOccupyMovement

•  ImportantLessons&ApproachesLearned(FromAc8vistArchivists&Otherprojects)

•  OtherprojectstoMonitor•  FurtherAppliedResearchneeded


Personalborn-digital(fromPDAConferences)

•  Correspondence/email•  Personalphotos/moviesandgroupcollec8ons•  Manuscriptdra\s,cameraoriginalfootage,roughcuts•  Personaldocuments•  Diaries•  HomemoviesAndhasbeenextendedtoencompass:•  Familyhistory•  Community/Ethnichistory&Movements•  Genealogy•  Digitalhumani8es


Inwhatenvironmentsdowefindthistypeofmaterial?

•  ArchivesandLibrarySpecialCollec8ons•  Collec8onsdocumen8ngacommunity•  Collec8onsdocumen8nganethnicgroup•  Collec8onsdocumen8ngasocialmovement•  Collec8onsdocumen8ngtheworkofanyothertypeofgroup(agroupofArchitects,asetoflaw-makers,etc.)

•  Collec8onsdocumen8nganevent-


Documen8nganEvent


Documen8nganEvent


3/6/20

2

Documen8nganEvent


Well-funded“an8-racist”ac8vists,withbackingfrombillionaireJewsandgovernmentins8tu8ons,preventracially-awareWhitepeoplefromengaginginonlinecommerce.TheNa8onalAllianceandCosmotheistChurch,forexample,havebeeneffec8velybannedfromonlinecommercefornearlytwoyears.NowtheJewsandtheirpawnsaretryingtoextendthiscensorshiptototallynon-poli8calvenueslikethisBloomington,Indianafarmers’market.

https://nationalvanguard.org/2019/06/indiana-pro-white-farmers-threatened-persecuted-by-jewish-funded-leftists/

Documen8ngaCommunityGroup


Documen8ngaSocialMovement


Señal3LaVictoria(San8ago,CL)

10Besser-Berkeley3/6/2020





3/6/20

3

Copwatch(DA?)


ArchivingSocialMovementstoday(Video)hQps://www.ny8mes.com/video/world/asia/100000006970549/coronavirus-chinese-ci8zens.html


HomeVideoDay—NYCChinatown


HomeVideoDay—NYCChinatown


CenterforAsianAmericanMediaCAAMchannel“MemoriestoLight”

•  “HomemoviesoccupyauniqueplaceinAmericanculture.Thoughgenerallydismissedfortheiramateurquali8es,homemoviesprovideuswithexceedinglyauthen8candhonestmovingimages.Premisedonthehistoric,cultural,andar8s8cvalueofthehomemovie,MemoriestoLight:AsianAmericanHomeMoviesisana8onalpar8cipatoryartsprojectthatcollec8velyandaesthe8callyconstructssharedsocial,cultural,andpoli8calrepresenta8onsofAsianAmericadirectlyfromthecommunityitself.Sincethemainstreammediahasgivenussofewauthen8cimagesoftheAsianAmericanexperience,homevideosbecomethemostrealwaytoseehowourgrandparents,mothers,fathers,auntsanduncleslivedtheirlives.”


CAAMchannel“MemoriestoLight”


3/6/20

4

Interna8onalDigitalEphemeraProject


GENERALPROBLEMSOFTHISTYPEOFBORN-DIGITALCONTENT


Intheanalogworld

•  Tradi8onally,wehavecometounderstandtheworkofwriters,scien8sts,filmmakersbyscholarsstudyingtheirpapersandrough-cutsinSpecialCollec8onsandArchives

•  Theircorrespondenceandprogressivelydifferentdra\sofpapersandrough-cutsrevealtheirchangingthoughtsandcra\

•  ButhowdowegathertheseintheDigitalAge?


AlasdairGray'sLanark(GlasgowULibrary)


Correspondence


Wherecanwefindthesetoday?

•  DopeoplewriteleQersonpaper?Canweseetheitera8onsofchangesonmanuscripts?DopeoplesavetheirEDLs?

•  Wherecanwefindtoday’sequivalentofthese?


3/6/20

5

Stagesoftheproblem

•  Stage#1:Peoplerecordondigitalmediainsteadofanalog

•  Stage#2:Peoplenolongerstoretheirdigitalworksinplacesoverwhichtheyhaveabsolutecontrol– Emailservices(gmail,yahoo)– Cloudstoragefordocuments(googledocs)– Socialnetworkservices(Vimeo,YouTube,Instagram)


Stage#1Issues-digitalreplacesanalog


Stage#1Issues-digitalreplacesanalog

•  Thiswillrequire– newinterven8ons(likechangingcreators’workflow,savingEDLs,orinterveninginemailhandlingso\ware)

– Newtools(likeforanalyzingemail)– newapproacheslikedigitalarcheology,forensics


Stage#2Issues-contentnolongeronharddisk

•  RiseofOnlineServicesandSocialMediaischangingwherethiscontentresides(andisimposingTOSrestric8onsthatgobeyondtherightsholder)


MuchContentisonFacebook


Iden8tarianPlarormonlyonWeb


3/6/20

6

CoreMul8-loca8onProblems

•  It’sdifficultenoughwhensomeone’sphotosormoviesarespreadthroughouttheirharddisk.Buttodaysomeimagesthere,butothersontheirphone(s),YouTube,Vimeo,Instagram,Flickr,Facebook,inTweets,etc.

•  Similarproblemsplagueemail•  MostSocialNetworkTOSpoliciesprohibittheownerfromgivingtheirpasswordtoanyoneelse(evenLibrary)


Andhowdowehandledona8onsa\eranimportantpersondies?


AndtheseissuesarealsotrueforCommunityGrps&Assns

•  w/SocialMedia,groupac8vityismoreimportantthanever

•  Buteachpersoninthegroupisanindividualcollector.Andfrequentlyasetofindividualcollec8onsformsthegroupcollec8on.


Documen8ngProtests

Besser-Berkeley3/6/2020

-photo from Activists Guide to Archiving Video

34

Whenaggregated,manydifferentpersonalcollec8onsformanimportant

pictureof:•  Anevent•  Anethnicgroup•  Acommunity•  Asocialmovement•  Asetofarchitects•  Asetoflaw-makers

•  Whatisimportanttothem,howtheygoabouttheirbusiness,…


Andweknowfrompastworksthataggrega8onscreatenewmeanings

•  Aggrega8ngallthephotosandhomemoviesoftheDigitalDiasporaishugelymoremeaningfulthanasinglephoto-

•  OnetweetsaysveryliQle,butthousandsoftweetscanshowtrendsordepictapar8culareventorday


3/6/20

7

DigitalDiasporaFamilyReunion


DigitalDiasporaFamilyReunion


Andweknowfrompastworksthataggrega8onscreatenewmeanings

•  Aggrega8ngallthephotosandhomemoviesoftheDigitalDiasporaishugelymoremeaningfulthanasinglephoto

•  OnetweetsaysveryliQle,butthousandsoftweetscanshowtrendsordepictapar8culareventorday


Butaggrega8ngitemsfromdisparatesourcescausessignificantproblems

•  Vastquan8tyofuser-contributedmaterial•  RightsIssues•  Noeasywaytocontrolforquality,fileformat,metadata(notevenanyconsistencyforanyofthese)-


EveryImageCollectorhasaDifferentApproach

•  Differentfile-namingconven8ons•  Differentfileformats•  Differentcompressionschemes•  Differentmetadata•  Storedindifferentarrangements/hierarchies•  Storedindifferentplaces(cellphone,personalharddisk,YouTube,Vimeo,Facebook,…)


THEPARTICULARPROBLEMOFMESSAGING


3/6/20

8

Born-DigitalMessaging•  Increasingly,importantrecordsetshavemovedtoemailsandtextmessaging

•  Muchoftheback-and-forthincollabora8onhappensthroughmessaging–  “let’strythisinstead”–  “increasebudgetcategoryXby$12K,anddecreasecategoryYbythesameamount”

–  “shouldweinsertthisphotobelowparagraph3?”–  “reviewthedocumentIsentyouthismorning”

•  IncreasinglythistypeofmessagingisonphoneservicesorTwiQer


Born-DigitalCorrespondence

•  Emailarchivingisbadenough– Anddiscussionsmightshi\backandforthbtwnworkandpersonalemailaccounts

•  Butwhathappenswhenemaildiscussionsswitchbackandforthtotextmessaging(likewhensomeoneisaskedtosendaphotothatresidesontheirphone)?

•  Andphonemessagingthreadscankeepjumpingaroundbtwnservices-


Phonemessagingthreadscankeepjumpingaroundbtwnservices

•  Somemessagessenttodistribu8ongroupwillbeansweredone-on-one(sotheoriginalques8on/contextisinacompletelydifferentthread)-

•  Mightjumpfromoneservicetoanother,thenbackagain– Standardtextmessagingnotworkingwithindepartmentstore,soswitchovertoWhatsAppondeptstoreWifi,thenbacktostandardtextmessagingwhenoutside


Howard’sTextMessaging


Howard’sTextMessaging



•  Somemessagessenttodistribu8ongroupwillbeansweredone-on-one(sotheoriginalques8on/contextisinacompletelydifferentthread)

•  Mightjumpfromoneservicetoanother,thenbackagain– Standardtextmessagingnotworkingwithindepartmentstore,soswitchovertoWhatsAppondeptstoreWifi,thenbacktostandardtextmessagingwhenoutside


3/6/20

9

TextMessagetoWhat’sApp



•  Somemessagessenttodistribu8ongroupwillbeansweredone-on-one(sotheoriginalques8on/contextisinacompletelydifferentthread)

•  Mightjumpfromoneservicetoanother,thenbackagain– Standardtextmessagingnotworkingwithinconfhall(ordeptstore),soswitchovertoWhatsApponconfcntr(deptstore)Wifi,thenbacktostandardtextmessagingwhenoutside


Phonethreadssome8messwitchtoEmail

•  Forexample,retrievingandsendingdocumentsorphotosthatareoncomputer,notonphone


IssueswithPrivacy-orientedMessagingApps--Signal

•  Privacygoesbeyondend-to-endencryp8on•  Nocloudstorageofmessagecopies•  Disables“screencapture”func8on•  AllmessagesstoredinencryptedDB(30characterkey?)


RUponreceipt,differentphonechannelsare

joinedtogetherthroughAlerts…


Butnotthreadedtogetherforlaterviewing

•  withinphonefiles•  withinmessagecaptureu8li8es•  btwnphoneandemail


3/6/20

10

THEPDACONFERENCES


PersonalDigitalArchiving2015hQp://personaldigitalarchiving.com/


PDA:WhoAQends&Presents

•  Ci8zenArchivists–  Peoplewhowanttostepinandrescuecontentinperil–  Peoplewholiketocreateso\ware/Apps/Guidelinestohelpothersfacingsimilarproblems

•  CommunityorEthnicgroupsandac8vistswan8ngtosavepor8onsoftheirheritage

•  Professionallibrarians&archivists(andtheirprogrammingsupportstaff)

•  Regularso\waredevelopers•  Researchers(bothacademicandcomputerindustry)


PDAGoals—Sharingknowledge

•  Whatworkedandwhatdidn’t;whatpartsturnedouttobemoredifficultthanan8cipated

•  Newanddifferenttypesofcontenttocollect•  Guidelines,procedures,workflows,methodologies

•  So\ware


PDAHistoryIni8allystartedbyInternetArchivewithco-sponsorshipfromNetherlandsSound&Vision,LC/NDIIPPandCNI•  2010InternetArchive•  2011InternetArchive•  2012InternetArchive•  2013UnivofMaryland•  2014IndianaStateLibrary&ISU•  2015NewYorkUniversity•  2016UniversityofMichigan•  2017StanfordUniversity•  2018UniversityofHouston•  2019UniversityofPiQsburgh


SamplePDATalks-Indianapolis2014hQp://visions.indstate.edu/pda2014/conference-program.html

•  TheSocialLifeofPersonalInforma8on,•  DefiningthePersonalinDigitalArchivingandCommuni8es•  PersonalArchivingasaGatewaytoDataLiteracy•  FindingRoots,Gems,andInspira8on:UnderstandingtheUl8mateUsesofDigital

Materials•  Opportuni8esandChallengesinAccessioningPersonalDigitalArchives•  PublicLibrariesandPersonalDigitalArchiving:OutreachLessonsLearned•  ClearasGlass:ControllingtheDataGeneratedbyWearableTechnology•  PersonalArchivingWithintheScholarlyWorkflow:ZoteroasConnectorfor

Collec8ngFacultyWork•  PreservingYourDigitalPhotosUsingFreeorLow-CostSo\ware•  Collabora8veDataManagementandProvenanceasCorePillarsofPersonalDigital

Cura8on•  FamilyArchiveasaNarra8veOrganiza8on•  DesigningaPersonal&FamilyArchiveforthe21stCentury


3/6/20

11

PDA2015hQp://blogs.loc.gov/digitalpreserva8on/2015/08/report-on-the-personal-digital-

archiving-2015-conference/


PDA2015


PDA2015Topics

•  PreservingDigitalPhotos•  DigitalPreserva8onofArt•  Crea8veuseofOpenSourcetools-•  CollegeLibrarysponsoredPersonalDigitalArchivingDays

•  CommunityEngagement•  VideoPreserva8on•  DigitalHumani8es&SocialScience•  Workshops-


Crea8veuseofOpenSourcetools•  JasonScoQ,“WhentheEmulatorsBrokeFree.”ScoQ’sbasicmessagewasthattheInternetArchive

hasmadesignificantadvanceswithemula8onofoldgamesonline.•  Jus8nMckinney,MarkSimonHaydnandAshleyBlewer,“

DoesBitTorrent+PrivateTrackers=TheNewFilmArchive?”(PDF).MckinneysaidBitTorrentisnotinherentlyevil,it’sjustatool,andpeople—especiallyprofessionalarchivistsandlibrarians—shouldbemoreopenmindedaboutusingBitTorrentasatested,effec8vetooltotransferlarge(legal)filesquicklyonline.

•  CalLeeandKamWoods,“BitstreamConfiden8al:Considera8onsandApproachestoCura8ngPoten8allySensi8vePersonalDatainCollec8ons”(PDF)

•  WendyHagenmaier,“PDAasanOpportunityforCollabora8veAdvocacyandMurderMysteryIntrigue”(PDF)and“AnExplora8onofthePoten8alImpactofWearableCompu8ngTechnologiesonDigitalArchivingandPreserva8on”(PDF)

•  CheyenneJansdaQer,“BuildingDigitalCollec8onsInOmekaForTheLayperson”•  AshleyBlewer,“Don’tknowaboutyou,butI’mfeelinglikeSHA-2!”•  PeterChan,“5.25inchfloppydisks”(PDF).


WorkshoponDo-It-YourselfPersonalDigitalArchiving


WorkshoponArchiveMa8ca


3/6/20

12

WorkshoponEmailArchiving


SOMEKEYCONCEPTSFROMINTERPARES,PDPTV,ACTIVISTARCHIVISTS Besser-Berkeley3/6/2020 68

KeyConcepts&ChallengesfrommypriorworkwithothertypesofDigitalContent

•  InterPARES—Ifwehopetopreserveelectronicrecords,archivistsneedtobeinvolvedearlyinthelife-cycleofthatrecord,longbeforetherecordentersthearchive

•  PreservingDigitalPublicTelevision—Pushingmetadatagatheringupstreamintotheproduc8oncycle-


Preserving Digital Public Television Workflow in Production Process-

•  Site Visits to productions •  Interview Production staff •  Diagrams of Workflow-

Besser-Berkeley 3/6/2020

Pushing Metadata Gathering Upstream: The Problem

TRADITIONALLY… •  Very little metadata required for

preservation accompanies an object to a repository.

•  Archives, libraries and other repositories must create (or re-create) most of the necessary metadata.

•  This requires many manual hours, and significant resources - both time and money.

IN THE DIGITAL WORLD… •  This doesn’t scale up. Repositories

will be unable to continue in this manner, as more metadata than ever is required.


But much of the necessary metadata has already been gathered during production

•  For each element/clip, production team usually notes source, date, place, people, and other descriptive info

•  But this is treated as internal information, and often various parts of the info are distributed among the personal notebooks of different production assistants

•  There is seldom a central location for this info, and the info is seldom turned over to the archive (which later tries to recreate much of it)


3/6/20

13

When the Archive tries to re-create this info, it is seldom successful

Producers know much more about the content of their productions than the archivists do. Archivists wanting accurate info must go back to the production staff (often years later) to start brainstoriming over the info

“Once the (television) program is finished, it is passed on to the archive or library for safe keeping. Librarians will catalog and classify the content, possibly using a proxy copy, and enter the resulting informative metadata in their database so they can retrieve it in the future. However, rarely if ever is the metadata from the rest of the process passed onto them, except, perhaps, for the title, tape number, and basic technical information about recording formats. It has to be re-created, with all the associated risk of errors and lack of accuracy--not to mention the work and time involved.”

- Cox, Tadic, and Mulder, Descriptive Metadata for Television (2006)


We need to find ways to push metadata access upstream

•  Digital requires even more metadata than Analog –  As the workflow becomes file-based, the need for robust and accurate

metadata will become critical. File relationships, video codecs, bit rates, and rights information must be explicit, accurate, and immediately accessible. This will require a much deeper level of metadata than is currently captured in tape-based archives.

–  We can’t continue to supply this metadata at ingest; that won’t scale •  Obtaining the necessary metadata at the end of production and

broadcast life cycle is not feasible. Metadata will need to be systematically gathered during the production lifecycle and submitted with the programs to the preservation repository.



Examined Potential Points of Metadata Capture


Examined Potential Points for Metadata Capture •  Much of the necessary metadata for preservation is already

generated by the production unit, but discarded after their internal use. This needs to be captured throughout the workflow.

•  “Those in the production unit are the creators and have first

hand knowledge of who, what, where, when, and why the content was created.” -- Mary Ide and Leah Weisse, WGBH Archivists.

Proposed Solutions…?

•  Preservation becoming a shared responsibility between content

creators, distributors, curators, and preservationists.

•  Partnerships are needed to come to unified solutions.

•  Preservationists seek reliable metadata back upstream in the production workflow...


WorldFocus •  Nightly news program begun Oct 2008 •  We began working with Workflows six months before program

began •  Had ability to engineer metadata gathering into the creation/

production process


3/6/20

14

Ac8vistArchivistshQp://ac8vist-archivists.org/(useWaybackMachine)

hQps://www.facebook.com/Ac8vistArchivists/

•  NYUMIAPstudentsandgradsoriginallyworkingonarchivingmediafromtheOccupymovement

•  Guidelinesbothac8vistcreatorsandarchives•  Developednewerlow-impactmethods


HowOccupymaterialresembleswhatwe’llbefacinginthefuture

•  Vastquan8tyofuser-contributedmaterial•  Noeasywaytocontrolforquality,fileformat,metadata–  noenforcingguidelinesaswithorganiza8onalrecords–  nosemi-consistencyasinasingleindividual’spersonalrecords

•  MuchofthematerialcanmosteasilybefoundonSocialNetworks

•  …weneedtofindsmartwaystoharvestmetadataandanalyzefiles,aswellastoinfluencebehaviorofpoten8alcontributors


Ac8vistArchivistWebsite


Ac8vistArchivistsProjects-

•  “WhyArchive”postcard&video•  7TipstoEnsureYourVideoIsUsableintheLongTerm

•  Studyofmetadatalossthroughuploadingtoservices•  BestPrac8cesforCreators/Collectors•  “Toolkit”forOccupyarchiving•  Coordina8ngdiscussionsamongvariousgroupsarchivingdifferentpartsofOccupy

•  Exploringmethodsforobscuringiden88es


IMPORTANTLESSONSFROMACTIVISTARCHIVISTS(&OTHERS)


LessonsLearnedforArchivists-

•  CommunicatewellwithyourfutureContributors•  DevelopCoopera8veRela8onships•  Makeiteasyforfuturecontributorstocreate“archival-friendly”works

•  ForCoopera8veProjects,allowforinstruc8onsnotbeingfollowed

•  FindsmartwaystodealwithScale•  HandlePrivacy&Securityresponsibly


3/6/20

15

CommunicatewellwithyourfutureContributors-

•  Learntospeaktheirlanguage•  Helpthemtorealizetheimportanceofarchiving


“WhyArchive”video


“WhyArchive”postcard•  ACCOUNTABILITY.Archivescollectevidencethatcanholdthoseinpower

accountable.•  SELF-DETERMINATION.Wedefineourownmovement.Weneedto

createandmaintainourownhistoricalrecord.•  SHARE.Archivesareapointofentrytoourmovement’srichrecord.We

canusethemtoensuretransparency,generatediscussion,andenabledirectac8on.

•  EDUCATE.Today’svideos,flyers,web-pages,andsignsarematerialfortomorrow’sskill-shares,classes,andmobiliza8ons.

•  CONTINUITY.Justaspastmovementsinspireus,newac8vistswilllearnfromtheexperienceswedocument.

•  RECORD&COLLECTwhat’shappeningaroundyou.•  PRESERVEtherecord.


DevelopCooperaWveRelaWonships-

•  TrytobeQerunderstandwhattheiraimsare;getinvolvedintheirac8vi8es

•  Developpartneringrela8onships


par8cipatedinSelf-helpac8vi8es:Skill-sharesforOccupiers


Self-helpac8vi8es:

OtherArchiveShare-DayandHackathonac8vi8es

•  BatchdownloadfromFLICKRwithselectedaQributes(#OWS,Crea8veCommons,EXIFmetadata,tagged-textmetadata)

•  Re-mixingofolderfootage•  Crea8ngavisual8meline•  Miningmaterialfordata(eg.numberofco-loca8onsofanofficer’snamewith“pepperspray”)


3/6/20

16

Makeiteasyforfuturecontributorstocreate“archival-friendly”works-

•  Low-hangingfruit•  Easyinstruc8onalmaterialthatappealstowhattheythinkisimportant

•  Instruc8onsforredundantmetadatacollec8on(tomakesurethatitiscaptured)


Low-Hangingfruit

•  TurnGPSon•  Developstrategiesforautoma8ngaprofileanduploads(ouridealApp)


7TipstoEnsureYourVideoIsUsableintheLongTerm

•  Collectdetailswhilefilming•  Keepyouroriginalrawfootage,unaltered•  Makeyourvideodiscoverable•  Contextualizeit•  Makeitverifiable•  Allowotherstocollectandarchive•  Orarchiveityourself


BestPrac8cesforContentCreators

•  Security–  Hiddencameralaws,par8es’consentlaws

•  CapturingContent–  Highestquality,setdateand8me-stamps,noteloca8on

•  OffloadingContent–  Rawfilesdirectlyontocomputer,keepmaterialorganized

•  UploadingContent–  Importanceoftagging,reviewofdiffservices

•  Deposi8ngwithanArchive•  Copyright


OccupyArchivingKit•  WhyArchive?•  Whatisan“archive”?HowdoIcreateanarchive?•  Crea8ngarchiving-friendlycontent•  HowcanIcollectmaterialsforthearchive?•  WhatshouldIsave?•  HowshouldIorganizemymaterials?HowdoIgetitintothearchive?•  Descrip8on/Metadata•  MediaManagement•  Storage&Preserva8on•  Access•  Exhibi8onandPresenta8on/Outreach•  RightsandRe-Use


WITNESS:Ac8vists’GuidetoArchivingVideo,YvonneNg

hQp://archiveguide.witness.org/


3/6/20

17

Collec8ng–ThinkTank


ThinkTankmetadataredundancies•  Guideliness8pulatethatpersonholdingrecordingdevicewillchecktoseethat8meanddatestamparecorrectbeforebeginningrecording(mostlydidn’thappen)

•  Guideliness8pulatethatascriptbereadverba8matthebeginningoftherecording,withdate,8me,proposedsubject,etc.(andwouldeventuallyallowvoice-recogni8onso\waretocreateappropriatemetadata).Scriptalsostatedthatallpar8cipantsagreedtoCrea8veCommonslicensingoftherecording

•  Guidelinesrequestedthatdate/8mebeembeddedintheappliedfile-name


FindsmartwaystodealwithScale-


Collec8ngStreamingMediaTheNYUMellonComposersProject

•  Tradi8onalWebCrawlers(Heritrix) follow links and capture most web content. But they are less successful with streaming video and dynamic content executed in the browser (like JavaScript).

•  NYU collaborated with IA to create a combined crawler and browser-


BROZZLER!

“browser” | “crawler” = BROZZLER

Logo: Noah Levitt 101Besser-Berkeley3/6/2020 102Besser-Berkeley3/6/2020

3/6/20

18

StreamcapturereliesonYoutube-dlhQps://rg3.github.io/youtube-dl/supportedsites.html


TamimentYouTubecollec8ng

•  TamimentArchivewasselec8velybrowsingthroughYouTubeOccupyvideos,tryingtochoosewhichonestokeep,thencatalogingthemwith–  Title,Creator,Crea8onDate,UploadDate,Descrip8on,URL,YoutubeUsername,License,Format,Codec,SourceMedia,OnInternetArchive,CCLicensetype

•  Buttheydidn’trealizethatthiswouldn’tscale!


March24,2012YouTubestats(just6monthsa\erstartofmovement)

•  “#Occupy”169,000•  “OccupyWallStreet”98,400•  “OccupyProtest”70,500•  “OccupyMovement”54,800•  “#OWS”50,300•  “OccupyOakland”13,400•  “Zuco�Park”6,690


Alterna8veapproachtoYouTubeSelec8onprocess

•  DevelopcategoriesofimportantYouTubevideos– Celebrityvisits,Internalworkings(library,kitchen,media),Confronta8onswithpolice,Labor,Housing,etc.

•  HaveOccupiersfillinanonlineformlis8ngthe5mostimportantvideosineachcategory


AdvantagesofYouTubeCollabora8veFilteringSelec8onProcess

•  Scalableandmanageable•  ConsistentwithOccupyideasofinclusivenessandofmanagingownstory

•  Tamimentcans8llchoosetobeselec8veincollec8ngonlyapor8onofwhatisvotedin,butthetotalsetforreviewisamanageablescale


HandlePrivacy&Securityresponsibly-


3/6/20

19

“Inaneffortto

todevelopmethodologyandanapproachandwillredacttheemailaddressesorotherpersonallyiden8fiableinforma8onfrombroadpublicpresenta8on.”Formoreseelibrary.ucla.edu/service/scl/rights-toolkit

UCLA Deed of Gift template


Promo8ngPrivacyProtec8onExamplefromWITNESS

•  “ObscuraCamisavisualprivacyappforphotoandvideo,thatgivesyouthepowertobeQerprotecttheiden8tyofthosecapturesinyourphotos,beforeyoupostthemonline”

•  DevelopedbyGuardianProjectinconjunc8onw/HumanRightsgroupWITNESS-


ObscuraCam


DiscussissuesaroundcommercialserviceswithCreators/Recorders-

•  DisappearanceofembeddedmetadatafromYouTube&Vimeo

•  MoregeneralRightsissues•  GivearchivestheIPrighttodownload


Studyofmetadatalossthroughuploadingtoservices


Evenwithgoodrecords,RightsIssuesremain1999WTOSeaQleProtest20thanniversaryVideoPreserva8on


3/6/20

20





YouTubeUserAgreement

•  5B“YoushallnotdownloadanyContentunlessyouseea‘download’orsimilarlinkdisplayedbyYouTubeontheServiceforthatContent.”


Crea8veCommonsGuidance•  Crea8veCommonsletsyoumix-and-matchfourdifferent

condi8ons:–  AQribu8on:Youletotherscopy,re-useanddistributeyourvideo,butthey

mustcredityou.–  Share-Alike:Youletotherscopy,re-useanddistributeyourvideo,onlyifthey

dothesamewiththeworktheycreate.–  Non-Commercial:Youletotherscopy,re-useanddistributeyourvideofor

non-commercialpurposesonly.–  NoDeriva8veWorks:Youletotherscopyanddistributeyourvideo,butnotto

createnewworksusingit.•  Youcanusethesecondi8onsindifferentcombina8onstoshareyourwork

inacontrolledway.Crea8veCommonslicensesarelegaltoolsthatdependonpre-exis8ngcopyrightlaws.HavingaCrea8veCommonslicenseonyourworkmaygiveyoulegalrecourse,butitmaynotactuallypreventpeoplefromdownloadingandre-usingyourvideoillegally.


MarkingCrea8veCommonslicenses•  ThereareafewwaystomarkyourvideowithaCrea8veCommons

license.OnewayistoincludeaCrea8veCommons“bumper”ortextcardinyourvideo.Crea8veCommonshascreatedsomewithgraphicsthatyoucandownloadfromtheirwebsite.Thismethodisusefulifyourvideoisgoingtobesharedoffline(e.g.onDVD,livescreenings),asthelicenseinforma8onisaQachedtothevideoitself.

•  AnotherwaytomarkyourvideowithaCrea8veCommonslicenseistopublishyourvideoonplarormsthatareCrea8veCommons-enabled,suchasYouTube,Vimeo,orInternetArchive.Theseplarormsallowyoutoeasilyselectalicenseduringtheuploadprocess.Thismethodisusefulbecausethelicenseismachine-readable.Asearchengine,forexample,candetectthelicense.


TipsforArchivistsonOutreachtoCommuni8es

•  Buildtrust•  Speakintheirlanguage(notarchive-speak)•  Iden8fywaysyoucanmeetneedstheyalreadyperceive

•  Approachprojectsascollabora8onwheneverpossible

•  Don’tonlyfocusoncontentandmetadata,butalsorightsthatcanbeanimpedimenttopreserva8on


3/6/20

21

OtherProjectstomonitor-

•  Documen8ngtheNowhQps://www.docnow.io/

•  Interna8onalDigitalEphemeraProjecthQp://idep.library.ucla.edu/

•  ePADDhQps://library.stanford.edu/projects/epadd


Documen8ngtheNowhQps://www.docnow.io/

•  Documen8ngtheNowrespondstothepublic'suseofsocialmediaforchroniclinghistoricallysignificanteventsaswellasdemandfromscholars,students,andarchivists,amongothers,seekingauser-friendlymeansofcollec8ngandpreservingthistypeofdigitalcontent.Documen8ngtheNowhasastrongcommitmenttopriori8zingethicalprac8ceswhenworkingwithsocialmediacontent,especiallyintermsofcollec8onandlong-termpreserva8on.ThiscommitmentextendstoTwiQer'sno8onofhonoringuserintentandtherightsofcontentcreators.Theprojectisacollabora8veeffortbetweenShi\Design,Inc.,theUniversityofMaryland,andtheUniversityofVirginia.


Interna8onalDigitalEphemeraProjecthQps://idep.library.ucla.edu/


ePADDhQps://library.stanford.edu/projects/epadd

•  “ePADDisfreeandopensourceso\waredevelopedbyStanfordUniversity'sSpecialCollec8ons&UniversityArchivesthatsupportstheappraisal,processing,preserva8on,discovery,anddeliveryofhistoricalemailarchives.ePADDincorporatestechniquesfromcomputerscienceandcomputa8onallinguis8cs,includingmachinelearning,naturallanguageprocessing,andnameden8tyrecogni8ontohelpusersaccessandsearchemailcollec8onsofhistoricalandculturalvalue.”


RemainingAppliedResearchQues8ons

•  HowdodealwithScalewiththetsunamiofborn-digitalcontent(appraisal,descrip8on,discovery,workflow,…)

•  Howtoavoidourcollec8onscon8nuingtoreflecttheworldoftherich,well-known,andpowerful

•  BeQermethodsforcollec8ngsocialmediaandthreadedphonemessages

•  Tensionbtwnpreserva8on&privacy(&handlingmassiveamountsofredac8on)

•  ImprovingpublicpolicyandTOSinareaslikeIP,privacy,andwhathappensupondeath

Besser-Berkeley3/6/2020 125 Besser-Berkeley3/6/2020

•  hQp://besser.tsoa.nyu.edu/howard/Talks•  hQp://idep.library.ucla.edu/•  hQp://ac8vist-archivists.org/(useWayback)•  hQps://www.facebook.com/Ac8vistArchivists/•  hQps://archive.org/details/personaldigitalarchiving•  hQp://www.docnow.io/•  hQp://blogs.loc.gov/digitalpreserva8on/2015/08/report-on-the-personal-digital-

archiving-2015-conference/

ArchivingtheNon-Organiza8onalBorn-Digital:TheChallengesPosedbyMaterialfromIndividuals,Communi8es,&Events

126

3/6/20

22

ArchivingtheNon-Organiza8onalBorn-Digital:TheChallengesPosedbyMaterialfromIndividuals,

Communi8es,&Events


personal born-digital in what environments do we...

Documents