Long-Term Knowledge Retention

Joshua LubellManufacturing Systems Integration Division, NIST

FIRM’s Forum at FOSEMarch 20, 2007

The problem

• Too much digital data!– It takes about 15 minutes for the world to churn out new digital

information equivalent to the entire collection in US Library of Congress

• Proprietary file formats– Expected lifetime of typical manufacturing software application

only 3 years• Short-lived Computing hardware and software

– Expected lifetime of today’s storage/retrieval technologies only 10 years

• Products often outlive computer software/hardware by an order of magnitude– Aircraft can last 50 years or more– Healthcare records should be preserved through the patient’s

lifetime, and perhaps beyond

Data standards

• Necessary to avoid being locked into a vendor format or application that could disappear in the near future

• Likely to be more stable than proprietary tools/formats

• But data standards are only part of the solution– Information is more than just data!

Information = Data + Interpretation

DataDataObjectObject

RepresentationRepresentationInformationInformation(metadata)(metadata)

InformationInformationObjectObject

from Reference Model for an Open Archival Information System (ISO 14721:2003)

An information package

ContentInformation

PreservationDescriptionInformation

InformationInformation ObjectsObjects

•ReferenceReference•ProvenanceProvenance•ContextContext•FixityFixity

Sub-categoriesSub-categories

Tools for tackling LTKR

• Standards for representing digital artifacts– STEP – ISO 10303 (product data)– XML (documents)– Graphics, audio, video, multimedia standards– Scientific modeling standards

• Standards for representing preservation information– Ontology languages– Packaging standards (METS, XFDU)

• Digital format registries (UK Archives, Harvard, Univ. of Maryland)

March 2006 LTKR workshop

• Diverse group of 35 met at NIST– Industry, academia, government equally represented

• Immediate goal: identify challenges, research, and implementation issues in digital preservation of information– Emphasis on design and manufacturing

• Next step: develop roadmap identifying areas of investigation and experimental testbeds for archival of design and manufacturing information

Observations

• LTKR seen by many as a process– Apply archiving methodology (e.g. OAIS

reference model) to collection of digital artifacts

– “Repository-centric”

• Alternative “document-centric” view– Preservation and authenticity paramount– Archival process and data representation

secondary

More observations

• Barriers to archiving– Lack of understanding, institutional support– Each scenario has its own unique requirements– Lack of formal methods and standards

• Need a way to measure the quality of an archiving process• Library of Congress digital format sustainability criteria a

good starting point

• Recommendations– Create tools, methods for capturing business and

manufacturing process workflows– Collect and preserve case studies of archiving

successes and failures– Develop metrology for digital archiving

Upcoming workshop

• Long Term Sustainment of Digital Information: Putting the Pieces Together– April 24-25, 2007 at NIST in Gaithersburg, Maryland– Part of Interoperability Week @ NIST– http://digitalpreservation.wikispaces.com/LTKR+2007+Call+for+participation

• Questions we will attempt to answer– How can you predict the future effectiveness of a digital

preservation solution? – What combination of technologies is optimal for achieving

success at a reasonable cost?

• Registration deadline: April 9

The time is now

• Industry is feeling the pain– From a major aerospace company Vice President: Lack of

archiving support could derail our efforts to move from a drawing-centric to a model-centric business model

– Federal regulators recently fined Morgan Stanley $15M for failing to produce emails sought in investigations

• Government recognizes the need– “Maintenance of and access to long-lived science and

engineering data collections and Federal records” a funding priority according to Networking and Information Technology Research and Development (NITRD) supplement to the President’s FY2007 budget

• The technologies we need are becoming increasingly available

Links/Contacts

• Interoperability Week @ NIST– April 23-25, 2007– http://www.mel.nist.gov/div826/msid/sima/interopweek

• March 2006 LTKR workshop– Report: http://www.nist.gov/msidlibrary/doc/NISTIR_7386.pdf– Website: http://edge.cs.drexel.edu/LTKR/

• Me– Email: lubell@nist.gov– Phone: (301) 975-3563