digital libraries, archives, and large data sets alexa t. mccray national library of medicine...
Post on 29-Jan-2016
217 Views
Preview:
TRANSCRIPT
Digital Libraries, Archives, and Digital Libraries, Archives, and Large Data Sets Large Data Sets
Alexa T. McCray
National Library of Medicine
Bethesda, Maryland USA
mccray@nlm.nih.gov
WHOI, June 3, 2004
What is a digital library?What is a digital library?
“… an electronic information access system that offers the user a coherent view of an organized, selected, and managed body of information.” (Lynch, 1995)
An organization that provides the resources “… to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use…” (Waters, 1998)
Data Creation
Data Capture, Management, and
Preservation
Data Access
Conceptual Model of aDigital Library
Content Creators
Users
Long Term Preservation & Long Term Preservation & ArchivingArchiving
•OAIS (Open Archival Information System) standard
-Developed by NASA for long term preservation, archiving, data management, and access • Both digital and physical archives
•Address impacts of changing technology
-New media and data formats
-Changing user community
OAISOAIS
•Framework for data management
•Functional model for
-Preservation planning
-Data management
-Archival storage
-Persistent access
Digital Libraries InitiativeDigital Libraries Initiative
•Research initiative lead by the National Science Foundation in collaboration with a number of other Federal agencies
•Research goal is to investigate improved methods for creating, managing and accessing large information resources and repositories
Research FociResearch Foci
•Content and Collections
•Systems-centered digital library research
•Human-centered digital library research
•Testbeds and Applications
Content and CollectionsContent and Collections
•Data capture, representation, preservation
•Metadata
•Domain specific information objects
• Intellectual property rights
•New economic and business models for digital libraries
Systems-centered Systems-centered ResearchResearch
•Open, networked architectures
•System scalability
• Intelligent agents
•Systems evaluation and performance studies
•Data compression
•Authentication
Human-centered Human-centered ResearchResearch
• Information discovery and retrieval methods
• Intelligent user interfaces
• Information visualization
•User and usability studies
•Social implications of digital libraries
Testbeds and Testbeds and ApplicationsApplications
•Specialized tools for e.g.,
-Document mark up
-Metadata encoding
•Specialized applications for specific domains
•Allow development of new methods for knowledge discovery and data mining
The Vocabulary ProblemThe Vocabulary Problem
•Same string, different meaning
•Different string, same meaning
•Different string, similar meaning
-Unrecognized relationship
• Implicit conventions
• Implicit hierarchies
-Variety of relationships
Unified Medical Language Unified Medical Language (UMLS) System(UMLS) System
•Long term National Library of Medicine project
•Problem the UMLS is attempting to solve:
-Provide integrated access to biomedical information in disparate biomedical information systems• Bibliographic, factual databases, decision
support systems, knowledge-based systems
UMLS Knowledge SourcesUMLS Knowledge Sources
•Metathesaurus
-Large number of biomedical concepts
•SPECIALIST Lexicon
-General English and biomedical lexical items, tools for recognizing linguistic variation
•Semantic Network
-Conceptual framework for the UMLS
Metathesaurus Metathesaurus
•Metathesaurus
-Over one million concepts; 90 families of vocabularies
-Broad coverage of the vocabulary used in the biomedical sciences• Basic science research
• Clinical medicine
• Health services
Broad Coverage of Broad Coverage of BiomedicineBiomedicine
• Several perspectives
- clinical terms (SNOMED)
- information sciences (MeSH, CRISP)
- administrative terminologies (ICD-CM, CPT-4)
• Specialized vocabularies
- genomics (Gene Ontology, NCBI organism taxonomy)
- medical devices (UMD)
- anatomy (UWDA, Neuronames)
From the Vocabularies to the From the Vocabularies to the MetathesaurusMetathesaurus
•Vocabularies
- terms
-hierarchies
•Metathesaurus
-organizes terms
-organizes concepts
-Relates concepts to other concepts
•Metathesaurus = Thesaurus of Thesauri
Common UMLS RepresentationCommon UMLS Representation
•One concept, multiple terms and strings
- renal cell carcinoma• CUI: C0007134, LUI: L0007134, SUI:S0425056
- renal cell carcinomas• CUI: C0007134, LUI: L0007134, SUI:S0081526
-hypernephroma• CUI: C0007134, LUI: L0020489, SUI:S0420320
-Grawitz tumor• CUI: C0007134, LUI: L0018219, SUI:S0375417
Lexical ToolsLexical Tools
•Manage lexical variation
-Perform lexical transformations• Generate inflectional variants, normalized forms
•Depend the SPECIALIST lexicon
•Used for preliminary algorithmic mapping as new vocabularies are added to the Metathesaurus
Digital Library Case Study: Digital Library Case Study: ClinicalTrials.govClinicalTrials.gov
•Centralized system at NLM
-Content provided by individual data providers, both federal and from the private sector
•Standard set of data elements in XML (eXtensible Markup Language) format
-Summary; recruitment information; eligibility criteria; study design; intervention being studied, location and contact information
ClinicalTrials.gov
ClinicalTrials.gov
System Architecture: ClinicalTrials.gov
Digital Library Case Study:Digital Library Case Study:Profiles in ScienceProfiles in Science
•Large scale digital conversion project
•Archival collections of eminent biomedical scientists of the twentieth century
-Books, journal volumes, pamphlets, diaries, letters, manuscripts, photographs
•Materials in a variety of formats
-Text, audio, still images, video
•Testbed for experiments in digital preservation
Profiles in Science
Profiles in Science
Metadata-drivenMetadata-drivenDocument ConversionDocument Conversion
• Interpret metadata in broadest sense
•Use metadata to drive the entire system
•Metadata record is the basic unit in the system, managing the
-Digitization process
-Display and organization of the data
-Network-based resource discovery
-Archiving and Preservation
Metadata: Framework forMetadata: Framework forCollection ManagementCollection Management
•Metadata entry system manages all aspects of digitization process
-Unique identifiers bind digital master files, Web-derivatives, and metadata records
-Enforces quality control (pull-down menus, validation, error messages)
-Reports that manage workflow
-Security measures
Metadata: Display and Metadata: Display and Organization of the DataOrganization of the Data
•Series of programs generate Web pages from metadata database
- Include consistency checking, validation
•Programs generate alternative views
-alphabetical, chronological, resource type, content area
Metadata: Networked-based Metadata: Networked-based Resource DiscoveryResource Discovery
•“Dublin Core” metadata elements derived from metadata entry system
- simplicity
- semantic interoperability
- international consensus
Metadata: Ensuring Metadata: Ensuring Preservation and PersistencePreservation and Persistence
•Archiving responsibility
•Permanence rating
•Preservation actions
•History of origin
Broad Categories of Broad Categories of Metadata ElementsMetadata Elements
•Content specific
•Medium specific
•Process specific
•Storage information
•Physical characteristics
•Preservation/provenance information
System Architecture: Profiles in Science
Digital Resources at the Digital Resources at the National Library of MedicineNational Library of Medicine
•Four levels of permanence
-Permanent: unchanging content, e.g., Profiles in Science scanned document
-Permanent: stable content, e.g., MEDLINE record
-Permanent: dynamic content, e.g., NLM home page
-Permanence not guaranteed, e.g., fact sheets
Preservation of Preservation of Digital InformationDigital Information
•“The conclusion reached by the impressive group of 21 experts was alarming – there is, at present no way to guarantee the preservation of digital information.” (Rothenberg, 1999)
•“Technological obsolescence [is] the greatest threat to digital collections.” (Kenney & Rieger, 2000)
Preservation Research at the Preservation Research at the National Library of MedicineNational Library of Medicine
• Image Migration Framework
-Prototype for image conversion, analysis, and preservation
-Associated preservation metadata
-Current experiments converting from one image format to another
TIFF to PNG to TIFF
Concluding RemarksConcluding Remarks
•Digital library data management
-Requires technical decisions• Adherence to standards, planning for change
- Involves social issues• Sharing of data and knowledge
• Open access to information
- Implies promises to our users• Integrity, currency, and persistence of data
top related