Download - Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library
![Page 1: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/1.jpg)
Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library
Glen Newton1,2, Alison Callahan1, Michel Dumontier2
1National Research Council Canada, 2Carleton University
Second Workshop on Very Large Digital Libraries (VLDL) 2009
at ECDL 2009
Oct 2 2009 Corfu, Greece
![Page 2: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/2.jpg)
Outline
• Maps of Science• Background• Research Interests• Research Goals• Process• Scalability issues• Environment• Results• Conclusions• Future Work
![Page 3: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/3.jpg)
From Bollen et al 2009 PLOS1
![Page 4: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/4.jpg)
From Leydesdorff & Rafols 2006From Leydesdorff & Rafols 2006
![Page 5: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/5.jpg)
From Leydesdorff & Rafols 2006
![Page 6: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/6.jpg)
Background
• Canada Institute of Science and Technical Information (CISTI) == Canadian national science library
• ~3000 active researchers at NRC• Large full text collection of ~8.4m full-text + metadata articles, in
science, technology, medicine (STM)• 4100 journal titles• ~1995 to 2009
![Page 7: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/7.jpg)
Research Interests
• Domain-specific discovery• Improved discovery in STM domains through results visualization
and contextualization, browse/explore/refine• Results set visualization: “mapping”
![Page 8: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/8.jpg)
Research Goals
• Find way to extract journal (& article) semantic vector space• Latent Semantic Analysis (LSA) works for small/medium sized
corpora, does not scale to large scale of items and/or terms• New alternative: Semantic Vectors (SV): uses random vectors &
avoids expensive singular value decomposition (SVD)• Can SV scale & generate sensible semantic vector space of
journals on corpus of this size?• Can the visualization produced be useful for results query
visualization, refinement, discovery?
![Page 9: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/9.jpg)
Corpus
• Licensed journal articles from STM publishers: Elsevier, Springer, etc
• ~4100 journal titles, classified into 23 categories (by librarians)• ~8.4m journal articles• Selection of articles/journals:
– Only those with authors, abstract (no notices, obituaries, etc)– Only English language articles– Only journals with >50 articles in corpus– Resulting corpus: 5,733,721 articles from 2231 journals – Categories overlapping: 1.53 categories per journal
![Page 10: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/10.jpg)
Category # Journals per category
Agriculture & Biological Sciences 358
Arts and Humanities 70
Biochemistry, Genetics and Molecular Biology 240
Business, Management and Accounting 106
Chemical Engineering 126
Chemistry 226
Civil Engineering 64
Computer Science 218
Decision Science 50
Earth and Planetary Science 146
Economics, Econometrics and Finance 112
![Page 11: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/11.jpg)
Category # Journals per category
Energy and Power 73
Engineering and Technology 328
Environmental Science 138
Immunology and Microbiology 104
Materials Science 160
Mathematics 205
Medicine 671
Neuroscience 103
Pharmacology, Toxicology and Pharmaceutics
73
Physics and Astronomy 210
Psychology 126
Social Science 222
![Page 12: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/12.jpg)
Process
• Index full-text (only) with Lucene 2.4, aggressive stopword list, Porter stemming using LuSql tool
• Build Semantic Vectors (v1.18, parallelized) index from Lucene index, with 512 semantic dimensions
• Find item x item distance matrix from SV index of 512-dimensional vectors
• Using R, use multidimensional scaling (MDS) to reduce from 512-D to 2-D
![Page 13: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/13.jpg)
Scalability Issues
• #items, #unique terms– #unique terms: SV easily handles very well– #items: SV handles fairly well – #items: impacts size of distance matrix (#items x #items) – R cannot handle huge article distance matrix in MDS (i.e.
millions of articles vs. thousands of journals)
• Instead of using articles for items, use journals for items• Make single large full-text document from concatenation of all
articles of particular journal & index these
![Page 14: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/14.jpg)
Environment
• Dell PowerEdge 1955 Blade server, 2 x dual-core Xeon 5050 processors with 2x2MB cache, 3.0 Ghz 64bit, 32GB RAM, attached to a Dell EMC AX150 storage arrays via SilkWorm 200E Series 16-Port Capable 4Gb Fabric Switch.
• Operating system: Linux openSUSE 10.2 (64-bit X86-64), kernel 2.6.18.8-0.10-default \#1 SMP
• Java version 1.6.0.07 (build 1.6.0 07-b06) Java HotSpot 64-BitServer VM (build 10.0-b23, mixed mode).
• Processing 1.0 (processing.org)
![Page 15: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/15.jpg)
Results: Scalability
• Corpus: ~600GB full-text• Lucene index: 43GB
– LuSql: 13 hours 51 minutes to produce
• SV index: 58 minutes, 885 MB, 21.6m terms– Distance matrix: 6 minutes
![Page 16: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/16.jpg)
Results: Visualization
• Using Processing environment, built simple validation/visualization tool
![Page 17: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/17.jpg)
![Page 18: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/18.jpg)
![Page 19: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/19.jpg)
Harder sciences and engineering categories
![Page 20: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/20.jpg)
Chemistry
![Page 21: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/21.jpg)
Material Science
![Page 22: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/22.jpg)
Physics andAstronomy
![Page 23: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/23.jpg)
Engineering and Technology
![Page 24: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/24.jpg)
Mathematics
![Page 25: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/25.jpg)
Computer Science
![Page 26: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/26.jpg)
Civil Engineering
![Page 27: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/27.jpg)
Chemical Engineering
![Page 28: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/28.jpg)
Agriculture and biomedical categories
![Page 29: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/29.jpg)
Agriculture and Biological Sciences
![Page 30: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/30.jpg)
Biochemistry, Geneticsand Molecular Biology
![Page 31: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/31.jpg)
Immunology andMicrobiology
![Page 32: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/32.jpg)
Pharmacology
![Page 33: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/33.jpg)
Neuroscience
![Page 34: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/34.jpg)
MedicineMedicine
![Page 35: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/35.jpg)
Psychology
![Page 36: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/36.jpg)
Interdisciplinary and non-science categories
![Page 37: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/37.jpg)
Environmental Science
![Page 38: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/38.jpg)
Earth andPlanetary Science
![Page 39: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/39.jpg)
Energy and Power
![Page 40: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/40.jpg)
Decision Science
![Page 41: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/41.jpg)
Economics,EconometricsAnd Finance
![Page 42: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/42.jpg)
Social Sciences
![Page 43: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/43.jpg)
Business, Managementand Accounting
![Page 44: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/44.jpg)
Arts and Humanities
![Page 45: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/45.jpg)
Examination of outliers, extrema and cataloging errors
![Page 46: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/46.jpg)
Environmental Science
Ecotoxicology and Environmental Safety
Corporate EnvironmentalStrategy
Organic Geochemistry
![Page 47: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/47.jpg)
MedicineMedicine
Journal of X-Ray Science and Technology
Journal of Biomolecular NMR
![Page 48: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/48.jpg)
MedicineMedicine
Annales Henri Poincare
Colloidal and Polymer Science
![Page 49: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/49.jpg)
Medicine
Medicine
French language Medical & Psychology Journals
![Page 50: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/50.jpg)
Mathematics
Journal ofMedicalUltrasonics
Bulletin ofMathematical Biology
![Page 51: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/51.jpg)
Conclusions
• Reasonable mapping results• Full-text only (no citations, metadata) gives good results• Scalable to significant size
![Page 52: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/52.jpg)
Future Work
• Proper precision and recall evaluation using same corpus• Validate with NetNews-20 collection for P & R• Evaluate non-metric MDS• Project articles onto semantic journal space & build interactive
discovery interface & evaluate– Index journal 'documents' and journal articles– SV on all– Distance matrix only on journals– Do MDS– Use eigenvectors to transform N-d article vector to 2-D
• Explore 3-D interface (MDS N-d → 3D)
![Page 53: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/53.jpg)
Acknowledgements
• Greg Kresko, Andre Vellino, Jeff Demaine @ NRC-CISTI
![Page 54: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/54.jpg)
Demo
• Link to project demo page
![Page 55: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/55.jpg)
License
Creative Commons Attribution-Noncommercial-No Derivative Works 2.5 Canada License
![Page 56: Semantic Journal Mapping for Search Visualization in a Large Scale Article Digital Library](https://reader034.vdocuments.site/reader034/viewer/2022051411/5462857eb4af9f671c8b4843/html5/thumbnails/56.jpg)