Web Tools for Peer Reviewers… and Everyone

Richard Akerman

ICSTI Public Conference 2007

June 21, 2007

The Motivation for Science

• If Mr. Cavor made [the gravity-blocking substance], it would go down to posterity as Cavorite or Cavorine, and he would be made an F.R.S., and his portrait given away as a scientific worthy with Nature, and things like that. And that was all he saw!

- H.G. Wells, The First Men in the Moon (1901)

Peer Review also remains basically unchanged

• And has enduring value

• Some enhancement through– automation of review process– citation linking– plagiarism detection

• Perhaps increased recognition or openness– Publish list of reviewers annually– Publish reviewer comments after some embargo period

Overview

• Exploring the problem space

• New metrics

• Certification (and reward) challenges

• Some example web tools

Two Problem Spaces and Two Directions

• Find existing– Articles, experts, clusters, objects (article discovery)

• Discover new– Ideas, concepts, relationships (knowledge discovery)

• Retrospective– The existing body of scientific knowledge, the citation web,

known author relationships

• Prospective (real-time)– New and unconnected or weakly-connected work and authors

(Ramanujan)

Retrospective Finding

• Start with some nucleus– idea, keywords, article, author, (collection), your history

• Find a network of related objects

• How? Recommender services.– Use metrics/features and find closeness/similarity in some

feature space

Features/Quality Metrics

• Networks of– Citations– Authors (reputation)

• Groups and Projects

– Certification (journal publication)

• Position in these networks

http://www.flickr.com/photos/darkmatterpaintball/251552464/

Forensics & Major Miner Problems

• Examining the corpse / dead trees

• Fossilized trees -> coal -> mining!

• We need to recognize this knowledge metaphor

• We’re mining material that is already refined

http://www.flickr.com/photos/mekin/399220499/

Thinking about mining

• Currently most demonstrations are only on abstracts due to limited full-text availability– So we are mining a refinement of the refinement– Rights are a major problem– Lack of mining rights is one factor that is driving open access

• Also knowledge discovery leads into difficult areas of machine reasoning

• What does it mean to move beyond mining?

Talking a walk in the the forest

• How do we find the new Mr. Cavor?

• Exploring the World Wide Web (including repositories) as opposed to the World Wide Literature

• Blogs, wikis, videos, datasets– And pre-prints

• The retrospective metrics we could use are either weaker or non-existent

http://www.flickr.com/photos/jzakariya/191481917/

What New Metrics Can We Use?

• Derived reputation (e.g. Nature blogs)• Page rank and similar citation-like connections

– Bookmarks• Intentional rank• Comments• Page hits / usage / viewing time• Derived quality

– (important person or group) favours this object– (many many people) favour this object

• Lots of privacy issues

Challenges: Download/Views

• The versions challenge – many copies of the article in many places– http://www.lse.ac.uk/library/versions/– eprintweb is linking pre-prints to published versions

• Search engines (spidering)

Challenges for Certification

• What should be certified?– Articles in repositories– Blog entries– Versioned Wiki entries– Data sets– Videos– Annotated mashups

• How is certification asserted?– Some sort of digital signature

• What sort of scientific rewards can we provide for the new Mr. Cavor?

Example: Eigenfactor

Example: eprintweb.org

astro-ph/0607051 (July 2006)Bulk viscosity of Mixed nucleon-hyperon-quark Matter in Neutron starsNa-Na Pan, Xiao-Ping Zheng and Jia-Rong LiReceived. 04 July 2006 Last updated. 04 July 2006Abstract. We calculate the coefficient of bulk viscosity …Journal-ref. Mon.Not.Roy.Astron.Soc. 371 (2006) 1359Published Article doi: 10.1111/j.1365-2966.2006.10759.x

Example: ScienceBlogs

Example: Postgenomic

Example: ChemRank

Thank You

•Richard dot Akerman at NRC dot CA

•Supplementary bookmarks athttp://www.connotea.org/user/scilib/tag/icsti2007akerman

•© 2007 Government of Canada and licensed in the Creative Commonshttp://creativecommons.org/licenses/by-nc-sa/2.5/