The Reviewing ProcessMarie desJardins

AAAI-13 PanelConference Reviewing Best Practices

Conference Reviewing Program committees

Selection process Different models/roles:

Senior vs. area chair vs. regular members Paper assignments

Keyword-based Similarity-based (compare to reviewer’s published

papers) Self-selection (bidding)

Discussion and author response Decisions

7/17/132Review Process and Quality

Purpose of a Review Evaluate the paper’s scientific merit

Check the validity of the paper’s claims and evidence Judge the paper’s relevance and significance

Provide constructive feedback to the author

7/17/133Review Process and Quality

AAAI-13 Review Criteria Goal: identify “interesting but flawed” papers

and make sure they are reviewed fairly Different ways in which a paper can make a contribution Avoid a conference full of “technically acceptable but

uninteresting and incremental” papers Quality criteria:

Technical quality Experimental analysis Formal analysis Clarity/presentation

Innovation criteria: Novelty/innovation of question / solution

Impact criteria: Breadth of interest Potential for impact

7/17/13Review Process and Quality 4

Good Reviews Are...

Polite Fair Concise Clear Constructive Specific Well documented Represent the scientific community

7/17/135Review Process and Quality