ACL Reviewer Guidelines

The following guidelines can be used by conferences and journals who adopt the ACL Policies for Submission, Review and Citation.

For further motivation and discussion, we refer to the report of the working group appointed by the ACL Executive Committee: File:ACL Guidelines for Submission, Review and Citation.pdf.
For corresponding author guidelines, see: ACL Author Guidelines.

Preserving Double Blind Review

Reviewing may involve an online search for related work. However, do not set out to uncover the identity of the authors and try not to let any suspicions regarding author identity affect your judgement. To reduce the risk of bias, it is a good idea to read the paper and draft an initial review before you carry out any online searches that risk discovering authorship. You should be especially careful if an area chair has informed you that a non-anonymized version of the paper is available online.
If you do become aware of the authors' identity (by whatever means), let the area chairs (but not your fellow reviewers) know.
If you come across a preprint or paper that has a substantial text overlap with the submission, report this to the area chairs even if the existence of a preprint has been declared, since you do not know whether the authors of the discovered paper are the authors of the submission under review, and so the possibility of plagiarism needs to be considered.

Evaluate papers based on your knowledge of the subject, regardless of where you gained this knowledge. If you are aware of relevant publicly available research that has not been cited in the paper you are reviewing, you should bring it to the attention of the authors, irrespective of whether it is described in a preprint or a refereed publication. However, if the work appears only in a preprint, especially one that is recent and/or not widely cited, you should in general give authors the benefit of the doubt and not penalize them for the missing citation.
Papers should include appropriate discussion of related work, potentially including both preprints and refereed publications. However, it is not reasonable to expect a time-consuming empirical comparison with work that has appeared less than 3 months before the submission deadline. In this situation, the two papers are considered to be contemporaneous. Contemporaneous papers may affect your perspective on the quality of a submission, but should be disregarded when evaluating the novelty of the submission, unless there is evidence that they have influenced the submission.