Difference between revisions of "ACL Reviewer Guidelines"
Jump to navigation
Jump to search
| Line 6: | Line 6: | ||
* Reviewing may involve an online search for related work. However, do not set out to uncover the identity of the authors and try not to let any suspicions regarding author identity affect your judgement. To reduce the risk of bias, it is a good idea to read the paper and draft an initial review before you carry out any online searches that risk discovering authorship. You should be especially careful if an area chair has informed you that a non-anonymized version of the paper is available online. | * Reviewing may involve an online search for related work. However, do not set out to uncover the identity of the authors and try not to let any suspicions regarding author identity affect your judgement. To reduce the risk of bias, it is a good idea to read the paper and draft an initial review before you carry out any online searches that risk discovering authorship. You should be especially careful if an area chair has informed you that a non-anonymized version of the paper is available online. | ||
| − | * If you do become aware of the | + | * If you do become aware of the authors' identity (by whatever means), let the area chairs (but not your fellow reviewers) know. |
* If you come across a preprint or paper that has a substantial text overlap with the submission, report this to the area chairs even if the existence of a preprint has been declared, since you do not know whether the authors of the discovered paper are the authors of the submission under review, and so the possibility of plagiarism needs to be considered. | * If you come across a preprint or paper that has a substantial text overlap with the submission, report this to the area chairs even if the existence of a preprint has been declared, since you do not know whether the authors of the discovered paper are the authors of the submission under review, and so the possibility of plagiarism needs to be considered. | ||
| + | |||
| + | == Citation and Comparison == | ||
| + | |||
| + | * Evaluate papers based on your knowledge of the subject, regardless of where you gained this knowledge. If you are aware of relevant publicly available research that has not been cited in the paper you are reviewing, you should bring it to the attention of the authors, irrespective of whether it is described in a preprint or a refereed publication. However, if the work appears only in a preprint, especially one that is recent and/or not widely cited, you should in general give authors the benefit of the doubt and not penalize them for the missing citation. | ||
| + | * Papers should include appropriate discussion of related work, potentially including both preprints and refereed publications. However, it is not reasonable to expect a time-consuming empirical comparison with work that has appeared ''less than 3 months before the submission deadline''. In this situation, the two papers are considered to be ''contemporaneous''. Contemporaneous papers may affect your perspective on the ''quality'' of a submission, but should be disregarded when evaluating the ''novelty'' of the submission, unless there is evidence that they have influenced the submission. | ||
Revision as of 04:27, 18 October 2017
The following guidelines can be used by conferences and journals who adopt the ACL Policies on Submission, Review and Citation.
- For further motivation and discussion, we refer to the report of the working group appointed by the ACL Executive Committee: File:ACL Guidelines for Submission, Review and Citation.pdf.
- For corresponding author guidelines, see: ACL Author Guidelines.
Preserving Double Blind Review
- Reviewing may involve an online search for related work. However, do not set out to uncover the identity of the authors and try not to let any suspicions regarding author identity affect your judgement. To reduce the risk of bias, it is a good idea to read the paper and draft an initial review before you carry out any online searches that risk discovering authorship. You should be especially careful if an area chair has informed you that a non-anonymized version of the paper is available online.
- If you do become aware of the authors' identity (by whatever means), let the area chairs (but not your fellow reviewers) know.
- If you come across a preprint or paper that has a substantial text overlap with the submission, report this to the area chairs even if the existence of a preprint has been declared, since you do not know whether the authors of the discovered paper are the authors of the submission under review, and so the possibility of plagiarism needs to be considered.
Citation and Comparison
- Evaluate papers based on your knowledge of the subject, regardless of where you gained this knowledge. If you are aware of relevant publicly available research that has not been cited in the paper you are reviewing, you should bring it to the attention of the authors, irrespective of whether it is described in a preprint or a refereed publication. However, if the work appears only in a preprint, especially one that is recent and/or not widely cited, you should in general give authors the benefit of the doubt and not penalize them for the missing citation.
- Papers should include appropriate discussion of related work, potentially including both preprints and refereed publications. However, it is not reasonable to expect a time-consuming empirical comparison with work that has appeared less than 3 months before the submission deadline. In this situation, the two papers are considered to be contemporaneous. Contemporaneous papers may affect your perspective on the quality of a submission, but should be disregarded when evaluating the novelty of the submission, unless there is evidence that they have influenced the submission.