Difference between revisions of "2016Q3 Reports: Program Chairs"

From Admin Wiki
Jump to navigation Jump to search
Line 16: Line 16:
 
Appendices were added to help address the replication problem. The relevant passage in the Call for Papers reads as follows:
 
Appendices were added to help address the replication problem. The relevant passage in the Call for Papers reads as follows:
  
''ACL 2016 also encourages the submission of supplementary material to report preprocessing decisions, model parameters, and other details necessary for the replication of the experiments reported in the paper. Seemingly small preprocessing decisions can sometimes make a large difference in performance, so it is crucial to record such decisions to precisely characterize state-of-the-art methods.''
+
* ''ACL 2016 also encourages the submission of supplementary material to report preprocessing decisions, model parameters, and other details necessary for the replication of the experiments reported in the paper. Seemingly small preprocessing decisions can sometimes make a large difference in performance, so it is crucial to record such decisions to precisely characterize state-of-the-art methods.''
  
''Nonetheless, supplementary material should be supplementary (rather than central) to the paper. It may include explanations or details of proofs or derivations that do not fit into the paper, lists of features or feature templates, sample inputs and outputs for a system, pseudo-code or source code, and data. The paper should not rely on the supplementary material: while the paper may refer to and cite the supplementary material and the supplementary material will be available to reviewers, they will not be asked to review or even download the supplementary material. Authors should refer to the contents of the supplementary material in the paper submission, so that reviewers interested in these supplementary details will know where to look.''
+
* ''Nonetheless, supplementary material should be supplementary (rather than central) to the paper. It may include explanations or details of proofs or derivations that do not fit into the paper, lists of features or feature templates, sample inputs and outputs for a system, pseudo-code or source code, and data. The paper should not rely on the supplementary material: while the paper may refer to and cite the supplementary material and the supplementary material will be available to reviewers, they will not be asked to review or even download the supplementary material. Authors should refer to the contents of the supplementary material in the paper submission, so that reviewers interested in these supplementary details will know where to look.''
  
  

Revision as of 07:07, 13 July 2016

Innovations

As compared to previous ACL conferences, this year’s main innovations were:

  • In addition to best paper awards, a larger number of outstanding papers were selected. The original plan was to identify roughly 1-2% of submissions; ultimately 11 papers out of 1290 submissions (0.85%) were identified as outstanding by the awards committee.
  • Instead of 20-minute talks, long papers were presented in 15-minute talks, plus 5 minutes for questions and changing speakers. Short papers were given 12 minutes plus 4 for questions and transition.
  • Authors were allowed to submit an appendix of any length with their papers. Reviewers were not required to read appendices.
  • Area chairs provided meta-reviews for all papers that were not obvious rejections.
  • The deadline for short papers was ahead of the deadline for long papers.
  • We introduced a new area labeled “Other” to handle papers that did not fit into traditional areas, as well as a few COI papers.

Rationale

Outstanding papers were added because the community is growing and so is the number of papers at each conference, including the number of papers that deserve recognition as being of particular importance and quality.

Appendices were added to help address the replication problem. The relevant passage in the Call for Papers reads as follows:

  • ACL 2016 also encourages the submission of supplementary material to report preprocessing decisions, model parameters, and other details necessary for the replication of the experiments reported in the paper. Seemingly small preprocessing decisions can sometimes make a large difference in performance, so it is crucial to record such decisions to precisely characterize state-of-the-art methods.
  • Nonetheless, supplementary material should be supplementary (rather than central) to the paper. It may include explanations or details of proofs or derivations that do not fit into the paper, lists of features or feature templates, sample inputs and outputs for a system, pseudo-code or source code, and data. The paper should not rely on the supplementary material: while the paper may refer to and cite the supplementary material and the supplementary material will be available to reviewers, they will not be asked to review or even download the supplementary material. Authors should refer to the contents of the supplementary material in the paper submission, so that reviewers interested in these supplementary details will know where to look.


arealong reviewedlong acceptedshort reviewedshort acceptedtotal submissionspercentage of total submissionstotal acceptedpercentage of total acceptedarea acceptance rateoutstanding papers
Semantics11443661318014.0%5617.1%31.1%3
Information Extraction, Question Answering, and Text Mining1222748917013.2%3611.0%21.2%1
Sentiment Analysis and Opinion Mining7792831058.2%123.7%11.4%
Document Analysis53134981027.9%216.4%20.6%
Machine Translation5815369947.3%247.3%25.5%1
Tagging, Chunking, Syntax, and Parsing4818339816.3%278.2%33.3%2
Social Media396308695.4%144.3%20.3%
Machine Learning4615226685.3%216.4%30.9%1
Resources and Evaluation4417216655.0%237.0%35.4%
Other3410275614.7%154.6%24.6%
Discourse and Pragmatics4215184604.7%195.8%31.7%
Summarization295192483.7%72.1%14.6%
Multilinguality196245433.3%113.4%25.6%
Phonology, Morphology, and Word Segmentation236104332.6%103.0%30.3%1
Generation20893292.3%113.4%37.9%
Dialog and Interactive Systems20780282.2%72.1%25.0%1
Cognitive Modeling and Psycholinguistics17781251.9%82.4%32.0%1
Vision, Robots, and Grounding14362201.6%51.5%25.0%
Speech611070.5%10.3%14.3%
total825231463971288100.0%328100.0%25.5%11