2016Q3 Reports: Program Chairs

Innovations

As compared to previous ACL conferences, this year’s main innovations were:

• In addition to best paper awards, a larger number of outstanding papers were selected. The original plan was to identify roughly 1-2% of submissions; ultimately 11 papers out of 1290 submissions (0.85%) were identified as outstanding by the awards committee.
• Instead of 20-minute talks, long papers were presented in 15-minute talks, plus 5 minutes for questions and changing speakers. Short papers were given 12 minutes plus 4 for questions and transition.
• Authors were allowed to submit an appendix of any length with their papers. Reviewers were not required to read appendices.
• Area chairs provided meta-reviews for all papers that were not obvious rejections.
• The deadline for short papers was ahead of the deadline for long papers.
• We introduced a new area labeled “Other” to handle papers that did not fit into traditional areas, as well as a few COI papers.

Rationale

Outstanding papers were added because the community is growing and so is the number of papers at each conference, including the number of papers that deserve recognition as being of particular importance and quality.

Appendices were added to help address the replication problem. The relevant passage in the Call for Papers reads as follows:

• ACL 2016 also encourages the submission of supplementary material to report preprocessing decisions, model parameters, and other details necessary for the replication of the experiments reported in the paper. Seemingly small preprocessing decisions can sometimes make a large difference in performance, so it is crucial to record such decisions to precisely characterize state-of-the-art methods.

• Nonetheless, supplementary material should be supplementary (rather than central) to the paper. It may include explanations or details of proofs or derivations that do not fit into the paper, lists of features or feature templates, sample inputs and outputs for a system, pseudo-code or source code, and data. The paper should not rely on the supplementary material: while the paper may refer to and cite the supplementary material and the supplementary material will be available to reviewers, they will not be asked to review or even download the supplementary material. Authors should refer to the contents of the supplementary material in the paper submission, so that reviewers interested in these supplementary details will know where to look.

Meta-reviews were strongly recommended by the NAACL 2015 program co-chairs.

The deadline for short papers was moved ahead of the deadline for long papers to coordinate deadlines with NAACL, such that rejected NAACL submissions could be reworked into ACL submissions.

The “Other” area was introduced to give papers in non-traditional research areas a better chance at fair review. Rather than conventional bidding, the “Other” area chairs were given access to the reviewer pool directly (while bidding was taking place for papers in the rest of the areas), so that these papers were given the first chance at the most appropriate reviewers.

Submissions and Presentations

ACL 2016 received a total of 1288 valid submissions, of which 825 were long papers and 463 were short papers. 21 long papers and 9 short papers were rejected without review due to non-anonymity or formatting issues. The remaining submissions were each assigned to one of 19 areas, and managed by a program committee of 38 area chairs and 884 reviewers (including secondary reviewers indicated on the review forms). 231 (28%) of the 825 qualifying long papers and 97 (21%) of the 463 qualifying short papers were selected for presentation at the conference. Of the accepted long papers, 116 were selected for oral presentation, and 115 for poster presentation. Of the accepted short papers, 49 have oral and 48 have poster presentations. The oral versus poster decision was made based on the recommendations of reviewers, which we took as a noisy signal of the intended audience’s preference of format for each paper.

In addition, 25 TACL papers will be presented at ACL – 24 as talks and one as a poster. Including TACL papers, there will be 189 oral and 163 poster presentations at the main ACL conference. The table below shows the number of reviewed submissions in each area for long and short papers, as well as the number of papers accepted in each area. Approximately 59 short and 52 long papers were withdrawn before review was completed; these are not included in the table.

area	long reviewed	long accepted	short reviewed	short accepted	total submissions	percentage of total submissions	total accepted	percentage of total accepted	area acceptance rate	outstanding papers
Semantics	114	43	66	13	180	14.0%	56	17.1%	31.1%	3
Information Extraction, Question Answering, and Text Mining	122	27	48	9	170	13.2%	36	11.0%	21.2%	1
Sentiment Analysis and Opinion Mining	77	9	28	3	105	8.2%	12	3.7%	11.4%
Document Analysis	53	13	49	8	102	7.9%	21	6.4%	20.6%
Machine Translation	58	15	36	9	94	7.3%	24	7.3%	25.5%	1
Tagging, Chunking, Syntax, and Parsing	48	18	33	9	81	6.3%	27	8.2%	33.3%	2
Social Media	39	6	30	8	69	5.4%	14	4.3%	20.3%
Machine Learning	46	15	22	6	68	5.3%	21	6.4%	30.9%	1
Resources and Evaluation	44	17	21	6	65	5.0%	23	7.0%	35.4%
Other	34	10	27	5	61	4.7%	15	4.6%	24.6%
Discourse and Pragmatics	42	15	18	4	60	4.7%	19	5.8%	31.7%
Summarization	29	5	19	2	48	3.7%	7	2.1%	14.6%
Multilinguality	19	6	24	5	43	3.3%	11	3.4%	25.6%
Phonology, Morphology, and Word Segmentation	23	6	10	4	33	2.6%	10	3.0%	30.3%	1
Generation	20	8	9	3	29	2.3%	11	3.4%	37.9%
Dialog and Interactive Systems	20	7	8	0	28	2.2%	7	2.1%	25.0%	1
Cognitive Modeling and Psycholinguistics	17	7	8	1	25	1.9%	8	2.4%	32.0%	1
Vision, Robots, and Grounding	14	3	6	2	20	1.6%	5	1.5%	25.0%
Speech	6	1	1	0	7	0.5%	1	0.3%	14.3%
total	825	231	463	97	1288	100.0%	328	100.0%	25.5%	11