2016Q3 Reports: TACL Journal Editor

(We are very grateful to have received an extension on filing our report until July 22 from ACL Secretary Shiqi Zhao. --the TACL editors in chief)

Personnel updates

The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson may or may not stay on past that time.

We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee Tou Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)

218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deeply, truly grateful for --- the rest have agreed to rejoin.

We anticipate issuing invitations to new people as well soon.

We sometimes joke that the "A" in TACL stands for "artisanal". The reviewers and action editors are, on the whole, putting a huge amount of time and craft into their "pieces", broadly construed. (For one bit of proof, see the statistics below.) Whether this "personal touch" can (or should) be scaled to a conference-size production line is an important question.

Interface with conferences

Our checks for undeclared multiple/overlapping submissions with main conferences are surprisingly time-consuming but unfortunately do turn up problematic cases. As one consequence, we have clarified our policy language to state that no material in a TACL submission can be under consideration elsewhere while under review by TACL (see website for exact policy).

Author-response periods for conferences --- so that authors see reviews before they receive an official decision --- raise the question from authors, "After getting negative reviews from a conference, if we withdraw the paper from the conference (and revise it), can we submit it to TACL?" We consider a paper that has received reviews and is not accepted by a conference to be "rejected" from TACL's point of view. Hence, submission of a heavily revised version to TACL is allowed if the TACL submission follows the rules for papers rejected from *ACL/EMNLP; but undeclared (revisions of) such submissions will be rejected from TACL.

Counts of TACL papers (to be) presented at this year's conferences: 10 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps influenced by the conferences' decisions to have shorter talk slots and hence more talk slots available. However, it's not clear whether this is consistent with prior conference's policies that the poster/oral decision be based on suitability of topic to presentation type.

Interface with preprint servers/workshop proceedings

In the copy-editing phase, we are requesting that authors replace references to preprints such as arXiv papers and other non-refereed papers with references to peer-reviewed versions, if available. We understand that this decision could influence authors' choices about whether to post preprints.

We have agreed that papers for shared tasks familiar to the ACL community whose papers are put in a separate proceedings instead of the main meeting proceedings will be treated as workshop papers, and thus not subject to the multiple-submission restrictions.

Transition to new (old) domain name

We are transitioning from using URLs referencing tacl2013.cs.columbia.edu to transacl.org ([1]) . Old links (for both http and https) should still work, but we ask that people try to use the transacl.org format and update links they control, if possible. The plan is to get DOIs that are linked to transacl.org URLs.

Some statistics

number of submissions and timing of first decision

Depicted below is the history of the number of distinct submission IDs that have received a decision, and the average time to first decision, grouped by round (=nearest first-of-the-month). The decision time for papers submitted within in a month is counted as starting from the first of the next month.

The drop in the last two months is due at least in part to 13 papers submitted in that period that have not yet received a decision. Also not counted: papers handled using START (the early days of TACL), papers, and the 106 papers archived for some technical or formatting problem.

Observations:

While we don't count technical/format problems, we "do" count papers that receive an editorial decision of rejection without going out for full review. One way we EiCs are trying to keep reviewing loads down and submissions interesting for our reviewers is sometimes doing a preliminary consultation with an action editor or highly relevant reviewer for submissions that seem highly unlikely to be accepted. Such consultations and subsequent discussion can take two-three weeks, since we strive to be cautious about making such decisions, but also want to factor in the value of conserving reviewer effort.

Waves in the number of submissions seem correlated with upcoming conference deadlines.

Average decision times per month appear to correspond well with an ideal but attainable schedule of:

ideally 7 days to make a load-balanced assignment of submissions to action editors (AEs)

ideally 7 days for AEs to get three reviewers confirmed to review

ideally 21 days for reviewers to review

ideally 10 days for AE to coordinate discussion among reviewers and come to a decision

which would yield an ideal target of 45 days turnaround.

For reviewers that complete their reviewers, the average completion time is in fact 23 days.

Decision statistics

First decision breakdown for our 589 submissions.

  9% (a) = accepted as is
 22% (b) = conditional accept:  acceptance guaranteed if conditions met
 31% (c) = rejected, encourage resubmission but no guarantee of acceptance 
 38% (d) = rejected with 1-year moratorium on TACL submission.  
         =  32% rejected based on full review, 6% rejected without full review (but received editorial consideration)

72 (=12%) of these papers are revised versions of (c) papers: TACL treats these as new submissions. Of these 72 papers, the first-decision breakdown is:

 44% (a)
 11% (b)
 14% (c)
 30% (d)

Publishing statistics

29 papers have been published so far in 2016.

Another 8 are in the publication queue.