Difference between revisions of "2014Q3 Reports: TACL Journal Editor"

Revision as of 02:48, 22 June 2014

Statistics on papers from 2013

TACL published 35 papers in 2013, and at the time of writing (June 2014) it has published 19 papers in 2014. Papers have been presented at various conferences including ACL 2013 (16 papers), NAACL 2013 (6 papers), EMNLP 2013 (5 papers), and EACL 2014 (2 papers). 19 papers will be presented at ACL 2014.

Submission Numbers

The total number of papers submitted in 2013 was 117. The number of papers submitted each month was as follows:

 Jan 10
 Feb 4
 Mar 11
 Apr 3
 May 11
 Jun 10
 Jul 8
 Aug 8
 Sep 11
 Oct 13
 Nov 18
 Dec 15

Acceptance Rates, and Reviewing Times

At the time of writing all 117 papers submitted in 2013 had received first decisions.

On receiving reviews for a paper, the action editor makes 1 of 4 decisions for the paper: (a) Acceptance for publication as is, with encouragement to make minor revisions. (b) Acceptance for publication subject to specified revisions within two months. (c) Rejection with encouragement to revise and resubmit within 3-6 months. (d) Rejection with no possibility of resubmission within a moratorium period of one year. =

Breakdown by Decision Type

Of the 117 papers from 2013, the breakdown by decision type was as follows:

Category a:  4 papers  (3.4%)
Category b: 24 papers (20.5%)
Category c: 39 papers (33.3%)
Category d: 50 papers (42.7%)

It is worth noting that papers falling into category (b) are almost always accepted when the required revisions are carried out.

Reviewing Times

The mean time to first decision on a paper was 67 days for the 117 papers. Some other statistics on time to first decision (where a month is counted as 30 days):

 23 papers  (20%) had 1 month  13 days or less
 47 papers  (40%) had 1 month  24 days or less
 70 papers  (60%) had 2 months  7 days or less
 94 papers  (80%) had 3 months  1 days or less
105 papers  (90%) had 3 months 20 days or less
117 papers (100%) had 5 months  6 days or less

Recent Policy Decisions

Please see the circulated non-public appendix, due to sensitivity.

@@ Line 1: / Line 1: @@
+== Statistics on papers from 2013 ==
-This report gives some key points regarding the experience with TACL over the last year.
+TACL published 35 papers in 2013, and at the time of writing (June
+) it has published 19 papers in 2014. Papers have been presented at
+various conferences including ACL 2013 (16 papers), NAACL 2013 (6
+papers), EMNLP 2013 (5 papers), and EACL 2014 (2 papers). 19 papers will
+be presented at ACL 2014.
-== Submission numbers ==
+== Submission Numbers ==
+The total number of papers submitted in 2013 was 117. The number of
+papers submitted each month was as follows:
-TACL has now had 111 submissions in total since its launch in May 2012. Of those papers, 20 are still in review, and 91 have had decisions. Of those 91 papers, 28 papers have been accepted and published. Some of the remaining 63 papers fall into one of the "revise and resubmit" categories, so they may still be published after revisions.
+  Jan 10
+  Feb 4
+  Mar 11
+  Apr 3
+  May 11
+  Jun 10
+  Jul 8
+  Aug 8
+  Sep 11
+  Oct 13
+  Nov 18
+  Dec 15
-The distribution of submissions by month is as follows:
+== Acceptance Rates, and Reviewing Times ==
-May 1st 2012: 6  <br>
+At the time of writing all 117 papers submitted in 2013 had received first
-Jun 1st 2012: 6  <br>
+decisions.
-Jul 1st 2012: 3  <br>
-Aug 1st 2012: 3  <br>
-Sep 1st 2012: 8  <br>
-Oct 1st 2012: 15  <br>
-Nov 1st 2012: 17  <br>
-Dec 1st 2012: 11  <br>
-Jan 1st 2013: 8  <br>
-Feb 1st 2013: 5 <br>
-Mar 1st 2013: 8 <br>
-Apr 1st 2013: 2 <br>
-May 1st 2013: 3 <br>
-Jun 1st 2013: 6 <br>
-Jul 1st 2013: 10
-== Review Process/Benefits of "Revise and Resubmit ==
+On receiving reviews for a paper, the action editor makes 1 of 4
+decisions for the paper: (a) Acceptance for publication as is, with
+encouragement to make minor revisions. (b) Acceptance for publication
+subject to specified revisions within two months. (c) Rejection with
+encouragement to revise and resubmit within 3-6 months. (d) Rejection
+with no possibility of resubmission within a moratorium period of one
+year.
+=
+== Breakdown by Decision Type ==
+Of the 117 papers from 2013, the breakdown by decision type was as
+follows:
-There are 35 action editors for TACL. Each paper is assigned to an action editor shortly after submission. The action editor then chooses 3 reviewers for the paper; the reviewers have 3 weeks to review the paper. Once the reviews are in, a decision is made by the action editor for the paper.
+ Category a:  4 papers  (3.4%)
+ Category b: 24 papers (20.5%)
+ Category c: 39 papers (33.3%)
+ Category d: 50 papers (42.7%)
-The appendix shows the time taken to first decision for the 51 papers submitted Nov 1st 2012 or later, that now have decisions made. Some key statistics are as follows:
+It is worth noting that papers falling into category (b) are almost
+always accepted when the required revisions are carried out.
-%  (10 papers): 1 months 08 days or less <br>
+== Reviewing Times ==
-%  (20 papers): 1 months 12 days or less <br>
-%  (30 papers): 1 months 26 days or less <br>
-%  (40 papers): 2 months 08 days or less <br>
-% (51 papers): 3 months 16 days or less
-The maximum time (3 months 16 days) is a little misleading, as there are a couple of papers in the system that have taken over 3 months and do not yet have a final decision. However the statistics do show relatively fast turnaround in reviewing.
+The mean time to first decision on a paper was 67 days for the 117
+papers. Some other statistics on time to first decision (where a month
+is counted as 30 days):
-We view the "revise and resubmit" options within TACL as particularly beneficial.  Of the 28 papers accepted so far, 9 papers were immediate accepts, and 19 fell into one of the "revise and resubmit" categories. Most of these fell into the fast revise and resubmit category, with specific revisions required within 2 months; but some fell within the category of major revisions in the 3-6 month time period.
+papers  (20%) had 1 month  13 days or less
+papers  (40%) had 1 month  24 days or less
+papers  (60%) had 2 months  7 days or less
+papers  (80%) had 3 months  1 days or less
+papers  (90%) had 3 months 20 days or less
+papers (100%) had 5 months  6 days or less
-There are in our view multiple benefits for revise and resubmit options. From the reviewer's perspective, it allows a reviewer to specify what they see as necessary revisions before publication. From the author's perspective, a paper which may have been borderline - and thus at risk or rejection, followed by resubmission to a new conference, with a new set of reviewers - becomes an acceptance once specified revisions have been made. From the point of view of the community, the revise and resubmit cycle is likely to produce higher quality final papers.
+== Recent Policy Decisions ==
-Lastly, there are obvious benefits in reviewing efficiency, in that borderline papers have required corrections made, and are sent back to the same set of reviewers, rather than being resubmitted to another conference (potentially with recommended changes from reviewers being ignored).
+Please see the circulated non-public appendix, due to sensitivity.
-== Appearance of TACL papers at conferences ==
-Of the 28 accepted papers, 6 were presented at NAACL 2013, and 16 will be presented at ACL 2013. The other 6 papers are TBD.
-In addition to agreements with ACL 2013 and NAACL 2013, TACL has reached agreements with EMNLP 2013 and EACL 2014, with acceptance deadlines of August 31st 2013 and November 30th 2013 respectively.
-One issue arose at NAACL 2013 (and may arise at ACL 2013). The agreement with conferences was that TACL papers should have roughly the same proportion of posters vs talks as regular papers accepted to the conference. Some TACL authors were initially surprised that their paper received a poster slot rather than a talk (although we believe that they did not have a reason to be sure of a talk); we received some suggestions from various people in the community (including past ACL presidents) that all TACL papers should get talks, the danger being that otherwise people would not want to submit to TACL. We are not at all sure that this is feasible or desirable given the increasing necessity for poster slots at ACL and other conferences. But this issue should be discussed with the ACL exec.
-== Outstanding issues ==
-We believe that TACL has been a real success. However given what we have learned in this first year or so, there are some significant issues - which may require additional resources - which we believe need to be addressed. These issues will only become more pressing as TACL grows. Specifically:
-(a) Administrative support
-Having a few hours of admin support each week, for the day to day running of the journal, would be a great help. It is possible that we could find help for this from Columbia; this would require funding from the ACL exec. We would prefer to pay a professional admin person for this (rather than for example hiring a student).
-(b) Publication process
-There are many nuances to the publication process. At a low level, each camera ready paper needs to be prepared and published - this is a time consuming process, but could potentially be handled by a student helping with the journal. More significantly, there are many strategic/higher level issues such as: the design of style files for the journal; posting papers in a way such that they are indexed correctly by Google scholar; handling DOIs correctly; assigning ISSN numbers; having the journal registered with Thomson Reuters; interacting correctly with the ACL anthology; ensuring that the journal is archived correctly; monitoring statistics for the journal such as impact factor; and so on.
-For this reason it may make sense for TACL to assign somebody to the role of production or publications editor. We have some potential names in mind for this role.
-(c) Interaction with the conferences
-The interaction with conferences has in general been very productive. There are just a few issues.
-One is the issue of acceptance dates for TACL. These dates have to be negotiated with the various conferences, and must synchronize with reviewing schedules. The danger here is that this takes time, and that acceptance deadlines are announced late (as an example, the EMNLP 2013 acceptance date of August 31st 2013 was announced in mid April, which is rather late).
-Another is that TACL papers have to be handled essentially manually with respect to the conference program. For example the titles/authors were first sent to the ACL PC chairs; the PC chairs made decisions about posters vs talks; these decisions were then communicated to the TACL editors who passed on those decisions to the TACL authors. It would be helpful if TACL papers could be more closely integrated within the ACL process, although it is not entirely clear how to do this.
-== Appendix: Time to first decision, in sorted order ==
-Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.
-0 months 26 days <br>
-0 months 29 days <br>
-1 months 03 days <br>
-1 months 03 days <br>
-1 months 05 days <br>
-1 months 06 days <br>
-1 months 07 days <br>
-1 months 08 days <br>
-1 months 08 days <br>
-1 months 08 days <br>
-1 months 08 days <br>
-1 months 09 days <br>
-1 months 09 days <br>
-1 months 10 days <br>
-1 months 11 days <br>
-1 months 12 days <br>
-1 months 12 days <br>
-1 months 12 days <br>
-1 months 12 days <br>
-1 months 12 days <br>
-1 months 13 days <br>
-1 months 13 days <br>
-1 months 16 days <br>
-1 months 17 days <br>
-1 months 18 days <br>
-1 months 19 days <br>
-1 months 21 days <br>
-1 months 22 days <br>
-1 months 22 days <br>
-1 months 26 days <br>
-1 months 27 days <br>
-1 months 27 days <br>
-2 months 03 days <br>
-2 months 03 days <br>
-2 months 05 days <br>
-2 months 06 days <br>
-2 months 06 days <br>
-2 months 07 days <br>
-2 months 07 days <br>
-2 months 08 days <br>
-2 months 08 days <br>
-2 months 09 days <br>
-2 months 09 days <br>
-2 months 16 days <br>
-2 months 16 days <br>
-2 months 22 days <br>
-2 months 22 days <br>
-2 months 23 days <br>
-3 months 01 days <br>
-3 months 03 days <br>
-3 months 16 days