2020Q3 Reports: Workshop Chairs
Workshop co-chairs:
Milica Gasic (Heinrich Heine University Düsseldorf)
Dilek Hakkani-Tür (Amazon Alexa)
Saif M. Mohammad (National Research Council Canada)
Ves Stoyanov (Facebook AI)
It has been an unusual year and an unusual ACL. COVID-19 affected practically every aspect of our everyday life and affected the CL community.
In the early days of the pandemic, the NLP community rallied with projects aimed to help dealing with the pandemic. To help mobilize and organize the community, we introduced the emergency NLP for COVID workshop. Karin Verspoor from the University of Melbourne stepped up and led the organization of the workshop, which experimented with a new format: rolling submission and open rolling review. The workshop received 25 submissions in the months leading to the deadline and 50 more on the final submission day. Unfortunately, this last-minute load meant that the final submissions could not be reviewed in time for the workshop. Fortunately, the NLP for COVID will have a second iteration at EMNLP, which will allow the new submission to be reviewed and published. Many thanks for the organizing committee and Karin in particular for taking on this challenge and experimenting with the new format.
The virtual setup was particularly challenging for workshop organization as workshops come in many different sizes and flavors. We aimed to support different options allowing the workshop organizers to choose the format that suits their workshop best. Most workshops settled on a similar format with pre-recorded talks and live QAs.
Here are some lessons that we learned:
- FAQ document: A large amount of emails was going back and forth between the workshop organisers, the workshop chairs, the infrastructure chairs and SlidesLive, the company facilitating recordings. This made it hard to keep track of the latest. We prepared a two-page FAQ document summarising all important information and this was extremely helpful. We believe that this is something that should exist even if the workshops are not virtual.
- Short notice: Given how new the virtual form was for all of us, we often had to inform workshop organisers and ask for feedback on a very short notice. This is clearly not desirable, but eventually we managed to get everything running on time. The workshop organizers were amazing in how fast and responsible they were.
- Email list: Regarding the communication, having one master email to send information to all workshop organisers helps a lot. Softconf has only one email per workshop, and we found that it is useful to get hold of as many emails as possible. We created a Google group with all organizers and used it as the primary form of communication.
And here are some statistics: This year the joint call for workshop proposals for ACL/EMNLP/COLING/AACL-IJCNLP received 95 proposals (compared to 84 in 2019 and 58 in 2018). Out of the 95, 71 were accepted between the four venues. ACL 2020 will feature 19 workshops. We accepted 71 workshops out of 95 workshop proposal submission through the process of a joint cross-conference NLP workshops call, which for 2020 was shared between ACL, AACL-IJCNLP, COLING and EMNLP (Link to CFP email announcement). Workshop proposals were accompanied with the organizers' top two venue preferences. Submissions and reviews were coordinated through START, and reviewed and discussed by a joint cross-conference committee including the workshop chairs of all three conferences, including the three of us. We largely followed last year’s model and the reviewer load was 19 proposals/reviewer). During the joint selection process, we tried to allocate accepted workshops to their first or second choice of venue, however, this was not always possible. Of the 95 submissions, 54% had ACL as their first choice, and 19% as second choice. Due to higher interest for ACL, we first chose workshops for ACL from amongst the ones that had ACL as a choice, respecting the first choice of the proposers.
The ACL workshops were assigned to the two days based on room availability, projected registrations, and in limited cases to maintain topical balance. The Student Research Workshop was a "pre-admitted" workshop. In general for the organization and coordination process, we followed the excellent workshop chairs duties guide. We would like to emphasize that we followed earlier recommendations and underline the challenges of organising virtual workshops.