2020Q3 Reports: General Chair

ACL 2020 post-conference report: General Chair

Dan Jurafsky

The 58th annual meeting of the Association for Computational Linguistics (ACL) took place online from July 5th through July 10th, 2020.

Summary of what happened

• We managed to run a virtual conference! And it wasn't a disaster!
• Largest ever in # of papers (not surprising) but also in attendance!
• Also most diverse ACL ever!
• People had lots of great discussion on chat; ACL conferences should ramp up use of chat media.
• The team was awesome!! Every set of chairs was amazing and rose to the occasion to deal with the shift to virtual.
• Biggest problem: The infrastructure team had to work way too hard, especially the last two weeks + the conference, it was a completely unreasonable load on them. Beware! Advice on infrastructure below.

Overview of suggestions future conferences

Switching to Virtual: Advice on Personnel and Chairs

• Either pay for a completely professional conference organizer that does all infrastructure themselves, or if we're going to continue to use the Miniconf website that we used this year, then:
  1. Have double the number of virtual infrastructure chairs than we did. We had essentially 9 (6 virtual infrastructure chairs led by Hao Fang and Sudha Rao, plus 2 captioning chairs, plus 1 volunteer chair). Probably you need 18, including specific chairs for live Video (Zoom or whatever), the virtual website, talk recordings, the non-virtual website, chat, and more.
  2. Assign completely separate infrastructure chairs for the workshops, who can be closely in contact with the workshop chairs at all times.
  3. Volunteer coordinator should be more tightly integrated from the beginning with the infrastructure chairs, since the volunteers are mainly helping with infrastructure
• Either waythe workshops will need stricter deadlines for finalizing schedules and materials, much earlier stricter than with a physical conference.
• Create a Virtual Social chair (see https://www.aclweb.org/adminwiki/index.php?title=2020Q3_Reports:_Publicity_Chairs)
• Possibly have an ethics chair (see below)

Switching to Virtual: Advice on Software Partners

• We think the ACL should probably not use SlidesLive going forward. They caused a number of problems:
  • They initially gave us the impression they would handle creating all the zoom rooms for the conference, then changed their mind
  • Then they gave us the impression they would at least handle creating all the zoom rooms for the workshops, then changed their mind at the last minute, too late to create all the zoom rooms for the workshops.
  • They initially gave us the impression that they would do human captioning of live sessions, but then changed to ASR captioning
  • Their setup for allowing authors to correct captions was not easy to use.
  • Their recording setup didn't allow any author editing, which made it frustrating to have to record in single takes
  • We only realized later that they were willing to paste together multiple segments, so tutorials could have been recorded in multiple chunks.
  • Our contact, Alex Chandler, wasn't technical and didn't seem to be a good listener, so complaints often were not addressed.
  • One problem occurred on the day before the workshops when they repeatedly ignored requests for live session ids for the workshops, forcing the Infra Chairs to stay up until the wee hours desparately trying to get them on the phone or email.
  • They did not do well at helping speakers navigate the 40-second delay between the live zoom rooms, and the livescreen window (failing, for example, to explain to speaker that they had to turn off the audio on the livescreen when they were entering the zoom room). It was their lack of clear explanations and dry runs that caused the much-commented-upon 40-second delayed echo in the LTA and in many other speakers, and was especially a problem in the workshops.
• However, we don't know if there are strong alternatives:
  • One option is to find a professional virtual conference organizer that runs everything (including running the website), as well as the videos. We were unable to find such, so I don't know if this is poissible
  • SlidesLive seemed better for our needs than the alternative we considered (we chose Slideslive over Underline and Freeman when it was still a physical conference, and then they were the conference for ICLR and the ICLR chairs seemed happy with them; we didn't find a lot of alternative options at that point).
  • We assume by now there are probably other options, but inclusion is extremely important to keep in mind (making it possible to record for folks in countries without ability to buy video recording software), as is having professionals deal with the livestream events.
  • Slideslive also checked every video for sound or other problems

Switching to Virtual: Advice on Infrastructure

• QA sessions could possibly be improved by incorporating more poster-like elements (like having speakers do 5-minute spiels every 30 minutes or etc)
• Speakers should upload their slides as well as the talks
• How to do virtual live sessions: work out methods for taking questions, probably involving 2 moderators who screen and present questions (and so as to avoid the "more of a comment than a question" people)
• if you use slideslive or any such operator that uses a delay for transcription, need more training of each speaker about how to deal with delay.
• minor: if you use miniconf, fix so that the whole website can shift timezone.
• Nitin's suggestion: "use one of the many bot frameworks (Rasa, BotKit, Botpress, Hubot) to make a simple FAQ bot that can answer repeated questions"

Permissions:

• Decide whether to ask/require people for permission to record their chat conversations
• Explicitly require people to give permission to store the captions of their videos
• Somehow encourage people to correct the captions of their videos.

Global Inclusion:

• For videos (and docs and chats), decide whether to solve problem of viewability in China, Indonesia, Vietnam, etc., or else require VPNs.
  • Vimeo (used by ACL Anthology and also SlidesLive pre-recorded talks) not visible in China, Indonesia, Vietnam
  • SlidesLive live (but not pre-recorded, which uses Vimeo) is is visible in China
• Iran is a particularly difficult case, can't use Google or Microsoft either
• Proposal: we should assume that any single technology could be banned at any time by some government, so we should set up mirrors.

Ethics:

• Require an ethics statement of all papers, add ethics to reviewing form, add ethics, and also diversity & inclusion to all paper award decision criteria.
• Possibly have an ethics chair
• Iran is a particularly difficult case.
• Perhaps we should assume that any single technology could be banned at any time by some government, so we'll need to set up mirrors.

Switching to Virtual: Timeline of Decision to go online:

• Feb 5: Concern about coronavirus expressed by Seattle Convention and Tourist board.
• Mar 2: Companies begin to restrict travel, Stanford switches to virtual classes
• Mar 4: ACL Exec meeting, I briefly fill Exec in that virtual is an option, but still low-probability
• Mar 6: Things worsen. We change Mar 11 site visit to be a virtual meeting with hotel
• Mar 11: At meeting, Hyatt refuses to discuss refunds or cancellations until mid-April
• Mar 17: Things worsened more, I ask ACL Exec advice, they mostly advise going virtual, we wait to hear about hotel $$$
• Mar 25: Priscilla gets word that we can probably delay our Hyatt contract til NAACL 2021, removing the final obstacle to ogoing virtual
• April 2: We announce the decision to go virtual.

<