Tutorial chair handbook

From Admin Wiki
Jump to navigation Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.

Duties

Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in here.

Timeline

The timeline for the organisation of tutorials should follow the general conference planning schedule. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the conference planning schedule are highlighted in bold.
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below.
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

  • [T-12]
    • Learn from the general chair which conferences will have their tutorials coordinated with yours.
    • Establish contact with the tutorial chairs of these other conferences, and discuss
      • Recruitment of external reviewers
      • Proposal evaluation criteria
      • Soliciting tutorial proposals from particular potential lecturers on particular topics
      • Organizing a public poll about interest in the submitted tutorial
    • Discuss general strategies of the main conference with the general chair and the program chairs
      • promotion of some topics
      • mailing lists and conference alert portals to be used for all announcements
    • Set up a SoftConf space for proposal submission
      • Solicit the creation of a SoftConf space from support (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
      • If you plan to have external tutorial reviewers, invite them via SoftConf.
      • Set up a submission page.
    • Set up a joint email for tutorial chairs.
    • Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
    • Draft the joint call for tutorials (based on past calls).
  • [T-11] (in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)
    • Send out the call for tutorials.
    • For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
    • If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
    • Set up a SoftConf page for proposal reviews.
  • [T-10]
    • Deadline for tutorial proposal submission.
    • If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
    • Assign proposals to reviewers, send out invitations to review (via SoftConf).
  • [T-9]
    • Select tutorials jointly with other conference chairs.
    • Notify the authors (via SoftConf).
    • Liaise with local chair for space needs and scheduling.
    • Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
    • Liaise the conference website chair for updates about tutorials and authors' guidelines.
    • Liaise with the ACL Office for the registration form.
  • [T-4]
    • Collect short descriptions for proceedings from tutorial teachers.
    • Provide them to the webmaster and the general and program chairs .
  • [T-3]
    • Confirm the tutorials with authors.
    • Authors submit the final tutorial descriptions for the proceedings.
    • Arrange for printing of tutorial materials.
  • [T-2]
    • Arrange with the SoftConf support the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
    • Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.
  • [T-1]
    • Collect and check final tutorial materials.
    • Create the tutorial proceedings, including slides.
  • [T-0] Conference and tutorials
  • [T+1]
    • Invite the lecturers to update the slides at the ACL Anthology via this form (under CC-BY-4.0).
    • Update this tutorial page and the list of past tutorials with the data from your conference. For access to the ACL Conference Handbook, contact the ACL Secretary (currently Shiqi Zhou).

Tutorials in a virtual conference

ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from ICML. SlideLive was the subcontractor for the virtual infrastructure.

The tasks for tutorials included:

  • collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
  • defining time slots which would increase participation across various time zones
  • collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
  • setting up the schedule
  • informing the participants about virtual participation and about the detailed modalities for each tutorial
  • defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
  • creating automatic captioning, sending the captions to the teachers and uploading the edited versions
  • dry runs with teachers and SlidesLive people
  • archiving pre-recordings or recordings of the live sessions additionally to slides

More details are given in the final report.

Publication of tutorials

For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:

  • conference website
  • ACL Anthology
  • private repositories of the authors

The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects mirrors of some of them (but not all). The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many past tutorials, no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides in the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via this form.

It is recommended for tutorial chairs to:

  • announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
  • encourage the authors to also publish, under open licenses, their code and data used in the tutorials
  • include the initial versions of tutorial slides, together with abstract, in the proceedings
  • encourage the tutorial authors to upload the final versions of their slides after the conference via this form

Organization of past tutorials

For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

Year Coordinated conferences Tutorial chairs Submission platform # Submissions Reviewers # Reviews per proposal # Selected tutorials Selection criteria Useful links
2020 ACL
EMNLP
AACL-IJCNLP
COLING 		ACL: Agata Savary (France), Yue Zhang (China)
EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil)
AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA)
COLING: Daniel Beck (Australia), Lucia Specia (UK) 		SoftConf 43 Tutorial chairs + external experts 3 ACL: 8
EMNLP: 7
ACCL-IJCNLP: 6
COLING: 7 		Clarity and preparedness
Novelty or timely character of the topic
Lecturers' experience
Likely audience interest
Open access of the teaching material
Diversity aspects (multilingualism, gender, age and country of the lecturers)
Compatibility with the required venues 		Call
Report
ACL tutorials
EMNLP tutorials
ACCL-IJCNLP tutorials
COLING tutorials
2019 NAACL
ACL
EMNLP 		ACL: Preslav Nakov (Qatar), Alexis Palmer (USA)
NAACL: Anoop Sarkar (Canda), Michael Strube (Germany)
EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia) 		SoftConf 45 Tutorial chairs 2-3 ACL: 9
NAACL: 6
EMNLP: 7 		Presenters (1-5)
Description (1-5)
Interest Level (1-5)
Preparedness (1-5) 		Call
ACL tutorials
NAACL tutorials
2018 ACL
NAACL
EMNLP
COLING 		ACL: Yoav Artzi (USA), Jacob Eisenstein (USA)
COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA)
EMNLP: Mausam (India), Lu Wang (USA)
NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA) 		SoftConf 49 Tutorial chairs 2 ACL: 8
NAACL: 6
EMNLP: 6
COLING: 6 		Quality of plan
Diversity of topics
Qualifications of presenter
likely audience interest 		Call

Report
ACL tutorials
EMNLP tutorials
COLING tutorials

2017 ACL
EACL
EMNLP 		ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA)
EACL: Alex Klementiev (Germany), Lucia Specia (UK)
EMNLP: Nathan Schneider (USA), Alexandra Birch (UK) 		SoftConf 26 Tutorial chairs 3 ACL: 6
EACL: 6
EMNLP: 7 		Clarity (description and outline)
Topic 		Report
ACL tutorials
EACL tutorials
2016 ACL
NAACL
EMNLP 		ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands)
NAACL: Alexander Rush and Bishan Yang
EMNLP: Rebecca Hwa and Mohit Bansal 		SoftConf 32 Tutorial chairs 2 ACL: 8
NAACL: 6
EMNLP: 6 		Relevance to ACL community (15)
Quality of instructor (15)
Outline and depth/breadth of the proposal (15)

Potential attendance
Newly emerging area not previously covered in an ACL-related tutorial (yes or no)
Introduction into related fields (yes or no)
Overall score (15)
1st preference for venue as indicated in proposal
comments

Call
Report
NAACL tutorials
EMNLP tutorials

Past calls for tutorials

Reports from past tutorial chairs

Lessons learned by past tutorial chairs

  • [Agata Savary, 2020] If the conference is virtual:
    • Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also this post).
    • Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
    • If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
  • [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
  • [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
  • [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
    • Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
    • The chairs for each venue select their best choices in a round robin manner.
    • Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
  • [Agata Savary, 2020] While setting up a review form on SoftConf, include:
    • All evaluation criteria, including diversity, in the review form
    • Cutting edge vs. introductory category
    • Duration (3 or 6 hours)
  • [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
  • [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via this form. As a tutorial chair, make sure that
    • initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
    • tutorial authors are encouraged to upload the final versions of their slides after the conference via this form
  • [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
  • [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
  • [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the report)
  • [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats


Author: Agata Savary, July 2019-July 2020

Updates are welcome

Retrieved from "https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook&oldid=74886"