Difference between revisions of "Tutorial chair handbook"

From Admin Wiki
Jump to navigation Jump to search
 
(95 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
== Duties ==
 
== Duties ==
−
Tutorial chairs are nominated by the General Chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].  
+
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].  
  
 
== Timeline ==
 
== Timeline ==
−
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as followed for ACL 2020 tutorials. The dates from the [[conference planning schedule|conference schedule]] are highlighted in bold.</ br>
+
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''.<br />
−
In recent years, calls and selection of tutorial proposals are coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below.</ br>
+
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below.<br />
−
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the CL domain.
+
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.
  
 
* [T-12]   
 
* [T-12]   
−
** Learn from the General Chair which conferences will have their tutorials coordinated with yours.
+
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
−
** Establish contact with the Tutorial Chairs of these other conference.
+
** Establish contact with the tutorial chairs of these other conferences, and discuss
−
** Discuss general strategies of the main conference (e.g. concerning the promotion of some topics) with the General Chair and the Program Chairs.
+
*** Recruitment of external reviewers
−
** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
+
*** Proposal evaluation criteria
 +
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
 +
*** Organizing a public poll about interest in the submitted tutorial
 +
** Discuss general strategies of the main conference with the general chair and the program chairs
 +
***promotion of some topics
 +
***mailing lists and conference alert portals to be used for all announcements
 +
** Set up a SoftConf space for proposal submission
 +
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
 +
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
 +
*** Set up a submission page.
 
** Set up a joint email for tutorial chairs.
 
** Set up a joint email for tutorial chairs.
−
** If you plan to have external tutorial reviewers, recruit them now.
+
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
−
** Confirm the number and length of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen)
+
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).
−
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]])
 
  
−
* '''[T-11]''' Send out the call for tutorials
+
* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
 +
** Send out the call for tutorials.
 +
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
 +
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
 +
** Set up a SoftConf page for proposal reviews.
  
−
== Past tutorials ==
+
* [T-10]
−
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.
+
** Deadline for tutorial proposal submission.
 +
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
 +
** Assign proposals to reviewers, send out invitations to review (via SoftConf).
 +
 
 +
* '''[T-9]'''
 +
** Select tutorials jointly with other conference chairs.
 +
** Notify the authors (via SoftConf).
 +
** Liaise with local chair for space needs and scheduling.
 +
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
 +
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
 +
** Liaise with the ACL Office for the registration form.
 +
 
 +
* '''[T-4]'''
 +
** Collect short descriptions for proceedings from tutorial teachers.
 +
** Provide them to the webmaster and the general and program chairs .
 +
 
 +
* '''[T-3]'''
 +
** Confirm the tutorials with authors.
 +
** Authors submit the final tutorial descriptions for the proceedings.
 +
** Arrange for printing of tutorial materials.
 +
 
 +
* [T-2]
 +
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
 +
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.
 +
 
 +
* '''[T-1]'''
 +
** Collect and check final tutorial materials.
 +
** Create the tutorial proceedings, including slides.
 +
 
 +
* '''[T-0]''' Conference and tutorials
 +
 
 +
* [T+1]
 +
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
 +
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).
 +
 
 +
== Tutorials in a virtual conference ==
 +
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [https://icml.cc/ ICML]. SlideLive was the subcontractor for the virtual infrastructure.
 +
 
 +
The tasks for tutorials included:
 +
* collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
 +
* defining [https://acl2020.org/_pages/docs/ACL_2020_Tutorial_Timeslots.pdf time slots] which would increase participation across various time zones
 +
* collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
 +
* setting up the [https://acl2020.org/program/tutorials/ schedule]
 +
* informing the participants about [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ virtual participation] and about the [https://acl2020.org/blog/detailed-modalities-of-tutorials/ detailed modalities] for each tutorial
 +
* defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
 +
* creating automatic captioning, sending the captions to the teachers and uploading the edited versions
 +
* dry runs with teachers and SlidesLive people
 +
* archiving pre-recordings or recordings of the live sessions additionally to slides
 +
More details are given in the [https://www.aclweb.org/adminwiki/index.php?title=2020Q3_Reports:_Tutorial_Chairs final report].
 +
 
 +
== Publication of tutorials ==
 +
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.
 +
 
 +
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
 +
* conference website
 +
* ACL Anthology
 +
* private repositories of the authors
 +
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
 +
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.
 +
 
 +
As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].
 +
 
 +
It is recommended for tutorial chairs to:
 +
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
 +
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
 +
* include the initial versions of tutorial slides, together with abstract, in the proceedings
 +
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
  
 
== Organization of past tutorials ==
 
== Organization of past tutorials ==
Line 38: Line 116:
 
|-
 
|-
 
|2020
 
|2020
−
|ACL <br /> EMNLP <br /> AACL
+
|ACL <br /> EMNLP <br /> AACL-IJCNLP <br /> COLING
−
|ACL: Agata Savary (France), Yue Zhang (China) <br /> EMNLP: TBA <br /> AACL: TBA
+
|ACL: Agata Savary (France), Yue Zhang (China) <br /> EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) <br /> AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) <br /> COLING: Daniel Beck (Australia), Lucia Specia (UK)
 
|SoftConf
 
|SoftConf
−
|TBA
+
|43
−
|6 tutorial chairs + external experts
+
|Tutorial chairs + external experts
−
|TBA
+
|3
−
|ACL: TBA <br /> EMNLP: TBA <br />  ACCL: TBA
+
|ACL: 8 <br /> EMNLP: 7 <br />  ACCL-IJCNLP: 6 <br />  COLING: 7
−
|Clarity and preparedness <br /> Novelty or timely character of the topic <br /> Lecturers' experience <br /> Likely audience interest <br /> Open access of the teaching material <br /> Diversity aspects (multilingualism, gender, age and country of teh lecturers) <br /> Compatibility with the required venues
+
|Clarity and preparedness <br /> Novelty or timely character of the topic <br /> Lecturers' experience <br /> Likely audience interest <br /> Open access of the teaching material <br /> Diversity aspects (multilingualism, gender, age and country of the lecturers) <br /> Compatibility with the required venues
−
|TBA
+
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] <br /> [https://www.aclweb.org/adminwiki/index.php?title=2020Q1_Reports:_ACL_2020#Tutorial_Chairs Report] <br /> [https://acl2020.org/program/tutorials/ ACL tutorials] <br /> [https://2020.emnlp.org/tutorials EMNLP tutorials] <br /> [http://aacl2020.org/ ACCL-IJCNLP tutorials] <br /> [https://coling2020.org/pages/tutorials COLING tutorials]
 +
|
 
|-
 
|-
 
|2019
 
|2019
Line 52: Line 131:
 
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) <br /> NAACL: Anoop Sarkar (Canda),  Michael Strube (Germany) <br /> EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
 
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) <br /> NAACL: Anoop Sarkar (Canda),  Michael Strube (Germany) <br /> EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
 
|SoftConf
 
|SoftConf
−
|46-60
+
|45
−
|6 tutorial chairs
+
|Tutorial chairs
 
|2-3
 
|2-3
−
|ACL: 9 <br /> NAACL: 6 <br />  EMNLP: ?
+
|ACL: 9 <br /> NAACL: 6 <br />  EMNLP: 7
 
|Presenters (1-5) <br /> Description (1-5) <br /> Interest Level (1-5) <br /> Preparedness (1-5)
 
|Presenters (1-5) <br /> Description (1-5) <br /> Interest Level (1-5) <br /> Preparedness (1-5)
 
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] <br /> [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] <br /> [https://naacl2019.org/program/tutorials/ NAACL tutorials]
 
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] <br /> [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] <br /> [https://naacl2019.org/program/tutorials/ NAACL tutorials]
Line 61: Line 140:
 
|2018
 
|2018
 
|ACL <br /> NAACL <br /> EMNLP <br /> COLING
 
|ACL <br /> NAACL <br /> EMNLP <br /> COLING
−
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA)
+
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) <br /> COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) <br /> EMNLP: Mausam (India), Lu Wang (USA) <br /> NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
 
|SoftConf
 
|SoftConf
 
|49
 
|49
 
|Tutorial chairs
 
|Tutorial chairs
−
|
+
|2
 
|ACL: 8 <br /> NAACL: 6 <br /> EMNLP: 6 <br /> COLING: 6
 
|ACL: 8 <br /> NAACL: 6 <br /> EMNLP: 6 <br /> COLING: 6
−
|quality <br /> diversity <br /> likely audience interest
+
|Quality of plan <br /> Diversity of topics <br /> Qualifications of presenter <br /> likely audience interest
 
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] <br />
 
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] <br />
 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] <br />  
 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] <br />  
 
[https://acl2018.org/tutorials/ ACL tutorials] <br />
 
[https://acl2018.org/tutorials/ ACL tutorials] <br />
−
[http://naacl2018.org/tutorial.html NAACL tutorials] <br />
 
 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] <br />  
 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] <br />  
 
[https://coling2018.org/tutorial-list/ COLING tutorials]
 
[https://coling2018.org/tutorial-list/ COLING tutorials]
Line 77: Line 155:
 
|2017
 
|2017
 
|ACL <br /> EACL <br /> EMNLP
 
|ACL <br /> EACL <br /> EMNLP
−
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) <br /> EACL: Alex Klementiev (Germany), Lucia Specia (UK) <br /> EMNLP: ?
+
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) <br /> EACL: Alex Klementiev (Germany), Lucia Specia (UK) <br /> EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
 
|SoftConf
 
|SoftConf
 
|26
 
|26
Line 84: Line 162:
 
|ACL: 6 <br /> EACL: 6 <br /> EMNLP: 7
 
|ACL: 6 <br /> EACL: 6 <br /> EMNLP: 7
 
|Clarity (description and outline) <br /> Topic
 
|Clarity (description and outline) <br /> Topic
−
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] <br /> [http://acl2017.org/tutorials/ ACL tutorials] <br /> [http://eacl2017.org/index.php/tutorials EACL tutorials] <br /> [http://eacl2017.org/index.php/tutorials EMNLP tutorials]
+
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] <br /> [http://acl2017.org/tutorials/ ACL tutorials] <br /> [http://eacl2017.org/index.php/tutorials EACL tutorials]
 
|-
 
|-
 
|2016
 
|2016
Line 94: Line 172:
 
|2
 
|2
 
|ACL: 8 <br /> NAACL: 6 <br /> EMNLP: 6
 
|ACL: 8 <br /> NAACL: 6 <br /> EMNLP: 6
−
|Relevance to ACL community (15) <br /> Quality of instructor (15) <br /> Outline and depth/breadth of teh proposal (15) <br />  
+
|Relevance to ACL community (15) <br /> Quality of instructor (15) <br /> Outline and depth/breadth of the proposal (15) <br />  
 
Potential attendance <br /> Newly emerging area not previously covered in an ACL-related tutorial (yes or no) <br /> Introduction into related fields (yes or no) <br />  
 
Potential attendance <br /> Newly emerging area not previously covered in an ACL-related tutorial (yes or no) <br /> Introduction into related fields (yes or no) <br />  
 
Overall score (15) <br /> 1st preference for venue as indicated in proposal <br /> comments
 
Overall score (15) <br /> 1st preference for venue as indicated in proposal <br /> comments
−
|[[https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] <br /> [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] <br /> [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
+
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] <br /> [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] <br /> [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] <br /> [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
 
|}
 
|}
  
 
== Past calls for tutorials ==
 
== Past calls for tutorials ==
 +
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
 
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
 
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
 
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
 
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
Line 108: Line 187:
  
 
== Reports from past tutorial chairs ==
 
== Reports from past tutorial chairs ==
−
*[[2018Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Yoav Artzi and Jacob Eisenstein)
+
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
−
*[[2017Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Maja Popović and Jordan Boyd-Graber)
+
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
 +
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
 +
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
 
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)
 
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)
  
 
== Lessons learned by past tutorial chairs ==
 
== Lessons learned by past tutorial chairs ==
−
* Continue soliciting both cutting edge and introductory tutorials (Marine Capuat, 2019)
+
* [Agata Savary, 2020] If the conference is virtual:
−
* Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies (Marine Capuat, 2019)
+
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
−
* Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report]) (ACL 2018 chairs)
+
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
−
* Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats (ACL 2016 chairs)
+
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
 +
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
 +
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
 +
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
 +
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
 +
** The chairs for each venue select their best choices in a round robin manner.
 +
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
 +
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
 +
** All evaluation criteria, including diversity, in the review form
 +
** Cutting edge vs. introductory category
 +
** Duration (3 or 6 hours)
 +
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
 +
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
 +
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
 +
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
 +
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
 +
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies  
 +
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])  
 +
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats  
  
  
−
Author: Agata Savary, July 2019  
+
Author: Agata Savary, July 2019-July 2020
  
 
Updates are welcome
 
Updates are welcome
  
 
[[Category:Conference Handbook]]
 
[[Category:Conference Handbook]]

Latest revision as of 09:06, 19 August 2021

Duties

Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in here.

Timeline

The timeline for the organisation of tutorials should follow the general conference planning schedule. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the conference planning schedule are highlighted in bold.
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below.
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

  • [T-12]
    • Learn from the general chair which conferences will have their tutorials coordinated with yours.
    • Establish contact with the tutorial chairs of these other conferences, and discuss
      • Recruitment of external reviewers
      • Proposal evaluation criteria
      • Soliciting tutorial proposals from particular potential lecturers on particular topics
      • Organizing a public poll about interest in the submitted tutorial
    • Discuss general strategies of the main conference with the general chair and the program chairs
      • promotion of some topics
      • mailing lists and conference alert portals to be used for all announcements
    • Set up a SoftConf space for proposal submission
      • Solicit the creation of a SoftConf space from support (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
      • If you plan to have external tutorial reviewers, invite them via SoftConf.
      • Set up a submission page.
    • Set up a joint email for tutorial chairs.
    • Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
    • Draft the joint call for tutorials (based on past calls).
  • [T-11] (in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)
    • Send out the call for tutorials.
    • For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
    • If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
    • Set up a SoftConf page for proposal reviews.
  • [T-10]
    • Deadline for tutorial proposal submission.
    • If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
    • Assign proposals to reviewers, send out invitations to review (via SoftConf).
  • [T-9]
    • Select tutorials jointly with other conference chairs.
    • Notify the authors (via SoftConf).
    • Liaise with local chair for space needs and scheduling.
    • Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
    • Liaise the conference website chair for updates about tutorials and authors' guidelines.
    • Liaise with the ACL Office for the registration form.
  • [T-4]
    • Collect short descriptions for proceedings from tutorial teachers.
    • Provide them to the webmaster and the general and program chairs .
  • [T-3]
    • Confirm the tutorials with authors.
    • Authors submit the final tutorial descriptions for the proceedings.
    • Arrange for printing of tutorial materials.
  • [T-2]
    • Arrange with the SoftConf support the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
    • Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.
  • [T-1]
    • Collect and check final tutorial materials.
    • Create the tutorial proceedings, including slides.
  • [T-0] Conference and tutorials
  • [T+1]
    • Invite the lecturers to update the slides at the ACL Anthology via this form (under CC-BY-4.0).
    • Update this tutorial page and the list of past tutorials with the data from your conference. For access to the ACL Conference Handbook, contact the ACL Secretary (currently Shiqi Zhou).

Tutorials in a virtual conference

ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from ICML. SlideLive was the subcontractor for the virtual infrastructure.

The tasks for tutorials included:

  • collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
  • defining time slots which would increase participation across various time zones
  • collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
  • setting up the schedule
  • informing the participants about virtual participation and about the detailed modalities for each tutorial
  • defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
  • creating automatic captioning, sending the captions to the teachers and uploading the edited versions
  • dry runs with teachers and SlidesLive people
  • archiving pre-recordings or recordings of the live sessions additionally to slides

More details are given in the final report.

Publication of tutorials

For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:

  • conference website
  • ACL Anthology
  • private repositories of the authors

The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects mirrors of some of them (but not all). The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many past tutorials, no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides in the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via this form.

It is recommended for tutorial chairs to:

  • announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
  • encourage the authors to also publish, under open licenses, their code and data used in the tutorials
  • include the initial versions of tutorial slides, together with abstract, in the proceedings
  • encourage the tutorial authors to upload the final versions of their slides after the conference via this form

Organization of past tutorials

For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

Year Coordinated conferences Tutorial chairs Submission platform # Submissions Reviewers # Reviews per proposal # Selected tutorials Selection criteria Useful links
2020 ACL
EMNLP
AACL-IJCNLP
COLING 		ACL: Agata Savary (France), Yue Zhang (China)
EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil)
AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA)
COLING: Daniel Beck (Australia), Lucia Specia (UK) 		SoftConf 43 Tutorial chairs + external experts 3 ACL: 8
EMNLP: 7
ACCL-IJCNLP: 6
COLING: 7 		Clarity and preparedness
Novelty or timely character of the topic
Lecturers' experience
Likely audience interest
Open access of the teaching material
Diversity aspects (multilingualism, gender, age and country of the lecturers)
Compatibility with the required venues 		Call
Report
ACL tutorials
EMNLP tutorials
ACCL-IJCNLP tutorials
COLING tutorials
2019 NAACL
ACL
EMNLP 		ACL: Preslav Nakov (Qatar), Alexis Palmer (USA)
NAACL: Anoop Sarkar (Canda), Michael Strube (Germany)
EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia) 		SoftConf 45 Tutorial chairs 2-3 ACL: 9
NAACL: 6
EMNLP: 7 		Presenters (1-5)
Description (1-5)
Interest Level (1-5)
Preparedness (1-5) 		Call
ACL tutorials
NAACL tutorials
2018 ACL
NAACL
EMNLP
COLING 		ACL: Yoav Artzi (USA), Jacob Eisenstein (USA)
COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA)
EMNLP: Mausam (India), Lu Wang (USA)
NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA) 		SoftConf 49 Tutorial chairs 2 ACL: 8
NAACL: 6
EMNLP: 6
COLING: 6 		Quality of plan
Diversity of topics
Qualifications of presenter
likely audience interest 		Call

Report
ACL tutorials
EMNLP tutorials
COLING tutorials

2017 ACL
EACL
EMNLP 		ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA)
EACL: Alex Klementiev (Germany), Lucia Specia (UK)
EMNLP: Nathan Schneider (USA), Alexandra Birch (UK) 		SoftConf 26 Tutorial chairs 3 ACL: 6
EACL: 6
EMNLP: 7 		Clarity (description and outline)
Topic 		Report
ACL tutorials
EACL tutorials
2016 ACL
NAACL
EMNLP 		ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands)
NAACL: Alexander Rush and Bishan Yang
EMNLP: Rebecca Hwa and Mohit Bansal 		SoftConf 32 Tutorial chairs 2 ACL: 8
NAACL: 6
EMNLP: 6 		Relevance to ACL community (15)
Quality of instructor (15)
Outline and depth/breadth of the proposal (15)

Potential attendance
Newly emerging area not previously covered in an ACL-related tutorial (yes or no)
Introduction into related fields (yes or no)
Overall score (15)
1st preference for venue as indicated in proposal
comments

Call
Report
NAACL tutorials
EMNLP tutorials

Past calls for tutorials

Reports from past tutorial chairs

Lessons learned by past tutorial chairs

  • [Agata Savary, 2020] If the conference is virtual:
    • Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also this post).
    • Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
    • If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
  • [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
  • [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
  • [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
    • Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
    • The chairs for each venue select their best choices in a round robin manner.
    • Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
  • [Agata Savary, 2020] While setting up a review form on SoftConf, include:
    • All evaluation criteria, including diversity, in the review form
    • Cutting edge vs. introductory category
    • Duration (3 or 6 hours)
  • [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
  • [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via this form. As a tutorial chair, make sure that
    • initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
    • tutorial authors are encouraged to upload the final versions of their slides after the conference via this form
  • [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
  • [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
  • [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the report)
  • [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats


Author: Agata Savary, July 2019-July 2020

Updates are welcome

Retrieved from "https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook&oldid=74886"