2008Q3 Reports: ACL 2008
Jump to navigation
Jump to search
ACL08:HLT Conference Chair Report Kathleen McKeown Columbia University This year's conference, ACL:HLT08, is jointly sponsored by the Association for Computational Linguistics and the North American Chapter of the Association for Computational Linguistics and it thus brings together the traditions of both organizations. As is evident from the title, one of those traditions is the focus on research from all areas of Human Language Technology, including information retrieval, natural language processing and speech. The conference features invited speakers in speech and information retrieval and there are sessions devoted to all three of these areas. Since I was last involved in organizing the ACL Conferences back in the 90's, the conferences have grown dramatically. This year there were eight committees involved in the organization, most of which had two or more chairs. This included the program committee, tutorial organizers, student research workshop, demos organizers, publications, publicity and sponsorship. For committees that were involved in putting together the various aspects of the program, we had co-chairs drawn from the three main areas of the conference. In order to facilitate organization, in most cases, one chair was designated as the main coordinator and that person drove the organization of their committee, was my main contact, and communicated with the other co-chairs. This organization worked quite well. I was quite impressed with the energy and commitment of the different chairs. This made my job much easier. One main issue that I think should be seriously considered for future years of NAACL is the inclusion of speech and information retrieval. The intention was to have a conference that is equally balanced between all three areas. Submissions from the speech and IR area are low, however. Thus, for the program committee, the workload is very unevenly balanced between the co-program chairs. The vast majority of submissions came in NLP and thus the NLP chairs were more heavily burdened, particularly in the final week of decision making. Similarly, for tutorials and demos, while there is the intent to equally balance the offerings between the different areas, it is not clear whether either submissions or selection (of tutorials) is equal in all areas. For future years, NAACL must either do something different to raise participation from the other communities or should change the structure of the committees to reflect what actually happens. I don't think publicity is enough. This year we had three publicity chairs, one for each area. They did a great job in posting the event, developing a flyer and bringing it to other conferences. It is possible that that work could begin earlier, which would mean selecting those chairs in summer rather than fall, as I did. I believe, however, that there needs to be some additional coordinated event that involves invitations to members of the community in order to draw more participation. There were several other changes that we tried this year. A new plan for sponsorship was developed that would enable coordination among different ACL Conferences and across the years. This plan was finalized during the summer and thus, relatively late in the process of selecting chairs. It is an ambitious plan and we are fortunate to have one person who has agreed to serve in a multi-year, multi-conference capacity. However, finding people who are willing to serve in this larger capacity is difficult. I think the next conference chair needs to start finding people early for this task and the ACL should think about how to make the task more rewarding. We also instituted a change this year in the number of publication chairs. In previous years, there was only one and this year, we expanded that number to two. I think this change worked well. The task was divided by proceedings with Joakim Nivre handling the main proceedings and Noah Smith handling the workshop proceedings. Given the number of workshops, I think an extra person was needed to handle this. We also made it possible for them to find help within their local institutions, something that is also needed for this particular job. There was a fair amount of communication between the publication chairs to various other chairs and the publishers. I found that it seamlessly switched between Joakim and Noah and that when one was unavailable, the other would easily substitute. So I would recommend again having two publications chairs in future years. This year the number of student paper submissions was lower than usual and for future years, I would recommend asking the student research chairs to think about how to publicize. This was unexpected this year so we did not think of doing this ahead of time. Faculty at different universities should keep in mind that with a lower number of submissions, there is a better chance for the students to have papers accepted. The student papers are presented in a parallel track as are all other papers and students will have equal exposure. In conclusion, I want to thank the many people involved. I thank the committee chairs that I asked for agreeing to serve and for showing so much responsibility. I also think local arrangements deserves a special thanks for putting together such a nice event. ACL-08: HLT Program Chairs Report The program for ACL-08: HLT features a wide variety of avenues for authors to present their latest work in computational linguistics, information retrieval, and speech technology. The program includes: full papers, short papers, posters, demonstrations, and a student research workshop, as well as pre- and post-conference tutorials and workshops. In our program design, we attempted to combine the successful approach of ACL07, which had four parallel oral sessions of 25-min full paper presentations, with the HLT model of presenting late-breaking results in parallel sessions of 15-min short paper presentations. We also experimented with an idea adopted from Interspeech, in which authors can choose their desired mode of presentation, oral or poster, based on their assessment of how best to present their work. There is no distinction between posters and oral presentations in terms of quality or in terms of how they appear in the Proceedings. Although it will take more than one year to see this change fully taken up by the membership, we were happy to see some authors choose the poster option from the very outset. Area chairs also used their discretion in indicating which submissions would benefit from which mode of presentation. If the number of submissions continues to grow as it has done in the past few years, poster sessions will be one way to managing this growth without creating a large number of parallel sessions. This year, the program committee received yet another record-breaking number of submissions, with 470 full and 275 short paper submissions. Full papers were due in mid-January, and the program committee accepted 119 (25%) of these, 95 as oral presentations and 24 as posters. Short papers were due in mid-March, and the committee accepted 64 (23%) of these, 32 for oral presentation and 32 for poster presentation. (A breakdown by area is included at the end of this report.) We recruited 34 area chairs who recruited 720 reviewers, and who managed the reviewing process of both full and short papers in their areas. Reviewers wrote three reviews for each full paper submission, and two reviews for each short paper submission, for a staggering total of just under 2000 reviews! Miraculously, there were only a handful of late reviews. As the number of submissions and, consequently the number of area chairs, has risen over the last few years, the ACL program committee has moved away from having a face-to-face meeting of all area chairs. For ACL08: HLT, two of the program co-chairs met for two days at Edinburgh University, using email and teleconferencing to get input from the two program co-chairs not based in Europe, and all of the area chairs. For short paper decision making, three of the four program co-chairs held a teleconference, with input from the fourth co-chair by email as time zone differences permitted. Another first this year was our decision to award several outstanding paper prizes, rather than trying to identify a single best paper. We did this because we felt that it is typical for conferences as large as this to have several particularly exciting, innovative, and well-crafted papers, and it is extremely difficult to compare quality across areas. We asked area chairs to nominate papers for the various awards and then formed an Outstanding Paper Committee, who wish to remain anonymous, and to whom we owe a great debt of gratitude for their hard work at very short notice. There were three things that made our job possible. First, we were helped immensely by Jason Eisner, who has compiled an excellent web site on "How to Serve as Program Chair of a Conference" (http://www.cs.jhu.edu/~jason/advice/how-to-chair-a-conference.html). This saved us more than once! Second, we employed a recent PhD, James Clarke, to help us get started with START, and to simply deal with the large volume of work that must be processed within the first few days after submissions are received. Third, there is the invaluable START system for managing paper submission, reviewing, and decision making. Rich Gerber and the START team responded to questions/problems quickly, even modifying START overnight to provide functionality we asked for. Submissions and acceptances by area: LONG Papers #subm #accept #oral -------------------------------------------------------- Dialogue 16 4 (25%) 3 (19%) Discourse 12 3 (25%) 3 (25%) Eval 13 3 (23%) 3 (23%) IE 34 8 (24%) 7 (21%) IR 28 9 (32%) 8 (29%) LanguageGen 13 3 (23%) 3 (23%) LanguageRes 12 3 (25%) 2 (17%) MT 87 20 (23%) 15 (17%) Multimodal 5 2 (40%) 2 (40%) Phon/Morph 32 8 (25%) 7 (22%) QA 11 13 (27%) 2 (18%) Semantics 63 18 (29%) 13 (21%) Sentiment 20 3 (15%) 3 (15%) Speech 13 4 (31%) 4 (31%) Stat ML 42 9 (22%) 6 (14%) Summarisation 21 5 (24%) 3 (14%) Syntax 54 14 (26%) 12 (22%) -------------------------------------------------------- total 476 119 (25%) 96 (20%) SHORT Papers #subm #accept #oral ------------------------------------------------------ Dialogue 12 4 (33%) 1 (8%) Discourse 7 3 (43%) 1 (14%) Evaluation 10 2 (20%) 1 (10%) IE 18 4 (22%) 1 ( 6%) IR 19 4 (21%) 3 (16%) QA 4 1 (25%) 1 (25%) Sentiment 18 4 (22%) 1 ( 6%) Lang res 5 1 (20%) 1 (20%) NLG 5 2 (40%) 2 (40%) MT 42 10 (24%) 4 (10%) Multimodal 4 0 (0%) 0 (0%) Phon/Morph 12 4 (33%) 1 ( 8%) Semantics 30 6 (20%) 5 (17%) Speech 15 4 (26%) 3 (20%) Stat ML 33 7 (21%) 1 ( 3%) Summ 20 5 (25%) 1 ( 5%) Syntax 21 7 (33%) 5 (24%) ----------------------------------------------------- 275 68 (25%) 32 (12%) Johanna Moore, Simone Teufel, James Allan, Sadaoki Furui ACL-08:HLT Program Committee Chairs Report from ACL-HLT 2008 Workshop chair: The committee consisted of myself, Helen Meng, Chengxiang Zhai. We received 14 applications for workshops. The committee chose 11 Workshops including 2 two-day workshops and 9 one-day workshops. From the ratio of acceptance, the selection process is not very competitive. The review process was very simple: First the reviews were made by the committee members independently and then group discussion was made for the unsure cases. The dropped 3 applications were primarily due to the mismatch of the topics. For instance, some applications were not core-CL related. After the paper submission, we decide to cancel ACL08-Semantic-Evaluation based on the email by the organizer Eneko Agirre who reported just one submission was received. The final list of workshops are: Two days: 1. SIGDIAL, ACL08-SIGDIAL (two days) 2. The Third Workshop on Issues in Teaching Computational Linguistics, ACL08-Teaching-CL, (two days) One-day: 1. Third Workshop on Statistical Machine Translation, ACL08-SMT 2. SSST-2: Second Workshop on Syntax and Structure in Statistical Translation, ACL08-SSST 3. Software engineering, testing, and quality assurance for natural language processing, ACL08-NLP-Software 4. BioNLP 2008, ACL08-BioNLP 5. Computational Morphology and Phonology(SIGMORPHON), ACL08-SIGMORPHON, 6. Natural language processing and mobile devices, ACL08-Mobile-NLP 7. The 4th Workshop on Innovative Use of NLP for Building Educational Applications, ACL08-NLP-Education 8. Workshop on German Parsing, ACL08-German-Parsing I want to give special thanks to Chris Brew, Priscilla Rasmussen, Kathy Mckeown, Simone Teufel who provided fast and professional support for the WS organization. Ming Zhou, June 1st, 2008 This year we received 13 submissions for tutorial proposals. Of these, 3 covered topics on machine learning with applications to natural language processing, 5 dealt with information retrieval/information extraction/data mining, 3 were related to speech processing or dialogue systems. Some of the submissions were very similar to other tutorials given in past ACL and NAACL conferences. While we tried to avoid repetition of topics, the question came up of whether it is a good idea to accept "repeats" in cases where there was a big audience at the previous offering. Having an explicit agreement about this can facilitate the decision process for future tutorial chairs. After considering the various constraints: clarity of the proposal, covering of NLP, IR and speech, offerings in previous years, 6 tutorials were selected for presentation. T1: Introduction to Computational Advertising T2: Building Practical Spoken Dialog Systems T3: Semi-supervised Learning for Natural Language Processing T4: Advanced Online Learning for Natural Language Processing T5: Speech technology from research to industry T6: Interactive Visualization for Computational Linguistics Covering the speech area tutorials was most problematic as there were fewer submissions. We intentionally chose two---one for the morning (T2 on dialogue) and one for the afternoon session (T5 focusing more on underlying technologies for spoken language-based applications). In previous years the speech tutorials were often not very popular, sometimes because of coinciding with other highly popular tutorials. The offerings of two very interesting presentation this year would make possible to assess if the problem is general lack of interest. If this is the case, it will be good to discuss what can be done differently in future years. Core information retrieval proposal (as opposed to information extraction) were also not many. For both Speech and IR tutorials the chairs solicited proposals personally, beyond the general advertising of the conference. Some of these were rejected eventually, but the above is worth mentioning so that the number of submissions does not seem misleading. In printing the one page abstracts for the proceedings, it will be useful for future chairs to provide a sample style file with the necessary section titles and formatting in order to achieve consistencies. When given instructions, regardless of how clear they might, different people come up with different interpretations. ACL:HLT 2008 Tutorial Chairs Ani Nenkova Marilyn Walker Eugene Agichtein ACL-08: HLT --- Demo Chair Report (Jimmy Lin, University of Maryland) As the demo chair for the conference, I was responsible for arranging the demo program. This year, 21 submissions were received, 9 of which were selected for inclusion in the program after review by at least two members of the program committee. Care was taken to ensure a balanced demo program encompassing as many areas of HLT as possible. ACL-08: HLT Publications Co-Chairs Report As publications co-chairs for ACL-08: HLT, we have been responsible for producing the conference proceedings (in hardcopy, on DVD, and for the ACL Anthology) and for processing copyright transfer agreements. Below we report on the work done and suggest improvements for the future. 1. Preparatory work In order to facilitate the work on the proceedings, both for ourselves and for authors, we made several revisions to the style files and formatting guidelines in order to (a) make some of the more common space-saving devices part of the official guidelines, and (b) make the style files consistent with the guidelines. In addition, we promoted the idea, pioneered by Jason Eisner at last year's EMNLP-CoNLL, of allowing 8 pages of content + an unlimited number of pages for references, in order to encourage thorough citation. In the end, the limit for long papers (and for many of the workshops) was set at 8 pages of content + 1 page of references. We think this was a perfectly reasonable compromise for ACL-08: HLT, but we see no reason to restrict the length of the bibliography for future conferences, given that the demand for printed proceedings seems to be declining steadily. The revised style files and guidelines will be included in the ACLPUB repository for the benefit of future publications chairs. We also tried to make sure that important revisions to the ACLPUB software, made by the chairs of previous years, were actually committed into the repository. We are especially grateful to Eric Ringger and Jason Eisner for contributing several important improvements during this conference cycle that they had made previously. During the whole process we have fixed a number of bugs ourselves, and improved the functionality for converting LaTeX to HTML for the DVD. All these changes have been committed to the ACLPUB repository. 2. Main conference publications The main conference proceedings are divided into five logical books: main proceedings (containing all long papers), short papers, SRW, demos, and tutorial abstracts. When printed, the last four of these are bound together in a single companion volume. The main proceedings and short papers were handled by the publications co-chairs themselves. This work involved the usual fixing of papers that were not correctly formatted or did not have all fonts embedded. We decided not to be overly strict in the application of formatting guidelines, which resulted in 25 out of 119 long papers and 9 out of 68 short papers having to be sent back to the authors for correction. (Applying the guidelines strictly would have meant sending back more than two thirds of the papers.) Given that more than 90% of the problematic papers were produced using Word, rather than LaTeX, it is worth discussing whether we should provide Word document templates in addition to LaTeX style files in the future. A key issue here is whether adequate support for all character sets exists for LaTeX. The SRW and demos were handled by the respective chairs, while the tutorial abstracts were handled jointly by the publications co-chairs and the tutorials chairs. Overall, these volumes caused relatively few problems, but we question whether it is worth the effort to produce a 15 page volume containing the one-page tutorial abstracts. If this practice is continued in the future, this should be the responsibility of the tutorials chairs. Providing templates to the tutorial presenters will also make this a simpler job. 3. Workshop publications All ten workshops have printed proceedings. One workshop decided to take care of their own printing to allow for a later submission deadline for camera-ready papers. Unfortunately, this also meant that the proceedings of that workshop is not available on the DVD (although it will of course be available in the ACL Anthology). In the end, the difference between the two deadlines turned out to be less than two weeks, and we believe that it should be possible in the future to get all workshop proceedings printed in the same process and included on the DVD provided that deadlines are fixed earlier. The remaining nine workshop proceedings were produced by the respective chairs and delivered to the publications co-chairs. Despite some minor technical problems with the installation of the ACLPUB software on different sites, the production of each individual book went rather smoothly. However, a substantial amount of work had to be done afterwards to achieve consistency with respect to title pages, spine texts, citation stamps and pdf meta-data. We believe that some of this work can be avoided in the future with a slightly different work flow, where the publications chair provide all book chairs with customized templates. 4. DVD Generating the content for the DVD from the (files for the) printed version of the proceedings is largely automated by the ACLPUB software. However, it is our impression that this part of the ACLPUB package is less mature than the tools for producing the proceedings, and we had to make several improvements especially for the generation of the program and table of contents for the DVD. 5. Future work We believe that some of the problems we encountered along the way could have been avoided if the instructions for publications chairs in the conference handbook had been more complete and up to date. This should not be seen as criticism of our predecessors, only as recognition of the fact that the whole publication process has changed with the advent of the ACLPUB package, and that it has not been possible to update the documentation at the same pace. We therefore intend to create a "publications chair instructions" document, which will basically recapitulate what we did and what advice we have about each step. This will be included in the ACLPUB repository but can also be integrated into the conference handbook, perhaps in condensed form. The most important thing we want to leave behind is a timeline that eliminates all of the surprises and helps the next publications chair to know what he/she is in for. Joakim Nivre and Noah A. Smith Uppsala and Pittsburgh, May 2008 Report for ACL 2008 Student Research Workshop  Student co-chairs: Ebru Arisoy <arisoyeb@boun.edu.tr>, Bogazici University, Bebek, Istanbul, Turkey Wolfgang Maier <wo.maier@uni-tuebingen.de>, University of Tuebingen, Tuebingen, Germany Keisuke Inoue <kinoue@syr.edu>, Syracuse University, Syracuse, New York, USA Faculty Advisor: Jan Wiebe <wiebe@cs.pitt.edu>, University of Pittsburgh, Pittsburgh, Pennsylvania, USA 1. Program Committee The co-chairs of the ACL 2008 Student Research Workshop are Ebru Arisoy (Bogazici University, Turkey), Wolfgang Maier (University of Tuebingen, Germany) and Keisuke Inoue (Syracuse University, USA). Jan Wiebe (University of Pittsburgh, USA) is the faculty advisor. The program committee is formed by the co-chairs by asking previous SRW reviewers, previous SRW participants and other researchers from the community. The final program committee consists of 61 reviewers, of which 27 are students or young researchers and 34 are senior researchers. 2. Paper Submission and Acceptance  We received 27 submissions from 10 countries (see Table 1). All papers were assigned 5 reviewers (at least 1 senior and 1 student reviewer). We accepted 12 papers, of which 5 are regular (oral) presentations and 7 are posters. Country Submission Accepted ------------+------------+--------- Australia 1 1 China 1 1 France 3 1 Germany 1 0 India 2 0 Japan 2 1 Korea 1 0 Spain 1 0 Switzerland 1 0 UK 2 1 US 12 7 ----------------------------------- Total: 27 12 Table 1: Submission and acceptance by countries 3. Presentation Format The Student Research Workshop poster session is colocated with the main conference poster session on Day 1 of ACL (June 16, 2008). The five oral presentations are held in a single session parallel to the last main conference sessions on the same day. Each oral presentation consists of 15 minutes of talk, and 5 minutes each for panelist feedback and for general audience questions. 4. Panelists The co-chairs asked senior conference attendees to be on the panel to provide feedback to student authors. The panelists were selected for their knowledge in the area and availability during the workshop. At the time of this report, most papers have been assigned at least one panelist. Further positive responses are still expected. 5. Funding We submitted our request to the National Science Foundation in January 2008 and received the award in April 2008. The grant totaled $18,180, of which $15,950 was budgeted for student travel, registration, accommodation and meals. The remaining $2,230 will be used to cover the student co-chairs registration and to partially defray costs associated with the student research workshop such as refreshments during the breaks. We will be able to provide funds to every SRW participant. Because the cost of traveling to Columbus varies depending on the student's location, the level of funding is determined based on the cost of travel. We will award a guaranteed $305 in funds to students from Columbus, $1,505-$1,705 to students from Europe and $3,105 to students from Australia. In the distribution of the grant, we aim to cover student's expenses for airfare, ACL registration, and membership fee. We also aim to contribute toward 4 nights stay in a shared room in one of the conference hotels and meals during the conference. 6. Organization and Planning The Workshop was publicized by sending CFPs to mailing lists of computational linguistics and related fields. The Workshop webpage was placed prominently on the main conference website. We are grateful to the webmaster for swift updating on request. In addition, the ACL Newsletters helped to disseminate information on the Student Research Workshop. We are grateful to the main conference organizers for the support. We used the START system and Google Docs to manage the submission and review process. Both proved immensely helpful for managing the coordination among 27 submissions and 61 reviewers. None of the submissions showed formal problems, i.e., there were no double submissions without indication, all copyright forms were correctly signed, etc. 7. Suggestions and Considerations a) We believe that the success of the Student Research Workshop depends on the quality of the reviewer and panelist feedback to students. We were happy to find 61 reviewers and 12 panelists who are supportive of this educational goal. The community was very responsive, which shows that the Student Research Workshop is widely accepted and recognized. Also for students whose work could not be accepted, the elaborate reviews will be of great help. We recommend that future Workshop organizers continue the tradition of concentrating their efforts on assembling good reviewers and panelists. b) For many Workshop presenters, this is their first major conference attendance. Therefore, we thought it would be beneficial for students if we could arrange their poster/regular sessions early during the conference, such that they can begin networking and get the most out of the duration of the conference. We suggest that future Workshop planners communicate with the main conference organizers in the early stages of planning to ensure that the logistics for this situation work out. Both oral and poster presentations were scheduled and located to make the SRW look like a part of the main conference rather than a separate event. We believe this is beneficial for the students, as they get more attention from the general audience. c) This year, the submission deadline as well as the notification of acceptance for the SRW was set to be at the same time as that of the main conference. In the call for papers we clearly state that the students should indicate if a paper has been submitted to another conference or workshop. We did not receive papers that indicated double submissions. We also manually searched the authors of the accepted papers and their previous publications on the web to make sure that their papers are unpublished and original works. d) Since our funding came from the National Science Foundation, in order for students to be reimbursed for their airfare their flights have to comply with the Fly America Act. This essentially means that the flights have to be on a U.S. carrier or code-shared by a U.S. carrier, unless no such options are available (there are a few exceptions, primarily when the travel time would be dramatically increased). We announced this fact to the students in a few weeks after the notification of acceptance. Since the workshop will be held in the USA, students won't have any problems to find flights from U.S. carriers. ACL 2008 Sponsorship Report Based on email discussions with officers of the ACL, NAACL, and EACL, the Sponsorship Chairs developed a sponsorshop prospecturs, located at: http://www.ling.ohio-state.edu/acl08/files/ACLSponsorshipProspectus2008-2009.pdf. The total amount of sponsorship received was $33,500. The sponsorship details are as folows: Gold Level: Ohio State University. Silver Level: BBN Technologies, Google, Microsoft Research, PowerSet. Bronze Level: Center for Next Generation Localisation. Supporter: Thomson Reuters. In addition, IBM Yorktown Heights supported the Best Student Paper to the tune of $1000.Cincinatti Children's Hospital supported $700 for the reception and $500 for the keynote speaker. Google and PowerSet also sponsored booths. LanguageWeaver sponsored a conference bag. The University of Michigan School of Information Sciences sponsored an ad in the brochure along with a conference bag insert. Thomson Reuters also sponsored a conference bag insert. For a complete list of sponsors, please visit this webpage http://www.ling.ohio-state.edu/acl08/sponsors.html The Sponsorship Chairs: Inderjeet Mani imani@mitre.org Michael White mwhite@ling.osu.edu Josef van Genabith josef@computing.dcu.ie Publicity for ACL 2008 (Diane Kelly, Hal Daume III, Eric Fosler-Lussier) This year, we tried to extend the reach of ACL by not only advertising to the NLP community, but also the IR, ML, and Speech communities as well. We made an 8.5 x 11 printable flyer, which was put up at NIPS, IEEE ASRU, Digital Libraries, several research labs, and the SIGIR PC meeting. The flyer and CFP were also distributed on several mailing lists (ACL, IR, Digital Libraries, Info Science, several machine learning/statistics lists), as well as mailings to many of the major labs. One issue that came up is that it would be good to coordinate with the IEEE Speech and Language Technical Committee on cross-advertising of events. It might be a positive development if some deal could be worked out where events that might interest the other community are advertised via the other group's mailing list(s) as related events.