2020Q3 Reports: Student Research Workshop Chairs

From Admin Wiki
Jump to navigation Jump to search

The Student Research Workshop was held in conjunction with ACL 2020. The SRW gives student researchers in Computational Linguistics and Natural Language Processing the opportunity to present their work and receive constructive feedback and mentorship by experienced members of the ACL community.

This report discusses the various aspects of organizing the SRW at ACL 2020, including:

  • Website
  • Mentoring Programs
  • Submissions and Reviews
  • Virtual Conference
  • Funding
  • Recommendations

Organizers

The student co-chairs were:

  • Shruti Rijhwani (Carnegie Mellon University)
  • Jiangming Liu (The University of Edinburgh)
  • Yizhong Wang (University of Washington)
  • Rotem Dror (Israel Institute of Technology)

The faculty advisors were:

  • Omri Abend (Hebrew University of Jerusalem)
  • Sujian Li (Peking University)
  • Zhou Yu (University of California, Davis)

Website

We created a website, which was the primary source of information about the SRW: https://sites.google.com/view/acl20studentresearchworkshop/.

Additionally, we managed a Twitter account for posting updates to social media: (@acl_srw).

The call for papers was publicized through the ACL mailing list as well as the ACL 2020 Twitter account (@acl_meeting).

Mentoring Programs

We offered two mentoring programs this year. Pre-submission mentoring was available to all authors who wanted feedback before submitting their papers to the SRW. Post-acceptance mentoring program was available to authors of accepted papers, to get feedback on their camera-ready drafts and their conference presentations.

Pre-submission mentoring

We followed last year’s SRW in offering an optional round of pre-submission mentoring. This was designed to give students an opportunity to improve their submission, particularly the writing and presentation of the paper, before submitting their papers to the workshop for review.

Authors who submitted their papers in time for the pre-submission deadline received feedback from their assigned mentors before the final submission deadline, giving them time to integrate the mentor’s feedback into their final submission. A total of 57 papers participated in the pre-submission mentoring program. We recruited 30 mentors, all of whom are well-experienced researchers in the field, for the pre-submission program.

Papers were matched to mentors based on research areas. The mentoring feedback was sent to authors anonymously through the SRW organizers. Additionally, the majority of mentors also offered to participate in follow-up discussions with the authors directly via email until the main submission deadline.

Post-acceptance mentoring

Each accepted paper was assigned a post-acceptance mentor, to provide feedback to the authors about their camera-ready drafts and conference presentations (which were pre-recorded this year). 49 mentors were recruited, i.e., one for each accepted paper.

Grammarly Premium

During both the pre-submission and post-acceptance mentoring programs, writing assistance in the form of vouchers for one month's free use of Grammarly Premium was sent to all participating authors. These were provided by the ACL 2020 Diversity and Inclusion Committee.

Submission and Reviewing

Timeline

  • Pre-submission mentoring deadline: January 17, 2020
  • Pre-submission mentoring feedback: February 16, 2020
  • Paper submission deadline: March 6, 2020
  • Acceptance notification: April 17, 2020
  • Camera-ready deadline: May 15, 2020

Submission procedure

The SRW consisted of two submission tracks: research papers and thesis proposals. Research papers were intended to encompass completed work, as well as works-in-progress from graduate students, masters students, and advanced undergraduates. Thesis proposals were intended to be a venue for senior graduate students to get feedback on their thesis proposal and the broader ideas surrounding the appropriateness and impact of their chosen topic. The page limit for both types of submissions was 5 pages of content, with unlimited pages for references. For the camera-ready drafts of accepted papers, one additional page was permitted.

Multiple submissions had to be declared at submission time, and double submission to the ACL SRW and the main ACL conference/other ACL workshops was not allowed. Submissions (in either track) could be archival or non-archival.

Submissions were managed through the START conference system.

Number of submissions

We received 137 submissions in total: 10 thesis proposals and 127 research papers. Among these, 12 research papers were non-archival.

The SRW received submissions from students in 28 countries, with the majority of submissions coming from the United States, China, India, and Japan.

We accepted 49 papers, with an acceptance rate of 36%. After withdrawals and excluding non-archival papers, 42 papers appear in the SRW proceedings, including 6 thesis proposals and 36 research papers.

Because of changes in conference deadlines resulting from the COVID-19 pandemic, several authors requested a change from the non-archival to archival track. The SRW does not have a specific distribution requirement with respect to each track, and given the unprecedented circumstances, these changes were permitted.

All accepted papers (archival and non-archival) were presented at the conference.

Program committee

We were fortunate to recruit 262 members for the SRW Program Committee. Each paper was assigned 3 reviewers. No reviewer was assigned more than 2 papers.

Reviewing

We used the “keyword bidding” capability on the START system, where reviewers could indicate their research areas of expertise. Submissions were then automatically assigned to reviewers based on these bids. All assignments were manually verified (and modified if needed) by the SRW co-chairs.

We based the review form on the ACL 2020 format, removing the questions about presentation type and best paper award: https://acl2020.org/_pages/docs/ACL_2020_Submission_Form.pdf

Virtual Conference

The SRW papers are presented in conjunction with other ACL main conference papers from July 6 to July 8. All the papers accepted to the SRW i.e., research papers and thesis proposals in both the archival and non-archival tracks, were presented at the conference.

For the virtual presentations, we adopted the same format as the main conference, which featured pre-recorded talks, live Q&A sessions, and online messaging channels.

The SRW is traditionally held as a poster session at the same time and venue as that of the main conference. We tried to emulate this setting in order to allow the student authors of the SRW to receive similar exposure and feedback as they would during an in-person conference. In coordination with the ACL Program Chairs, the Virtual Infrastructure Chairs, and the Publication Chairs, the SRW was conducted as a track within the main virtual conference:

  • The virtual ACL website had the SRW papers in a separate track, listed alongside those of the main conference and system demonstrations.
  • The live Q&A sessions for SRW papers were in the same time slots as the papers of the main conference.
  • The schedule for the main conference included the SRW papers.

SlidesLive was used to host the pre-recorded presentations. Authors were required to record a 12 minute video for each paper.

Each paper was assigned two one-hour live Q&A sessions during the conference. Since the authors are located in different time zones, we conducted a survey about their available time slots, and tried to schedule the Q&A sessions according to their preferences.

Additionally, during the conference, each paper had its own RocketChat channel, for asynchronous Q&A and messages between authors and conference attendees.

Funding

The SRW received $15,000 USD in funding from the National Science Foundation and $10,000 USD from the Don and Betty Walker Scholarship Fund.

We worked together with the Student Volunteer Coordinator to ensure that we financially support as many students as possible. We encouraged all SRW authors to apply for the volunteer program, and receive free ACL membership and conference registration.

We also coordinated with the Diversity and Inclusion Committee, which had its own financial assistance program, to ensure that available funds are optimally distributed.

Since the conference was completely virtual, the only expenses for students were the ACL membership fee and the conference registration fee. We waived the conference registration fee for all SRW authors who applied for financial assistance. Additionally, the membership fee was waived for all authors who volunteered at the conference.

Recommendations

  • As in previous years, we managed the pre-submission mentoring program via email and online forms. However, we found the process of keeping track of emails and downloads quite tedious. We recommend that future SRWs use the START system (or equivalent) for pre-submission mentoring feedback, while giving mentors the option to offer additional feedback to the authors directly via email.
  • Post-acceptance mentoring was conducted by connecting the authors with the mentors via email, with encouragement to the authors to send their camera-ready drafts and presentations. While we received feedback from multiple authors that this program was very useful to them, some mentors informed us that the authors never communicated with them. Future SRWs can consider a more formal system of sending camera-ready drafts to the mentors and facilitating discussions between mentors and authors.
  • We conducted separate recruiting processes for each of the mentoring programs and the program committee. However, several researchers participated in all three groups, which leads us to believe that a single recruitment process will likely be more efficient in future SRWs.

Additional recommendations are in the report from the faculty advisors to the SRW.

Retrieved from "https://www.aclweb.org/adminwiki/index.php?title=2020Q3_Reports:_Student_Research_Workshop_Chairs&oldid=73977"