2020Q3 Reports: Diversity & Inclusion (D&I) Chairs

Prelude

Prepared by Cissi Ovesdotter Alm, RIT, and Vinodkumar Prabhakaran, Google, Inc.

D&I team members

Sushant Kafle (Google, Inc.), Masoud Rouhizadeh (JHU), Naomi Saphra (U. Edinburgh), Aakanksha Naik (CMU), Khyathi Chandu (CMU), Emily Prud’hommeaux (BC), Alla Rozovskaya (CUNY), Allyson Ettinger (UChicago), Ryan Georgi (KPMG), Tirthankar Ghosal (IIT Patna), Shruti Palaskar (CMU), Maarten Sap (UW), and Stephen Mayhew (Duolingo).

D&I sponsors

The D&I Champion sponsors were DeepMind and Microsoft and the D&I In-Kind sponsor was Grammarly.

Shift to the virtual setting

With the move to the virtual conference, original plans such as the childcare-related initiatives (provision of onsite childcare, a statement welcoming babywearing, welcome item/activity essentials for kids, Kidz Corner at the welcome reception, family accommodation and potential lending of baby/toddler earmuffs for noise at the social event, etc.), as well as efforts limited to a physical venue (designated D&I spaces, inclusive food options, improving non-alcoholic drink options at events, etc.) were abandoned. The other D&I efforts, involving four subcommittees--Accessibility (Kafle, Rouhizadeh, Saphra), Academic Inclusion (Chandu, Naik, Prud’hommeaux, Rozovskaya), Financial Access (Ettinger, Georgi, Ghosal), and Socio-cultural Inclusion (Palaskar, Sap)--were adapted to the virtual setting or new ones emerged.

This report focuses on the efforts that continued or were introduced under the virtual format.

D&I questions in the registration form

We updated the D&I questions in the registration form. These D&I questions included:

• What career stage do you identify with? (Undergraduate student; Graduate student; Early-career academic researcher; Early-career industry researcher; Senior academic researcher; Senior industry researcher; Other)
• Would you be interested in receiving mentorship at the conference through a virtual platform? (Yes, I would like to attend a group mentoring session for students; Yes, I would like to attend a group mentoring session for early-career researchers; No, does not apply [radio buttons])
• Would you be interested in providing mentorship at the conference through a virtual platform? (Yes, I would be happy to provide mentorship advice to students; Yes, I would be happy to provide mentorship advice to early-career researchers [check boxes])
• Will you need any access services such as sign language interpreting? If so, please describe your needs.
• If you are a presenter and have a hard scheduling constraint (due to caregiving, religious holidays, etc.), please explain.

We considered a question on pronouns however decided against it due to the potential privacy concerns around protecting such sensitive data, once collected. Instead, participants could communicate their pronouns through their RocketChat (or Zoom) profile names, directions for which were included in the instructions to log into the virtual website shared in the Welcome email.

The D&I questions above allowed us to collect information and plan ahead based on data from early registration, which we processed in batches, given that the early registration deadline had to be set later than usual this year.

Registration form recommendations

• We suggest revising the third question above, which aimed to elicit interest in providing mentorship but caused confusion among some registrants who selected “Yes” options to both the second and third questions for similar career stages, requiring extra effort for organizers to identify mentors. (Also, the term “early-career researchers” caused confusion, with answers from undergraduate students, postdoctoral researchers, and associate professors). An alternative wording could be:
  • If you are a (post-PhD) junior or senior researcher, would you be interested in providing mentorship at the conference through a virtual platform? (Yes, I am a junior or senior researcher, and would be happy to provide mentorship advice to students; Yes, I am a senior researcher and would be happy to provide mentorship advice to (post-PhD) early-career researchers [check boxes])
  • Alternatively, consider breaking career stages into more fine-grained options and asking for highest degree completed and years of experience after completing highest degree.

Accessibility

ACL as a community has room to improve on how accessible conferences and the materials they produce are to colleagues with disabilities. We took the following steps in an effort to begin to address some of these issues for ACL 2020.

• A blog post informing authors how to prepare accessible camera-ready papers (led by Sushant Kafle) was published on April 2, 2020; in advance of the camera-ready paper deadline.
• A blog post encouraging authors to prepare accessible presentations (led by Naomi Saphra) was published on May 30, 202; in advance of the deadline for uploading pre-recorded videos to SlidesLive. Both blog posts were appreciated on social media.
• Through regular updates from Priscilla on accessibility requests made during registration, we learned that researchers who used a screen-reader or captioning would attend the conference. (There were also some other requests which were not in the scope of accessibility, e.g., spoken language interpreting). All such requests generally involved individual follow-up to clarify actual needs.
• Sushant Kafle reviewed the virtual website to produce an Accessibility Report for ACL Virtual Conference website, with attention to how well it functioned with screen-reader software. These issues were shared with the infrastructure committee via email and as Github issue tickets.
• Sushant Kafle further compiled a list of Accessibility Issues with Rocket Chat which might need to be taken up with RocketChat for future conferences.
• The originally intended real-time captions (facilitated by Masoud Rouhizadeh) did not apply in the virtual setting. Instead, we advocated for the need for captioning of pre-recorded and live video materials, and continued to provide input to the new captioning chairs, Ananya and Klaus (link to their report). For the virtual conference, CART captioning for live plenaries ensured access for the hard-of-hearing, anyone who could not listen to the audio, and anyone who may have been at a disadvantage due to a language barrier. Authors of pre-recorded talks were also encouraged to correct captions for their talk, which will move into and become part of the ACL Anthology.

Accessibility recommendations

• Continue advocacy for accessible materials (papers, slides, videos, and websites).
• Ensure that a committee member is involved in assessing and providing input about the accessibility of the main conference website from the start (alternative texts for images, color selection, etc.), including for non-virtual conferences.
• Ensure quality captions for both live sessions and pre-recorded videos at virtual and non-virtual conferences. The social media activity on issues with auto-generated captions indicate this is a critical priority for authors generally.
• Elicit requests for access services in registration and ensure accessible websites.
• Ensure accessibility issues in the virtual conference website are fixed (detailed here), if the same codebase from ACL will be used in a future conference.
• Contact RocketChat to resolve the accessibility issues with their platform (detailed here), if their service will be used for future conferences.

Academic inclusion

The academic inclusion mentoring efforts involved four types of events (led by Aakanksha Naik and Khyathi Chandu):

• Panel for undergraduate students (new): This panel, which attracted around 90 participants, discussed topics such as graduate school and degrees, career path options, and also aimed to help undergraduate students navigate the NLP research landscape and the field’s future. Preferences on topics, panelists, and questions were elicited with a survey sent to undergraduate students. The panel was held on June 5th and was moderated by Aakanksha Naik and Khyathi Chandu (Academic Inclusion leads). There were 7 panelists, including faculty, industry researchers, and graduate students. The structure included panelist introductions by moderators, reflections from each panelist on three questions (below), and moderated Q&A with the panelists based on a curated set of 15 questions (from around 75 questions elicited from students):
  • What about NLP/computational linguistics excites you the most? How did you get interested in NLP/computational linguistics and how has that interest (and your career) evolved over the years?
  • What factors guided you in making important career decisions: graduate school, industry vs. academia, etc.
  • What is one thing that you know now that you wish you knew as an undergraduate student?
• Small group mentoring sessions: We organized 82 sessions of pre-assigned small group mentoring sessions for students or early-career researchers (ECR). These sessions were held on July 6, 7, and 8 in the same two time slots: midnight-1am PDT and 9am-10am PDT, aiming for a broad coverage of timezones. There were 944 mentees (571 students and 373 ECRs), 202 mentors (136 mentors for students and 66 mentors for ECRs), of which 34 mentors were ECRs mentees themselves. The session assignments were made using an automated matching process based on time and topic preferences collected through a survey. Each session had 1-3 mentors and approximately 5-13 mentees. The sessions were intentionally kept small in order to encourage more personal interactions that we believe is important to help the mentees, especially the newcomers. These sessions were also supported by 20 volunteer moderators who helped resolve challenges for mentors.
• Birds of a Feather meetups: We organized 29 open sessions over July 6 and 7, led by around 35 senior researchers. These sessions were centered on 23 themes listed in the ACL 2020 call for papers and did not require pre-registration.
• Open group mentoring sessions: Based on the popularity of the small group mentoring sessions (second item above), we organized 9 additional open group mentoring sessions on July 8. These sessions covered 10 popular topics from previous days. 24 mentors quickly signed up in response to an emergency call for mentors that went out on July 7 (PDT). This time, there was no matching process and the sessions could accommodate groups of any size, including attendees joining in late registration. For example, the session on the topic Long-term career planning + Becoming a research leader: building your professional identity attracted over 100 participants.
• Throughout the conference, mentoring events were broadcast in the Socials tab and the #announcement channel (over 20 such announcements) and often also announced by the mentors themselves. We also scheduled social media announcements via publicity chairs for messages to be sent out prior to the Birds of a Feather meetups, etc. We also sent an email to mentors clarifying about not sharing the small group mentoring links (e.g., on RocketChat) given the intended small-group format for those sessions.
• In general, social media activity revealed that the mentoring activities were popular among many attendees.

Academic inclusion recommendations

• For a virtual conference with individual Zoom rooms/accounts, shared accounts must be avoided and new links be provided for each mentoring session. When mentors forgot to log out from ACL-provided accounts after their sessions ended, this caused an access issue for other mentors who tried to use those accounts for a later session.
• Provide guidelines to mentors about recommended communication practices for participants’ interactions, including tools that are accessible across countries (Google applications are not) if mentors wish to pass on materials or connect participants in collaborative writing, to further improve inclusivity. Additionally, consider how ACL can provide such online interaction spaces directly to mentors and mentees. (This point is based on social media comments; thanks to Emily Bender for pointing this out.)
• If feasible, use the same infrastructure for paper live Q&A sessions for group mentoring sessions. This would ensure unique accounts and links for each mentoring group, along with dedicated RocketChat channels, which could be a medium for mentors who want to share materials or surveys with their mentees before or after their session.
• Attendees were not used to 24-hour schedules for virtual conferences, resulting in confusion about the early morning PDT time slots. For instance, many mentors mistakenly signed up for sessions in the 1am - 5am window, since they were not expecting sessions to be held at that hour. This necessitated dealing with a large number of rescheduling requests, adding pressure on coordinators.
• Make efforts to reach students early in their careers, including by scheduling one or more undergraduate student-focused events early in the conference, but also by considering non-conference efforts. Early exposure to research can help widening participation in the field.
• Add group mentoring themes and BoF themes at future conferences that respond to D&I in our community, e.g., Low-resource CL/NLP as a BoF, and Advice on navigating academia as a member of an underrepresented group as a group mentoring theme.
• Recommendations for improved communication with attendees:
  • Release a blog post explaining various types of mentoring events (purpose, logistics, sign-up process, etc.) prior to the start of EMNLP (or another conference's) registrations, so that mentors and mentees have more clarity about these events upfront.
  • Ensure the sign-up deadlines for all events that require registration are clearly highlighted in the conference registration form.
  • Use an ACL-supported mechanism for mass emailing mentoring surveys or group assignments to ensure these emails are not spam-filtered or blocked.
• Exploratory suggestion: It might be helpful to explore a tiered mentorship program; for example, late-stage PhD students acting as mentors to undergraduate and Master’s students, post-PhD junior researchers as mentors to PhD students, and so on. This may strategically increase the number of mentors to satisfy the mentoring demand.

Financial access

• For the various efforts under the financial access subcommittee, a summary is in the blog post titled “Increasing financial accessibility of ACL 2020” (Tirthankar Ghosal, Allyson Ettinger, Ryan Georgi), published on July 4. The blog post was well-received on social media.
• The virtual conference presented an opportunity to cost-effectively support more people to attend the conference, and to coordinate across support initiatives,
• We issued a call for application for D&I subsidies, which was open from May 29 to June 7. We reached out to Black in AI, LatinX in AI, AfricaNLP, and Deep Learning Indaba about the call.
• There were 123 applications submitted. Sub-Saharan Africa (22%), South Asia (20%), US and Canada (18%), and Europe and UK (17%) were most represented, with fewer applications from Middle East and North Africa, Latin America, Southeast Asia, and East Asia.
• Overall, 83 applicants from 29 countries received subsidies for a combination of the conference registration fee, ACL membership fee, and internet bandwidth. We awarded $8750 for registration and membership fees, and another $250 (excluding fund-transfer fees) for internet bandwidth subsidies. Subsidy offers for Sub-Saharan Africa made up around a quarter. Less represented regions included Middle East and North Africa, Latin America, Southeast Asia, East Asia.
• To simplify reimbursement, the registration and membership waivers were provided through the registration form. Priscilla received a list of D&I subsidy recipients.
• The internet bandwidth funds processing (led by Ryan Georgi) used a form to elicit information on how to transfer funds, since options differ by country of residence.
• Additionally, Tirthankar Ghosal led the effort to distribute 1131 access codes provided by Grammarly to the corresponding authors of accepted papers for the main conference, including ACL-Demos, SRW pre-submission phase, the WiNLP workshop, and a subset of other workshops.

Financial access recommendations

• Continue prioritizing outreach to researchers from underrepresented regions and groups.
• Consider new ways for integrating virtual access into our conferences as it has immense potential for broadening participation to diverse audiences.

Socio-cultural inclusion

• The efforts focused on contacting affinity groups in the AI community and offering to facilitate social events, organized by the affinity groups, at the conference (arranging Zoom links, publicizing socials, etc.).
• Two different affinity groups organized and announced events in the Socials tab, etc.:
  • The Queer in AI social was scheduled in two time slots on July 8 (outreach by Maarten Sap), featuring mini-speeches by Elin McCready (Social A), Robyn Speer (Social B) and Alex Hanna (both). There were 44 and 96 registrations for socials A and B, respectively.
  • The Black in AI socials included a Fireside Chat with Donia Scott on July 9 and a social with a presentation by Kianté Brantley on July 10 (outreach by Shruthi Palaskar).
• In addition, we posted a message on #announcements encouraging attendees from various affinity groups to create or join RocketChat channels.
• RocketChat censors words that are on a blocklist, replacing them with ***. We asked the infrastructure chairs (specifically, Hao Fang) to disable this option which seems to be enabled by default. Not disabling this would have been a significant barrier to diversity, due to the blocklist containing identity mentions (e.g., "queer", "lesbian") which made affinity groups (e.g., "Queer in AI") hard to discuss.

Socio-cultural inclusion recommendations

• Since the field involves multiple disciplines, seek to expand outreach through additional affinity groups in relevant disciplines, including for reaching out about financial access subsidies.
• Facilitate that session chairs/moderators have information on names and their pronunciation and, if speakers choose to provide them, the pronouns of people they introduce.
• Disable RocketChat's blocklist “BadWordsFilter” to keep the conference inclusive to marginalized identities (either in the code or in the RocketChat admin under “Message” -> “Allow Message bad words filtering”).

Concluding additional recommendations

• The ACL2020 D&I chairs can be contacted for additional resources that may support planning of D&I activities future virtual or nonvirtual ACL* conferences. Also see the 2020Q1 report.
• Use Microsoft Forms for all surveys to ensure access across countries.
• Return to childcare initiatives when applicable in the future (see outlined list in above and items 10-11 in the 2020Q1 report).
• Make an approved D&I budget part of the basic conference infrastructure. We spent a considerable amount of time iteratively building a D&I budget for standard efforts involving accessibility (backstop items), childcare, and financial access subsidies, etc.