2022Q1 Reports: Ethics Committee Co-chairs-Draft

From Admin Wiki
Jump to navigation Jump to search

_N.B.: This document is authored expressly for purpose of informing the ACL Executive Board as required for the Quarter 1 2022 documentation. It is meant to be read by the ACL Executive Board and is also suitable for the ACL membership and the general public.

General

The Ethics Committee was established by the ACL on October 6, 2021 [1], appointing a nine person team: three co-chairs with a five-year term, one from each major region of the ACL membership; and six members, each with a three-year term. Effort was made to balance both gender and regional representation. The current membership is:

  • Luciana Benotti (Americas)
  • Mark Dredze (Americas)
  • Karën Fort (Europe, Co-chair)
  • Pascale Fung (Asia)
  • Dirk Hovy (Europe)
  • Min-Yen Kan (Asia, Co-chair)
  • Jin-Dong Kim (Asia)
  • Malvina Nissim (Europe)
  • Yulia Tsvetkov (Americas, Co-chair)

Since the establishment of the committee, the chairs have held their own biweekly meetings to coordinate work, and the committee as a whole has convened monthly meetings to ensure targets are met.

Efforts

The Ethics Committee has made progress on a number of internal initiatives, largely organised around delegated subcommittees to make progress. These include

Ethics Survey

The AEC created and fielded a ACL membership-wide survey to capture ACL members' opinions about CL/NLP ethics. The AEC felt that this work was a key area to establish the foundation of the AEC, as we hope to best understand and serve our membership. While ethical conduct is a requirement in both professional and research committees – such that an ethics committee such as this AEC are necessary — we still sought to better understand the current opinions and understandings that the membership has of this requirement.

Edited over the October—December period of 2021, the survey was launched in mid-January 2021, and was open for a period of approximately five weeks. The survey's eight questions were designed to be short and non-leading as much as possible. Each question contained a structured component (multiple-choice question) for collecting aggregate statistics and an option to elaborate in more detail. The committee will produce a report to be presented at a suitable venue later in 2022, and archived within the AEC's public website in due course.

Yulia Tsvetkov and Karën Fort co-chair this subcommittee.

Ethical Reviewing Guidelines

Harmonized ethical reviewing guidelines are a core responsibility of the ARR. The AEC created three separate guidelines to facilitate ethics review, according to role:

  • Author: these are the primary guidelines for authors to consider in their work. Importantly, the guidelines do not specify a checklist approach as it is important for authors to consider ethics holistically. That is, authors blindly following a checklist may indeed validate their approach, yet fail to properly account for ethical issues due to their adherence to the checklist rather than following the overarching goals.
  • Reviewer: these guidelines guide reviewers to look for common ethical issues that occur in papers. It give examples of hypothetical ethical situations and how these may be a problem that requires further explanation or reconsideration that reviewers may want to flag authors for.
  • Organizer: these guidelines offer event organizers a set of recommendations to consider implementing. These concern the review timeline, and the organizational structure of the ethical reviewing pool, and the interface between ethics reviewing and other logistical and organization considerations.

In constructing the guidelines, the subcommittee aligned ethics guidelines from EACL, NAACL and ACL-ICJNLP 2021 and distilled common practices among the events. To be clear, the guidelines are suggestions and not mandates. To help with this, the documents distinguish between common practices and others that are controversial or elective.

Also, it was decided that the AEC will not be an operational arm for ethics review. That is, the AEC will not be involved in actual operationalization of ethics reviewing, nor will it keep a reviewing pool for events to draw on. Rather, the AEC's guidelines — as well as its committee members — are available for event organizers to consult. The AEC takes a long-term, strategic stance on proper ethical reviewing.

ACL Rolling Review (ARR) is a key venue for *ACL events, following the Exec's agreement to use ARR to channel submissions for all major *ACL conferences and workshops. The ARR ethical review guidelines are thus of central importance. As such the AEC hopes to work closely with the ARR to develop its guidelines as a standard bearer for NLP/CL ethical review. We note that ARR is headed by a separate, autonomous committee, so is not beholden or subordinate to AEC, but rather working together as a mutual partner on ethics.

This subcommittee is chaired by Yulia Tsvetkov, assisted by Karën Fort.

Ethics Education and Outreach

We noticed that the NLP membership expressed the need to better understand what ethics is and more specifically how to write an ethical considerations section. This subcommittee works towards these providing material to help "educating" and raising awareness in the community, primarily through events and outreach as well as through particular static public resources.

Events

The subcommittee has started brainstorming on means to help bring ethics awareness to our ACL membership more broadly. These efforts are likely keynote speeches, panels, workshops and tutorials at *ACL and sister organization events. As the subcommittee feels that pre-recorded tutorials and videos may also be effective and complement synchronous and in-place physical events, we plan to start a video series that may feature authors who tackle ethical issues, or take corrective action on their own previous work that may have had ethical issues.

The report on the ACL membership on ethics survey, is a likely target for dissemination at a suitable event, when suitable analytics has been completed after the survey's closing.

Ethics Union Bibliography

This reading list is a union bibliography of works related to CL/NLP ethical concerns. It is minimally structured as a Markdown file and hosted on Github to permit easy suggestions through a pull request to us (as the maintainers). Currently the bibliography structures resources taken from committee members' courses and their own reference bibliographies.

This subcommittee is chaired by Min-Yen Kan and Karën Fort

Committee Scope

The responsibility and the definition of ethics is construed differently among parties and the difference between scientific integrity, deontology and ethics is not recognized by all. This subcommittee works towards a clearer definition of the scope and responsibilities among the AEC and other internal committees within the ACL.

The committee's current understanding is that the ACL Exec does not have a specific scope for the AEC and in some cases, also not for other subcommittees of the ACL, passing the decision of scope to the subcommittees themselves. This subcommittee thus is working towards crafting an appropriate scope for the AEC's work.

As such, the AEC decided that the scope of the committee would largely be looking towards long-term strategic goals, instead of as an operational team to handle individual ethics cases.

The subcommittee is working towards a clearer demarcation of responsibilities with respect to:

  • The ACL Rolling Review (ARR): As the main source and pool of reviewers for *ACL venues, the ARR represents the key venue for fielding and distributing canonical ethical reviewing guidelines. The AEC has no direct jurisdiction over the ARR, but will seek to work together to develop a clear set of guidelines to recommend that ARR adopts. The AEC also does not operate an ethics reviewing pool, although it may serve as a repository of knowledge about who and how to recruit for ethics-based reviews. It is the AEC's hope that the ARR ethical reviewing guidelines epitomises the key decisions agreed by the AEC. We view this partnership as a key element for the AEC. The AEC membership also reflects this, as both Pascale Fung and Dirk Hovy serve in the ARR initiative (Pascale as co-chair and Dirk as Ethics review chair)
  • The ACL Anthology: Ethics violations may have repercussions on the publications supported by ACL. The AEC plans to look into a workflow that builds upon current workflows for the Anthology and its interface to publication venues (e.g., conference chairs).
  • The ACL Equity Director: As ethics spans many disparate viewpoints, diversity and inclusion are a key part. We are building better outreach for D&I for these purposes.
  • The ACL Professional Conduct Committee (PCC): Individual member-centric problems that directly affect another individual are understood to be in the scope of the PCC. We are not yet clear whether Dirk Hovy is also a member of both the AEC and the PCC.

Luciana Benotti, acts for AEC's relationships with Diversity and Inclusion; Malvina Nissim and Dirk Hovy act for the AEC's relationship with the PCC.

Committee Procedure

This header gathers items related to the decision workflow and administrative policies for the AEC.

Meetings: As of this time, the committee has agreed to hold conference calls monthly and chairs meetings biweekly. Most of the work with respect to initiatives are done offline, in their particular subcommittees. However, some discussions need to happen in the committee as a whole and time will be planned for this.

Software Stack: The AEC needs a secure document store for its official minutes and documents. We plan to use public webpages from Github to organize public facing information and to piggyback off of ACL's paid accounts (Office365) for document stores. Our ethics reading list is an example of this.

Communication Policy: The AEC plans to create a communication policy that will guide committee members in discussions about ethics and matters brought to attention to the committee. This is important as the AEC may be scrutinized as a public-facing committee of the ACL.

Retrieved from "https://www.aclweb.org/adminwiki/index.php?title=2022Q1_Reports:_Ethics_Committee_Co-chairs-Draft&oldid=75071"