2020Q3 Reports: Demonstration Chairs

Highlights of the Demo Track:

- We had the largest ever in number of demo papers and we only accepted 44 papers (one was withdrawn).

- We managed to run demo sessions similar to the main conference paper sessions with some exceptions.

- In the virtual conference setting, each demonstration paper included a short video description of the application and a link to the software application (e.g., github link). Due to several issues and limitations, in the virtual conference setting, we didn’t require that the demo papers have online application, however, most demo papers included a link to a website which hosted the application to enable online demonstrations. Other demo papers chose to include only a link to their application which can be downloaded and run offline.

Challenges

- Similar to regular paper submissions, for demo papers we performed a very thorough manual review of all camera-ready PDFs and asked several authors to fix margin and other formatting problems we discovered. Since we only had 43 accepted papers, this wasn’t as hard task as the main conference.

- We didn’t use the same review format as the ACL main paper review schema, so most reviewers notified us about that. It’s best to use main conference review format as most reviewers review papers for main and demo tracks.

- Even with 43 accepted papers, it was a lot of work for two demo chairs to go through the papers, checking the github links, collecting data from all authors about their time-zone preferences, setting up a complex scheduling to accommodate all papers in their corresponding time zones and have them present twice in their respective time-zones, etc.

Recommendations

- For the demo papers, most of the main paper issues and problems apply; so please check out the issues and recommendations sections from the Conf. Chairs and Publication Chairs.

- Consider getting help from student volunteers or others for some of the virtual conference related tasks.

- On Softconf, copy the paper review schema from the main paper submission to the demo submissions so the reviewers are not using standard but customized review forms designed for the main paper submissions. You may need to get help from the program chairs or soft conf for this.

- Communicating with the main conference chairs and publication chairs and aligning your track with the main conference ahead of time can save you time. Slack group channels is best way for this.

- Get help from volunteers to watch the demo paper presentations as some demos presentations might be too short for the audience to grasp the demo application (especially important if they are not proving a live demo application, and only a GitHub link to their demo). - If planning on selecting best demo paper and honorable mentions, form a committee including some ethics experts that should go over top-k best accepted papers. - Similar to the main conference, require an ethics statement of all papers, add ethics to reviewing form (e.g., to include potential ethical concerns that the reviewed demo paper might raise), add ethics, and also diversity & inclusion to all paper award decision criteria.