2015Q3 Reports: Demo Chair

Hsin-Hsi Chen and Katja Markert co-chaired the system demonstrations session at the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. The system demonstrations program offers the presentation of early research prototypes as well as interesting mature systems. Each demonstration paper consists of 6 pages (including references).

We received 62 submissions, of which 25 were selected for inclusion in the program (acceptance rate of 40.32%) after review by three members of the program committee. The program committee for system demonstrations was composed of 36 members. Each reviewer was allocated at most 6 papers.

The evaluation criteria we used referred mostly to regular papers, but we emphasized in their wording system- and application-relevant components.

Anonymisation of supplied demonstration links is still a problem (although most authors managed to work around it) and we would recommend that in future the submissions actually allow for a demo version of the system to be directly submitted with the paper as an attachment. One other problem we encountered is whether reviewers actually look at the provided demos (instead of just at the paper). Many seemed to not do that --- future evaluation criteria should have a special item dealing with this with maybe preferred acceptance of demos that are actually submitted with the paper. However, this leads to increased reviewing load.