5th Werewolf AI Competition (INLG 2023 Gerenation Challenge)
Call For Participation
# Summary:
Werewolf is a social, hidden identity game that requires debate between players and coalition building. The goal of our AIWerewolf contest is to build an AI agent that is able to play this game against other AI.
We provide three divisions, the Natural Langauge Division, the Protocol Division, and the Infrastructure Division. Their regsitration deadlines are as follows:
Natural Language Division: June 30th, 2023 (Anywhere on Earth)
Protocol Division: July 21st, 2023 (Anywhere on Earth)
Infrastructure Division: August 18th, 2023 (Anywhere on Earth)
Presetation of Results: September 11th-15th (in INLG 2023)
Our shared task is one of the Generation Challenges (GenChal) of INLG 2023 (16th International Natural Language Generation Conference), which will be held in Prague from September 11th to 15th, but it is not mandatry for our shared task participants to attend the INLG 2023 conference.
Please refer to the details of each division in this CFP and the websites below.
Natural Language Division: https://github.com/aiwolfdial/RandomTalkAgent
Protocol Division: https://github.com/aiwolf/CompetitionProtocolDivision
Infrastructure Division: https://github.com/aiwolf/CompetitionInfrastructureDivision
The AIWerewolf Project: http://aiwolf.org/5th-international-aiwolf-contest
INLG 2023: https://inlg2023.github.io/
SIGDial 2023, which is jointly held with INLG 2023 this year: https://2023.sigdial.org/
# Why AI Werewolf?
Recent achievements of generation models, e.g. ChatGPT, are gathering greater attentions. However, such a huge language model would not be sufficiently able to handle coherent responses, longer contexts, common grounds, and logics.
AIWolfDial 2023 is an international open contest for automatic players of the conversation game "Mafia". It requires players not just to communicate but to infer, persuade, deceive other players via coherent logical conversations, while having the role-playing non-task-oriented chats as well. We believe that this contest reveals current issues in the recent huge language models, showing directions of next breakthrough in the NLP area.
From the viewpoint of Game AI area, players must hide information, in contrast to perfect information games such as chess or Reversi. Each player acquires secret information from other players' conversations and behavior and acts by hiding information to accomplish their objectives. Players are required persuasion for earning confidence, and speculation for detecting fabrications.
# Natural Language Division
In this division, participants must build an artificial intelligence agent that can play the werewolf game as humans do, using natural language. Participants in this division will be evaluated by a panel of judges, who will grade the subjective quality of the dialog generated by the agent. Agents must communicate in Japanese or English.
## Registration
A team should send a mail to aiwolf [at] kanolab.net (replace at by @), describing your team name, a contact e-mail address, names and affiliations of its members (please mark a contact person when a team consists of multiple members). Registration is free.
## Agent Testing
A participant is required to implement an AI werewolf agent that connects to our AIWolf server at the specified time. Before that, we will provide an AIWolf server where participants can try connecting with dummy agents to check their system's behavior. Make sure that your system works correctly on the testing system before the actual competition run.
## Preliminary Run
Participants should run five agents connected to our server, running 50 games. Submit your game logs to the organisers via e-mail. If there are too many participants for the final run, organisers will select final run teams from these logs. In either case, these logs will be used in the final evaluation.
## Final Run
Each participant team is required to connect its agent to our server at a specified time to play games with other participants. If your team cannot remotely attend the final run, you should prepare your system so that the organisers can somehow trigger your remote system to connect our server and run, for example, via a web browser.
## System Evaluation
Participants should submit a system design description document to the organizers. This document and logs of the games might be used for research purposes and included and published in an overview paper without any further permission. Participants are encouraged to submit a paper to the INLG workshop. Reviewers will perform subjective evaluations on the game logs.
## Implementation of Natural Language Division Agents
Agents must communicate in English or Japanese. We might translate English-Japanese by machine translation internally when the number of participants for either of the languages is small. Participants are required to implement an AI werewolf agent that connects to our AIWolf server via the network. That is, there is no limitation on the client side regarding hardware or software other than the network I/O specifications.
Please refer to our sample agent code with runtime environment (java), distributed from our Github repository:
https://github.com/aiwolfdial/RandomTalkAgent
which also describes how to connect to our game server, etc.
## Resources
The Mafiascum Dataset provides is a collection of over 700 games of Mafia: https://bitbucket.org/bopjesvla/thesis/src/master/
# Protocol Division
In the protocol division, the AIWolf agents which communicate with each other via the AIWolf protocol, compete for win rate.
Agents are evaluated by their average win rate in 15-player games and 5-player games.
This division is divided into a qualifying competition and a final one.
In the qualifying competition, all participants play against each other, and 15 agents with the highest win rate can advance to the final competition as finalists.
In the final competition, finalists play against each other as many times as possible until the difference in win rate becomes signigicant.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionProtocolDivision
# Infrastructure Division
In the 5th International AI Werewolf Competition, we open an infrastructure division to evaluate the contribution to the development of AI Werewolf such as sample agents, libraries for making agents, alternative game servers, cool game viewers, etc.
Creations submited to this division will be evaluated by the participants and organizers of the AI Werewolf competition, and a vote will choose the best contribution of this year.
For more information, please visit the web site of this division.
https://github.com/aiwolf/CompetitionInfrastructureDivision
# Organizers
Yoshinobu Kano, Shizuoka University, Japan
Claus Aranha, Tsukuba University
Takashi Otsuki, Yamagata University, Japan
Fujio Toriumi, The University of Tokyo, Japan
Hirotaka Osawa, University of Tsukuba, Japan
Daisuke Katagami, Tokyo Polytechnic University, Japan
Michimasa Inaba, The University of Electro-Communications, Japan
Kei Harada, The University of Electro-Communications, Japan
On behalf of the AIWolf project organizers,
Yoshinobu Kano
Associate Professor, Shizuoka University
kano [at] inf.shizuoka.ac.jp