*** Apologies for cross-posting ***
Update regarding the First Task for Automatic Cyberbullying Detection for the Polish Language
http://poleval.pl/tasks/task6
Deadlines have been extended:
-- Now -- Training data available.
March 29, 2019 Test data available.
April 05, 2019 Results collected from participants.
April 12, 2019 PolEval 2019 results announced.
May 12, 2019 System descriptions delivered by participants as paper submissions.
May 31, 2019 PolEval 2019 submissions presented at the AI & NLP Day 2019.
PolEval 2019 Task 6: Automatic cyberbullying detection
Task homepage: http://poleval.pl/tasks/task6
Important dates: http://poleval.pl/dates/
Best regards and good luck,
Task organizers:
Michał Ptaszyński, Kitami Institute of Technology, Japan
Agata Pieciukiewicz, Polish-Japanese Academy of Information Technology, Poland
Paweł Dybała, Jagiellonian University in Kraków, Poland
--== Task definition ==--
Although the problem of humiliating and slandering people through the Internet has existed almost as long as communication via the Internet between people, the appearance of new devices, such as smartphones and tablet computers, which allow using this medium not only at home, work or school but also in motion, has further exacerbated the problem. Especially recent decade, during which Social Networking Services (SNS), such as Facebook and Twitter, rapidly grew in popularity, has brought to light the problem of unethical behaviors in Internet environments, which has been greatly impairing public mental health in adults and, for the most, in younger users and children. It is the problem of cyberbullying (CB), defined as exploitation of open online means of communication, such as Internet forum boards, or SNS to convey harmful and disturbing information about private individuals, often children and students.
To deal with the problem, researchers around the world have started studying the problem of cyberbullying with a goal to automatically detect Internet entries containing harmful information, and report them to SNS service providers for further analysis and deletion. After ten years of research [1], a sufficient knowledge base on this problem has been collected for languages of well-developed countries, such as the US, or Japan. Unfortunately, still close to nothing in this matter has been done for the Polish language. With this task, we aim at filling this gap.
In this pilot task, the contestants will determine whether an Internet entry is classifiable as part of cyberbullying narration or not. The entries will contain tweets collected from openly available Twitter discussions. Since much of the problem of automatic cyberbullying detection often relies on feature selection and feature engineering [2], the tweets will be provided as such, with minimal preprocessing. The preprocessing, if used, will be applied mostly for cases when information about a private person is revealed to the public.
The goal of the contestants will be to classify the tweets into cyberbullying/harmful and non-cyberbullying/non-harmful with the highest possible Precision, Recall, balanced F-score and Accuracy. In an additional sub-task, the contestants will differentiate between various types of cyberbullying, i.e., revealing of private information, personal threats, blackmails, ridiculing, gossip/insinuations, or accumulation of vulgarities and profane language.
--== Task description ==--
Task 6-1: Harmful vs non-harmful
In this task, the participants are to distinguish between normal/non-harmful tweets (class: 0) and tweets that contain any kind of harmful information (class: 1). This includes cyberbullying, hate speech and related phenomena.
Task 6-2: Type of harmfulness
In this task, the participants shall distinguish between three classes of tweets: 0 (non-harmful), 1 (cyberbullying), 2 (hate-speech). There are various definitions of both cyberbullying and hate-speech, some of them even putting those two phenomena in the same group. The specific conditions on which we based our annotations for both cyberbullying and hate-speech, which have been worked out during ten years of research [1] will be summarized in an introductory paper for the task, however, the main and definitive condition to distinguish the two is whether the harmful action is addressed towards a private person(s) (cyberbullying), or a public person/entity/large group (hate-speech).
--== Submission ==--
The participating team should prepare two folders, one for each task, with naming such as “task01” and “task02”. If the team aims only at participating in one of the tasks, only one folder is required. In each folder, the team should copy the file containing results (all labels, one per line, provided by their method), and an output file with results calculated using the evaluation Perl scripts. The folder(s) should be compressed in one file and submitted to the PolEval Submission System.
--== References ==--
[1] Michal E. Ptaszynski, Fumito Masui. (2018). “Automatic Cyberbullying Detection: Emerging Research and Opportunities”, IGI Global Publishing (November 2018), ISBN: 9781522552499.
[2] Michal Ptaszynski, Juuso Kalevi Kristian Eronen and Fumito Masui. (2017). "Learning Deep on Cyberbullying is Always Better Than Brute Force", IJCAI 2017 3rd Workshop on Linguistic and Cognitive Approaches to Dialogue Agents (LaCATODA 2017), Melbourne, Australia, August 19-25, 2017