SemEval 2014 Task 4 - Aspect Based Sentiment Analysis

Event Notification Type: 
Call for Participation
Abbreviated Title: 
SemEval 2014 Task 4 - Aspect Based Sentiment Analysis
Saturday, 23 August 2014 to Sunday, 24 August 2014
Contact Email: 
Suresh Manandhar
Submission Deadline: 
Wednesday, 30 April 2014


SemEval 2014 Task 4 - Aspect Based Sentiment Analysis

The aim of this task is to allow a finer-grained aspect based sentiment analysis (ABSA).  

The goal is to identify the aspects (e.g. battery, screen; food, service) of given target entities (cf. laptop, restaurant) and the sentiment expressed towards each aspect.

Sentiment analysis is increasingly viewed as a vital task both from an academic and a commercial standpoint. The majority of existing evaluations in Sentiment Analysis (SA) are aimed at evaluating the overall polarity of a sentence, paragraph, or text span. In contrast, this task will provide evaluation for detecting aspects and the sentiment expressed towards each aspect.

In particular, the task will consist of the following subtasks:

Subtask 1: Aspect term extraction
Given a set of sentences with pre-identified entities (e.g., restaurants), identify the aspect terms present in the sentences.

An aspect term names a particular aspect of the target entity (e.g., "I liked the service and the staff, but not the food”, “The food was nothing much, but I loved the staff”).

Subtask 2: Aspect term polarity
Given one or more aspect terms within a sentence, determine whether the polarity of each aspect term is positive, negative, neutral or conflict (i.e., both positive and negative).

For example:
“I loved their fajitas” → {fajitas: positive}
“I hated their fajitas, but their salads were great” → {fajitas: negative, salads: positive}
“The fajitas are their first plate” → {fajitas: neutral}
“The fajitas were great to taste, but not to see” → {fajitas: conflict}

Subtask 3: Aspect category detection
Given a predefined set of aspect categories (e.g., price, food), identify the aspect categories discussed in a given sentence. Aspect categories are typically coarser than the aspect terms of Subtask 1, and they do not necessarily occur as terms in the given sentence. For example, given the set of aspect categories {food, service, price, ambiance, anecdotes/miscellaneous}:
“The restaurant was too expensive”  → {price}
“The restaurant was expensive, but the menu was great” → {price, food}

Subtask 4: Aspect category polarity
Given a set of pre-identified aspect categories (e.g., {food, price}), determine the polarity (positive, negative, neutral or conflict) of each aspect category. For example:
“The restaurant was too expensive” → {price: negative}
“The restaurant was expensive, but the menu was great” → {price:negative, food: positive}

Two domain-specific datasets for laptops and restaurants, consisting of over 6,500 sentences with fine-grained aspect-level human annotations will be provided for training.

Participants can participate in either all or a subset of subtasks.

Trial data ready October 31, 2013
Training data ready December 15, 2013
Evaluation period March 15-30, 2014 Paper submission due April 30, 2014 [TBC]
SemEval workshop August 23-24, 2014, co-located with COLING and *SEM in Dublin, Ireland.

The Semeval-2014 Task 4 website includes further details on the training data, evaluation, and examples of expected system outputs:

Join our mailing list:

Ion Androutsopoulos (Athens University of Economics and Business, Greece)
Dimitris Galanis (“Athena” Research Center, Greece)
Suresh Manandhar (University of York, UK) [Primary Contact]
Harris Papageorgiou ("Athena" Research Center, Greece)
John Pavlopoulos (Athens University of Economics and Business, Greece)
Maria Pontiki (“Athena” Research Center, Greece)