Answering Yes-No Questions by Penalty Scoring in History Subjects of University Entrance Examinations

Yoshinobu Kano


Abstract
Answering yes–no questions is more difficult than simply retrieving ranked search results. To answer yes–no questions, especially when the correct answer is no, one must find an objectionable keyword that makes the question’s answer no. Existing systems, such as factoid-based ones, cannot answer yes–no questions very well because of insufficient handling of such objectionable keywords. We suggest an algorithm that answers yes–no questions by assigning an importance to objectionable keywords. Concretely speaking, we suggest a penalized scoring method that finds and makes lower score for parts of documents that include such objectionable keywords. We check a keyword distribution for each part of a document such as a paragraph, calculating the keyword density as a basic score. Then we use an objectionable keyword penalty when a keyword does not appear in a target part but appears in other parts of the document. Our algorithm is robust for open domain problems because it requires no training. We achieved 4.45 point better results in F1 scores than the best score of the NTCIR-10 RITE2 shared task, also obtained the best score in 2014 mock university examination challenge of the Todai Robot project.
Anthology ID:
W16-4413
Volume:
Proceedings of the Open Knowledge Base and Question Answering Workshop (OKBQA 2016)
Month:
December
Year:
2016
Address:
Osaka, Japan
Editors:
Key-Sun Choi, Christina Unger, Piek Vossen, Jin-Dong Kim, Noriko Kando, Axel-Cyrille Ngonga Ngomo
Venue:
WS
SIG:
Publisher:
The COLING 2016 Organizing Committee
Note:
Pages:
91–96
Language:
URL:
https://aclanthology.org/W16-4413
DOI:
Bibkey:
Cite (ACL):
Yoshinobu Kano. 2016. Answering Yes-No Questions by Penalty Scoring in History Subjects of University Entrance Examinations. In Proceedings of the Open Knowledge Base and Question Answering Workshop (OKBQA 2016), pages 91–96, Osaka, Japan. The COLING 2016 Organizing Committee.
Cite (Informal):
Answering Yes-No Questions by Penalty Scoring in History Subjects of University Entrance Examinations (Kano, 2016)
Copy Citation:
PDF:
https://aclanthology.org/W16-4413.pdf