Question Answering (State of the art)
Jump to navigation
Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
Answer Sentence Selection
The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.
- QA Answer Sentence Selection Dataset: labeled sentences using TREC QA track data, provided by Mengqiu Wang and first used in Wang et al. (2007).
Algorithm | Reference | MAP | MRR |
---|---|---|---|
Punyakanok (2004) | Wang et al. (2007) | 0.419 | 0.494 |
Cui (2005) | Wang et al. (2007) | 0.427 | 0.526 |
Wang (2007) | Wang et al. (2007) | 0.603 | 0.685 |
H&S (2010) | Heilman and Smith (2010) | 0.609 | 0.692 |
W&M (2010) | Wang and Manning (2010) | 0.595 | 0.695 |
Yao (2013) | Yao et al. (2013) | 0.631 | 0.748 |
S&M (2013) | Severyn and Moschitti (2013) | 0.678 | 0.736 |
Shnarch (2013) - Backward | Shnarch (2013) | 0.686 | 0.754 |
Yih (2013) - LCLR | Yih et al. (2013) | 0.709 | 0.770 |
References
- Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. Mapping dependencies trees: An application to question answering. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
- Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. Question answering passage retrieval using dependency relations. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
- Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA. In EMNLP-CoNLL 2007.
- Heilman, Michael and Smith, Noah A. 2010. Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions. In NAACL-HLT 2010.
- Wang, Mengqiu and Manning, Christopher. 2010. Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering. In COLING 2010.
- E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
- Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. Answer Extraction as Sequence Tagging with Tree Edit Distance. In NAACL-HLT 2013.
- Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. Question Answering Using Enhanced Lexical Semantic Models. In ACL 2013.
- Severyn, Aliaksei and Moschitti, Alessandro. 2013. Automatic Feature Engineering for Answer Selection and Extraction. In EMNLP 2013.