Multi-Task Networks with Universe, Group, and Task Feature Learning

Shiva Pentyala, Mengwen Liu, Markus Dreyer


Abstract
We present methods for multi-task learning that take advantage of natural groupings of related tasks. Task groups may be defined along known properties of the tasks, such as task domain or language. Such task groups represent supervised information at the inter-task level and can be encoded into the model. We investigate two variants of neural network architectures that accomplish this, learning different feature spaces at the levels of individual tasks, task groups, as well as the universe of all tasks: (1) parallel architectures encode each input simultaneously into feature spaces at different levels; (2) serial architectures encode each input successively into feature spaces at different levels in the task hierarchy. We demonstrate the methods on natural language understanding (NLU) tasks, where a grouping of tasks into different task domains leads to improved performance on ATIS, Snips, and a large in-house dataset.
Anthology ID:
P19-1079
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
820–830
Language:
URL:
https://aclanthology.org/P19-1079
DOI:
10.18653/v1/P19-1079
Bibkey:
Cite (ACL):
Shiva Pentyala, Mengwen Liu, and Markus Dreyer. 2019. Multi-Task Networks with Universe, Group, and Task Feature Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 820–830, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Multi-Task Networks with Universe, Group, and Task Feature Learning (Pentyala et al., ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1079.pdf
Video:
 https://aclanthology.org/P19-1079.mp4