On-device Structured and Context Partitioned Projection Networks

Sujith Ravi, Zornitsa Kozareva


Abstract
A challenging problem in on-device text classification is to build highly accurate neural models that can fit in small memory footprint and have low latency. To address this challenge, we propose an on-device neural network SGNN++ which dynamically learns compact projection vectors from raw text using structured and context-dependent partition projections. We show that this results in accelerated inference and performance improvements. We conduct extensive evaluation on multiple conversational tasks and languages such as English, Japanese, Spanish and French. Our SGNN++ model significantly outperforms all baselines, improves upon existing on-device neural models and even surpasses RNN, CNN and BiLSTM models on dialog act and intent prediction. Through a series of ablation studies we show the impact of the partitioned projections and structured information leading to 10% improvement. We study the impact of the model size on accuracy and introduce quatization-aware training for SGNN++ to further reduce the model size while preserving the same quality. Finally, we show fast inference on mobile phones.
Anthology ID:
P19-1368
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Editors:
Anna Korhonen, David Traum, Lluís Màrquez
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3784–3793
Language:
URL:
https://aclanthology.org/P19-1368
DOI:
10.18653/v1/P19-1368
Bibkey:
Cite (ACL):
Sujith Ravi and Zornitsa Kozareva. 2019. On-device Structured and Context Partitioned Projection Networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3784–3793, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
On-device Structured and Context Partitioned Projection Networks (Ravi & Kozareva, ACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/P19-1368.pdf