<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="enguehard-etal-2017-exploring">
    <titleInfo>
        <title>Exploring the Syntactic Abilities of RNNs with Multi-task Learning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Émile</namePart>
        <namePart type="family">Enguehard</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yoav</namePart>
        <namePart type="family">Goldberg</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tal</namePart>
        <namePart type="family">Linzen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Roger</namePart>
            <namePart type="family">Levy</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lucia</namePart>
            <namePart type="family">Specia</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vancouver, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent work has explored the syntactic abilities of RNNs using the subject-verb agreement task, which diagnoses sensitivity to sentence structure. RNNs performed this task well in common cases, but faltered in complex sentences (Linzen et al., 2016). We test whether these errors are due to inherent limitations of the architecture or to the relatively indirect supervision provided by most agreement dependencies in a corpus. We trained a single RNN to perform both the agreement task and an additional task, either CCG supertagging or language modeling. Multi-task training led to significantly lower error rates, in particular on complex sentences, suggesting that RNNs have the ability to evolve more sophisticated syntactic representations than shown before. We also show that easily available agreement training data can improve performance on other syntactic tasks, in particular when only a limited amount of training data is available for those tasks. The multi-task paradigm can also be leveraged to inject grammatical knowledge into language models.</abstract>
    <identifier type="citekey">enguehard-etal-2017-exploring</identifier>
    <identifier type="doi">10.18653/v1/K17-1003</identifier>
    <location>
        <url>https://aclanthology.org/K17-1003</url>
    </location>
    <part>
        <date>2017-08</date>
        <extent unit="page">
            <start>3</start>
            <end>14</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Exploring the Syntactic Abilities of RNNs with Multi-task Learning
%A Enguehard, Émile
%A Goldberg, Yoav
%A Linzen, Tal
%Y Levy, Roger
%Y Specia, Lucia
%S Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017)
%D 2017
%8 August
%I Association for Computational Linguistics
%C Vancouver, Canada
%F enguehard-etal-2017-exploring
%X Recent work has explored the syntactic abilities of RNNs using the subject-verb agreement task, which diagnoses sensitivity to sentence structure. RNNs performed this task well in common cases, but faltered in complex sentences (Linzen et al., 2016). We test whether these errors are due to inherent limitations of the architecture or to the relatively indirect supervision provided by most agreement dependencies in a corpus. We trained a single RNN to perform both the agreement task and an additional task, either CCG supertagging or language modeling. Multi-task training led to significantly lower error rates, in particular on complex sentences, suggesting that RNNs have the ability to evolve more sophisticated syntactic representations than shown before. We also show that easily available agreement training data can improve performance on other syntactic tasks, in particular when only a limited amount of training data is available for those tasks. The multi-task paradigm can also be leveraged to inject grammatical knowledge into language models.
%R 10.18653/v1/K17-1003
%U https://aclanthology.org/K17-1003
%U https://doi.org/10.18653/v1/K17-1003
%P 3-14

Markdown (Informal)

[Exploring the Syntactic Abilities of RNNs with Multi-task Learning](https://aclanthology.org/K17-1003) (Enguehard et al., CoNLL 2017)

• Exploring the Syntactic Abilities of RNNs with Multi-task Learning (Enguehard et al., CoNLL 2017)

ACL

• Émile Enguehard, Yoav Goldberg, and Tal Linzen. 2017. Exploring the Syntactic Abilities of RNNs with Multi-task Learning. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 3–14, Vancouver, Canada. Association for Computational Linguistics.