<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="el-kishky-etal-2018-entropy">
    <titleInfo>
        <title>Entropy-Based Subword Mining with an Application to Word Embeddings</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ahmed</namePart>
        <namePart type="family">El-Kishky</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Frank</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Aston</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stephen</namePart>
        <namePart type="family">Macke</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jiawei</namePart>
        <namePart type="family">Han</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop on Subword/Character LEvel Models</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Manaal</namePart>
            <namePart type="family">Faruqui</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hinrich</namePart>
            <namePart type="family">Schütze</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabel</namePart>
            <namePart type="family">Trancoso</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulia</namePart>
            <namePart type="family">Tsvetkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yadollah</namePart>
            <namePart type="family">Yaghoobzadeh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">New Orleans</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent literature has shown a wide variety of benefits to mapping traditional one-hot representations of words and phrases to lower-dimensional real-valued vectors known as word embeddings. Traditionally, most word embedding algorithms treat each word as the finest meaningful semantic granularity and perform embedding by learning distinct embedding vectors for each word. Contrary to this line of thought, technical domains such as scientific and medical literature compose words from subword structures such as prefixes, suffixes, and root-words as well as compound words. Treating individual words as the finest-granularity unit discards meaningful shared semantic structure between words sharing substructures. This not only leads to poor embeddings for text corpora that have long-tail distributions, but also heuristic methods for handling out-of-vocabulary words. In this paper we propose SubwordMine, an entropy-based subword mining algorithm that is fast, unsupervised, and fully data-driven. We show that this allows for great cross-domain performance in identifying semantically meaningful subwords. We then investigate utilizing the mined subwords within the FastText embedding model and compare performance of the learned representations in a downstream language modeling task.</abstract>
    <identifier type="citekey">el-kishky-etal-2018-entropy</identifier>
    <identifier type="doi">10.18653/v1/W18-1202</identifier>
    <location>
        <url>https://aclanthology.org/W18-1202</url>
    </location>
    <part>
        <date>2018-06</date>
        <extent unit="page">
            <start>12</start>
            <end>21</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Entropy-Based Subword Mining with an Application to Word Embeddings
%A El-Kishky, Ahmed
%A Xu, Frank
%A Zhang, Aston
%A Macke, Stephen
%A Han, Jiawei
%Y Faruqui, Manaal
%Y Schütze, Hinrich
%Y Trancoso, Isabel
%Y Tsvetkov, Yulia
%Y Yaghoobzadeh, Yadollah
%S Proceedings of the Second Workshop on Subword/Character LEvel Models
%D 2018
%8 June
%I Association for Computational Linguistics
%C New Orleans
%F el-kishky-etal-2018-entropy
%X Recent literature has shown a wide variety of benefits to mapping traditional one-hot representations of words and phrases to lower-dimensional real-valued vectors known as word embeddings. Traditionally, most word embedding algorithms treat each word as the finest meaningful semantic granularity and perform embedding by learning distinct embedding vectors for each word. Contrary to this line of thought, technical domains such as scientific and medical literature compose words from subword structures such as prefixes, suffixes, and root-words as well as compound words. Treating individual words as the finest-granularity unit discards meaningful shared semantic structure between words sharing substructures. This not only leads to poor embeddings for text corpora that have long-tail distributions, but also heuristic methods for handling out-of-vocabulary words. In this paper we propose SubwordMine, an entropy-based subword mining algorithm that is fast, unsupervised, and fully data-driven. We show that this allows for great cross-domain performance in identifying semantically meaningful subwords. We then investigate utilizing the mined subwords within the FastText embedding model and compare performance of the learned representations in a downstream language modeling task.
%R 10.18653/v1/W18-1202
%U https://aclanthology.org/W18-1202
%U https://doi.org/10.18653/v1/W18-1202
%P 12-21

Markdown (Informal)

[Entropy-Based Subword Mining with an Application to Word Embeddings](https://aclanthology.org/W18-1202) (El-Kishky et al., SCLeM 2018)

• Entropy-Based Subword Mining with an Application to Word Embeddings (El-Kishky et al., SCLeM 2018)

ACL

• Ahmed El-Kishky, Frank Xu, Aston Zhang, Stephen Macke, and Jiawei Han. 2018. Entropy-Based Subword Mining with an Application to Word Embeddings. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 12–21, New Orleans. Association for Computational Linguistics.