<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kampman-etal-2018-investigating">
    <titleInfo>
        <title>Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Onno</namePart>
        <namePart type="family">Kampman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Elham</namePart>
        <namePart type="family">J. Barezi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dario</namePart>
        <namePart type="family">Bertero</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pascale</namePart>
        <namePart type="family">Fung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Iryna</namePart>
            <namePart type="family">Gurevych</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yusuke</namePart>
            <namePart type="family">Miyao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Melbourne, Australia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We propose a tri-modal architecture to predict Big Five personality trait scores from video clips with different channels for audio, text, and video data. For each channel, stacked Convolutional Neural Networks are employed. The channels are fused both on decision-level and by concatenating their respective fully connected layers. It is shown that a multimodal fusion approach outperforms each single modality channel, with an improvement of 9.4% over the best individual modality (video). Full backpropagation is also shown to be better than a linear combination of modalities, meaning complex interactions between modalities can be leveraged to build better models. Furthermore, we can see the prediction relevance of each modality for each trait. The described model can be used to increase the emotional intelligence of virtual agents.</abstract>
    <identifier type="citekey">kampman-etal-2018-investigating</identifier>
    <identifier type="doi">10.18653/v1/P18-2096</identifier>
    <location>
        <url>https://aclanthology.org/P18-2096</url>
    </location>
    <part>
        <date>2018-07</date>
        <extent unit="page">
            <start>606</start>
            <end>611</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction
%A Kampman, Onno
%A J. Barezi, Elham
%A Bertero, Dario
%A Fung, Pascale
%Y Gurevych, Iryna
%Y Miyao, Yusuke
%S Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
%D 2018
%8 July
%I Association for Computational Linguistics
%C Melbourne, Australia
%F kampman-etal-2018-investigating
%X We propose a tri-modal architecture to predict Big Five personality trait scores from video clips with different channels for audio, text, and video data. For each channel, stacked Convolutional Neural Networks are employed. The channels are fused both on decision-level and by concatenating their respective fully connected layers. It is shown that a multimodal fusion approach outperforms each single modality channel, with an improvement of 9.4% over the best individual modality (video). Full backpropagation is also shown to be better than a linear combination of modalities, meaning complex interactions between modalities can be leveraged to build better models. Furthermore, we can see the prediction relevance of each modality for each trait. The described model can be used to increase the emotional intelligence of virtual agents.
%R 10.18653/v1/P18-2096
%U https://aclanthology.org/P18-2096
%U https://doi.org/10.18653/v1/P18-2096
%P 606-611

Markdown (Informal)

[Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction](https://aclanthology.org/P18-2096) (Kampman et al., ACL 2018)

• Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction (Kampman et al., ACL 2018)

ACL

• Onno Kampman, Elham J. Barezi, Dario Bertero, and Pascale Fung. 2018. Investigating Audio, Video, and Text Fusion Methods for End-to-End Automatic Personality Prediction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 606–611, Melbourne, Australia. Association for Computational Linguistics.