Using crowdsourcing for labelling emotional speech assets

Alexey Tarasov, Charlie Cullen, Sarah-Jane Delany

Research output: Contribution to conferencePaper

Abstract

The success of supervised learning approaches for the classification of emotion in speech depends highly on the quality of the training data. The manual annotation of emotion speech assets is the primary way of gathering training data for emotional speech recognition. This position paper proposes the use of crowdsourcing for the rating of emotion speech assets. Recent developments in learning from crowdsourcing offer opportunities to determine accurate ratings for assets which have been annotated by large numbers of non-expert individuals. The challenges involved include identifying good annotators, determining consensus ratings and learning the bias of annotators.
Original languageEnglish
DOIs
Publication statusPublished - 5 Oct 2010
Externally publishedYes
EventW3C workshop on Emotion Markup Language - Telecom ParisTech, Paris, France
Duration: 5 Oct 20106 Oct 2010

Workshop

WorkshopW3C workshop on Emotion Markup Language
CountryFrance
CityParis
Period5/10/106/10/10

Fingerprint

Labeling
Supervised learning
Speech recognition

Cite this

Tarasov, A., Cullen, C., & Delany, S-J. (2010). Using crowdsourcing for labelling emotional speech assets. Paper presented at W3C workshop on Emotion Markup Language, Paris, France. https://doi.org/10.21427/D7RS4G
Tarasov, Alexey ; Cullen, Charlie ; Delany, Sarah-Jane. / Using crowdsourcing for labelling emotional speech assets. Paper presented at W3C workshop on Emotion Markup Language, Paris, France.
@conference{df620971dd6b4d338ad77446bb49e8df,
title = "Using crowdsourcing for labelling emotional speech assets",
abstract = "The success of supervised learning approaches for the classification of emotion in speech depends highly on the quality of the training data. The manual annotation of emotion speech assets is the primary way of gathering training data for emotional speech recognition. This position paper proposes the use of crowdsourcing for the rating of emotion speech assets. Recent developments in learning from crowdsourcing offer opportunities to determine accurate ratings for assets which have been annotated by large numbers of non-expert individuals. The challenges involved include identifying good annotators, determining consensus ratings and learning the bias of annotators.",
author = "Alexey Tarasov and Charlie Cullen and Sarah-Jane Delany",
year = "2010",
month = "10",
day = "5",
doi = "10.21427/D7RS4G",
language = "English",
note = "W3C workshop on Emotion Markup Language ; Conference date: 05-10-2010 Through 06-10-2010",

}

Tarasov, A, Cullen, C & Delany, S-J 2010, 'Using crowdsourcing for labelling emotional speech assets' Paper presented at W3C workshop on Emotion Markup Language, Paris, France, 5/10/10 - 6/10/10, . https://doi.org/10.21427/D7RS4G

Using crowdsourcing for labelling emotional speech assets. / Tarasov, Alexey; Cullen, Charlie; Delany, Sarah-Jane.

2010. Paper presented at W3C workshop on Emotion Markup Language, Paris, France.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Using crowdsourcing for labelling emotional speech assets

AU - Tarasov, Alexey

AU - Cullen, Charlie

AU - Delany, Sarah-Jane

PY - 2010/10/5

Y1 - 2010/10/5

N2 - The success of supervised learning approaches for the classification of emotion in speech depends highly on the quality of the training data. The manual annotation of emotion speech assets is the primary way of gathering training data for emotional speech recognition. This position paper proposes the use of crowdsourcing for the rating of emotion speech assets. Recent developments in learning from crowdsourcing offer opportunities to determine accurate ratings for assets which have been annotated by large numbers of non-expert individuals. The challenges involved include identifying good annotators, determining consensus ratings and learning the bias of annotators.

AB - The success of supervised learning approaches for the classification of emotion in speech depends highly on the quality of the training data. The manual annotation of emotion speech assets is the primary way of gathering training data for emotional speech recognition. This position paper proposes the use of crowdsourcing for the rating of emotion speech assets. Recent developments in learning from crowdsourcing offer opportunities to determine accurate ratings for assets which have been annotated by large numbers of non-expert individuals. The challenges involved include identifying good annotators, determining consensus ratings and learning the bias of annotators.

UR - https://www.w3.org/2010/10/emotionml/agenda.html

U2 - 10.21427/D7RS4G

DO - 10.21427/D7RS4G

M3 - Paper

ER -

Tarasov A, Cullen C, Delany S-J. Using crowdsourcing for labelling emotional speech assets. 2010. Paper presented at W3C workshop on Emotion Markup Language, Paris, France. https://doi.org/10.21427/D7RS4G