Please use this identifier to cite or link to this item: https://doi.org/10.1007/s10579-012-9176-1
Title: Perspectives on crowdsourcing annotations for natural language processing
Authors: Wang, A.
Hoang, C.D.V.
Kan, M.-Y. 
Keywords: Annotation
Crowdsourcing
Games with a purpose
Human computation
Mechanical Turk
NLP
Wikipedia
Issue Date: 2013
Source: Wang, A., Hoang, C.D.V., Kan, M.-Y. (2013). Perspectives on crowdsourcing annotations for natural language processing. Language Resources and Evaluation 47 (1) : 9-31. ScholarBank@NUS Repository. https://doi.org/10.1007/s10579-012-9176-1
Abstract: Crowdsourcing has emerged as a new method for obtaining annotations for training models for machine learning. While many variants of this process exist, they largely differ in their methods of motivating subjects to contribute and the scale of their applications. To date, there has yet to be a study that helps the practitioner to decide what form an annotation application should take to best reach its objectives within the constraints of a project. To fill this gap, we provide a faceted analysis of crowdsourcing from a practitioner's perspective, and show how our facets apply to existing published crowdsourced annotation applications. We then summarize how the major crowdsourcing genres fill different parts of this multi-dimensional space, which leads to our recommendations on the potential opportunities crowdsourcing offers to future annotation efforts. © 2012 Springer Science+Business Media B.V.
Source Title: Language Resources and Evaluation
URI: http://scholarbank.nus.edu.sg/handle/10635/39496
ISSN: 1574020X
DOI: 10.1007/s10579-012-9176-1
Appears in Collections:Staff Publications

Show full item record
Files in This Item:
There are no files associated with this item.

SCOPUSTM   
Citations

28
checked on Dec 13, 2017

WEB OF SCIENCETM
Citations

18
checked on Dec 13, 2017

Page view(s)

65
checked on Dec 9, 2017

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.