UNDERSTANDING AND FACILITATING TASK DESIGN IN MICROTASK CROWDSOURCING

Abstract

Microtask crowdsourcing, as a major format of crowdsourcing, has arisen as a popular research topic with the emergence of platforms such as Amazon Mechanical Turk and CrowdFlower. However, microtask crowdsourcing is not widely used in daily jobs. From the literature review, we found that this might result from the difficulty for most requesters to design and create a crowdsourcing solution, especially when there involves content manipulation in the microtasks and workflows among the microtasks with current microtask creation toolkits. To bridge the gaps, this thesis develops a functional and easy-to-use toolkit to design microtasks with different workflows. The specific objectives of the research are as follows: (1) study the specific characteristics of content manipulation microtasks and the specific mechanism of designing them with workflows. (2) propose a "programming by example" approach for designing content manipulation microtasks with workflows. (3) develop a working system that applies the alternative approach and supports the entire designing process. (4) investigate different usage scenarios by utilizing the system. Overall, this thesis contributes to a greater understanding of the entire process of designing microtasks with workflows and unlocks new avenues for exploring the microtask design methods.