Institute of Information Science, Academia Sinica

Amazon Mechanical Turk (MTurk) was launched to the public in 2005. Since then, researchers have developed a tremendous amount of research around the platform. In this talk, via the lens of computer system development, we will walk through three stages of the development of "crowd reach", which can also be referred to as "human computation", "crowdsourcing", or "human-in-the-loop" architectures: (1) labeling datasets and providing human feedback for computer systems, (2) developing workflows for human workers, and (3) creating real-time interactive systems that are, fully or partially, run by the crowd.