Game-Powered Machine LearningGame-Powered Machine Learning
Luke Barrington (UC San Diego), Doug Turnbull (Computer Science, Ithaca College), Gert Lanckriet (UC Sand Diego)
Published in the Proceedings of the National Academy of Science (PNAS) - March 2012
Abstract:
Searching for relevant content in a massive amount of multimedia information is facilitated by accurately annotating each image, video, or song with a large number of relevant semantic keywords, or tags. We introduce game-powered machine learning, an integrated approach to annotating multimedia content that combines the effectiveness of human computation, through online games, with the scalability of machine learning. We investigate this framework for labeling music. First, a socially-oriented music annotation game called Herd It collects reliable music annotations based on the “wisdom of the crowds.” Second, these annotated examples are used to train a supervised machine learning system. Third, the machine learning system actively directs the annotation games to collect new data that will most benefit future model iterations. Once trained, the system can automatically annotate a corpus of music much larger than what could be labeled using human computation alone. Automatically annotated songs can be retrieved based on their semantic relevance to text-based queries (e.g., “funky jazz with saxophone,” “spooky electronica,” etc.). Based on the results presented in this paper, we find that actively coupling annotation games with machine learning provides a reliable and scalable approach to making searchable massive amounts of multimedia data.
Link to the journal paper:
http://www.pnas.org/content/early/2012/03/27/1014748109.abstract
Link to Herd It - a social music annotation game on Facebook:
http://apps.facebook.com/herd-it/
About The Proceedings of the National Academy of Science (PNAS):
PNAS is one of the world's most-cited multidisciplinary scientific serials. Since its establishment in 1914, it continues to publish cutting-edge research reports, commentaries, reviews, perspectives, colloquium papers, and actions of the Academy. Coverage in PNAS spans the biological, physical, and social sciences. PNAS is published weekly in print, and daily online in PNAS Early Edition.
https://www.ithaca.edu/intercom/article.php/20120408183920500