Ithaca College

Turnbull (Computer Science) recently published the following paper: 

SLIONS: A Karaoke Application to Enhance Foreign Language Learning
Dania Murad, Riwu Wang, Douglas Turnbull, Ye Wang
ACM Multimedia (ACM MM ’18)
Seoul, Korea, October 2018

This paper explores the use of a novel mobile karaoke app as a tool to help students learn foreign languages. Prof. Turnbull worked with collaborators on this project while on sabbatical at the National University of Singapore. ACM Multimedia is a highly-competitive conference with only 8% of submitted manuscripts being accepted for oral presentation.  

The SLIONS app is available in both the Apple iTunes Store and the Google Play Store. 

More information can be found at:

https://dougturnbull.org/index.php/2018/05/11/the-winning-slions-karaoke-app/

Paper Abstract:

Singing songs can be an engaging and effective activity when learning a foreign language. In this paper, we describe a multi-language karaoke application called SLIONS: Singing and Listening to Improve Our Natural Speaking. When developing this application, we followed a user-centered design process which was informed by conducting interviews with domain experts, extensive usability testing, and reviewing existing gamified karaoke and language learning applications. The key feature of SLIONS is that we used automatic speech recognition (ASR) to provide students with personalized, granular feedback based on their singing pronunciation. We also provided multi-modal instruction: audio of music and singing tracks, video of a professional singer and translated text of lyrics to help students learn and master each song in the foreign language. To test the efficacy of SLIONS, we conducted a one-week pilot study with English and Chinese language learning students (N=15). The initial quantitative results show that our application can improve pronunciation and may improve vocabulary. In addition, the qualitative feedback from the students suggests that SLIONS is both fun to use and motivates students to practice speaking and singing in a foreign language.