Augmenting the Reading Experience with Synchronous Reading and Listening


Recent open source libraries apply machine learning for forced alignment of speech and text and make it possible to produce text that is synchronized to audio in various languages. This makes it possible to produce a new type of reading experience: the videobook! This contribution talks about a pilot project that aims to produce videobooks based on hundreds of audiobooks and texts that are already in the public domain (such as librivox and the gutenberg project). Possible applications in the educational domain are discussed: such as language learning, accessibility and e-learning applications