SpeechRestore is a revolutionary communication device for speech impaired dysarthric speakers that enables them to have more fluent face-to-face conversation. SpeechRestore is automatic speech recognition for dysarthric speakers.
The SpeechRestore speech recognition application on the user’s smartphone / tablet recognizes the speech impaired user’s words and then plays the words clearly articulated through the smartphone’s speaker. SpeechRestore offers the option of either speaking the keyword or touching a button on the screen to play each word or phrase.
No costly equipment is required – SpeechRestore is an app that is easily loaded on the user’s smartphone or tablet.
No connection to the “cloud” is required, so that users do not need mobile data or Wi-Fi network connection for SpeechRestore to work.
A Proof of Concept SpeechRestore application was developed for a user with Primary Lateral Sclerosis (PLS), a type of ALS that affected his ability to speak, diagnosed as dysarthria (unclear articulation of speech). SpeechRestore helped him communicate.
Here is how it works: The SpeechRestore speech recognition application on the Android smartphone / tablet recognizes the user’s words and then plays the word or phrase through the smartphone’s speaker using text-to-speech (TTS). The user either speaks the keyword or selects a button on the screen to play each phrase.
Initially state of the art automatic speech recognition (ASR) applications were tested, including Amazon Echo, Dragon NaturallySpeaking, and Google Voice. They recognized none of our user’s words, a word error rate (WER) of 100%. These applications have acoustic models trained by data sets of many thousands of people’s speech. However, the voices of speech impaired persons are not well represented in these data sets. These ASRs also do not expose APIs to allow the creation of new acoustic models or limiting the dictionary.
An AI based ASR was selected that allowed adapting and creating new acoustic models, limiting the dictionary, and optimizing the design. Also, the decoder engine was required to run locally on smartphones (Android and iOS) with acceptable performance. With the decoder running locally, the user does not have to be connected to mobile data or a Wi-Fi network.
A custom acoustic model for our user was created on Linux using recordings of our user’s words and the recognition engine was optimized for minimum WER. This model and the decoding application were moved to the Android smartphone / tablet platform. The result was a WER of less than 10%.
SpeechRestore uses the recognition of a word to play a word or phrase. For example, recognizing the keyword “water” plays “I need some water” and for “help”, “I need help!”. Buttons with the phrase name and keyword are provided on the smartphone / tablet screen.
The user has the option of speaking the keyword or touching the button to play a phrase. There are four buttons per row, with the phrases arranged by rows, by these categories: help, actions, questions, greetings, positive responses, negative responses, ending a conversation, and information. Using the screen scrolling feature in Android, there is no limit to the number of phrases. SpeechRestore can be used on both tablets and smartphones.
SpeechRestore
SpeechRestore is a revolutionary communication device for speech impaired dysarthric speakers that enables them to have more fluent face-to-face conversation. SpeechRestore is automatic speech recognition for dysarthric speakers.