SPEECH-TO-TEXT CONVERSION WEARABLE SMART GLASSES FOR THE HEARING-IMPAIRED USING ESP 32

Abstract

Hearing-impaired individuals face communication challenges, such as daily interactions with other people due to partial or complete loss of hearing, which shows impact on their quality of life in society. The assistive technologies, such as hearing aids, sign language and speech-to-text applications have significantly improved accessibility and enhanced the way of communication for those with hearing disabilities. Ongoing research aims to develop more effective solutions for better integration and inclusivity in society. This paper presents a device ESP-32, which will be the core microcontroller board in order to integrate Deepgram’s API for the purposes of real-time speech recognition and conversion tasks as a hearing assistant. The device hosts an INMP441 digital microphone, which captures audio from the surroundings and converts it to a digital signal. The integrated 24-bit ADC converts signals analog to digital ones and then communicates to the ESP-32 microcontroller board using the I2S protocol. The ESP-32 then takes up the responsibility of processing digital audio data and transmitting it via Wi-Fi to the Deepgram API. On the other hand, the API transcribes the speech into text and sends it back to the ESP-32 microcontroller. This text is eventually displayed as subtitles on the glasses with the help of a TOLED connected to the ESP-32 through the I2C protocol. Consequently, this proposed innovation will enable users to communicate effectively in social, educational, or business scenarios, creating a shared space between people who can hear and the hearing impaired through the use of technology.