SYSTEMS/APPLICATIONS CLASS
Blind Spot App
An app created to help visually impaired people navigate the world around them, implementing text to speech and object detection.
OVERVIEW
Blind Spot was created to help visually impaired people navigate the world around them. BlindSpot implements text to speech using TensorFlow object detection.
ROLE/SCOPE
This group project was completed for a Mobile System and Applications class in 2021.
Blind Spot is designed to help users with visual impairments navigate their day-to-day lives. It does this by detecting objects in the users environment, and giving them verbal alerts.
OBJECT DETECTION SOFTWARE
Our implementation includes a GitHub source that teaches how to use live object detection models provided by TensorFlow pre-trained models. Using COCO (Common Objects in Context) datasets with different labels and a TFLite model, we implemented the model with the CameraX feature. The feature takes multiple images of an object on streaming and attempts to classify objects in each frame of the stream. Finally, a bounding box is placed over the object with a label and prediction chance that the classification was correct.
IMPLEMENTATION
Incorporating our innovative design approach, we integrated a sophisticated text-to-speech function that enables users to tap the screen and hear descriptions of the objects in front of them. We opted for this method because naming objects in real-time proved too complex and rapid for users to comprehend, given the high-speed decoding of objects every second. Attempts to implement a queue-based text-to-speech system resulted in performance lag, as the app struggled to keep pace with naming each object precisely as it was detected. To ensure seamless user experience, every activity within the app is equipped with a responsive text-to-speech feature, facilitating effortless navigation throughout the application. Additionally, we included a Google Form for users needing assistance, along with an FAQ section addressing common questions. The app is meticulously designed for intuitive user accessibility.
