A Personal companion device for blind people to help them for navigation,communication and environmental awareness
| Framework | Platform |
|---|---|
| React Native | Android/ios |
| Framework | Usage |
|---|---|
| Firebase | Infrastructure and security |
| MODELS | PURPOSE |
|---|---|
| Gemini | for answering questions |
| Gemini Vision pro | for describe the image |
| yolo | for object detection |
| OpenCV | for face recognition |
| Easy OCR | for extract images |
- Raspberry pi 5
- Microphone
- Webcam
- Power Bank (20000 mah)
- GPS Module
- Gyroscope sensor
- Enviromental Awarness
- AI Assistant
- Navigation System
- Live Location Tracking
- Alert System
- Voice Messaging
- Caregiver App
- Face Recognition
Searches for faces and once detected it measures the distance between the particular face and the camera.If the distance is less than the given safe distance it warns the person by displaying a warning text and directs them to take a different path.
Utilizing the gyroscope sensor data obtained from Pi 5, we process the information using our model to determine whether the user has fallen or not.
It checks for all the objects that are visible is the camera and displays the name of the object which are saved in YOLO V8, along with the probability.
Once a object is detected it uses the color of the object to measure the distance between the object and the camera and if the distance is less than the given safe distance then it warns the person by displaying a warning text and directs them to take a different direction.
This is the main python file which include all the speech to text converting functions which helps in reading out the warnings and texts from all the above mentioned files.And also with the help of Gemini and Gemini Vision Pro the blind person is able to have a full fledged human like conversation and at the same time fulfilling their day to day needs and requirements
This is a React Native project for sixth sense [caretaker app].
To get started with this project, follow these steps:
Make sure you have the following installed:
- Node.js
- npm
- Clone the repository:
git clone https://github.com/Kishorecoder96/sixth-sense.gitcd Mobile_app2.Install dependencies:
npm install
npm startfor python code :
pip install -r requirments.txtpython main.py95 % of prototype is ready and fully functional .we need to improve on navigation and increase the features for navigation