The system offers a navigation solution that empowers visually impaired users with real-time auditory information to create a mental map of their surroundings. It also serves as a platform for clear communication between students with hearing impairments and their peers. Additionally, the system connects blind and visually impaired individuals with sighted volunteers via video calls to provide support for daily tasks and challenges.

Special features:

• The system helps visually impaired users navigate by providing real-time auditory information about their surroundings, assisting them in creating a mental map of the environment. For instance, the system can audibly announce, "You have a restaurant located approximately 15 meters to the northeast." It also offers a categorized list of nearby places, including restaurants, shops, and mosques.

Inspired by the concept of the app Lazarillo GPS for the blind.

• In the context of students with hearing impairments, this app functions as a platform for clear communication. Users can exchange text messages or use speech recognition to communicate, significantly reducing communication barriers. For example, if a hearing-impaired student wants to order a coffee, they can write their order in the app, and it will utter the text immediately. When a waiter replies, their speech will be converted into text on the student's screen.

Inspired by the concept of the app Connect by BeWarned.

• When students with visual impairments encounter difficulties or need help, our system allows them to connect with sighted volunteers via real-time video calls for guidance and support. For instance, when a student at a large university gets lost, they can use the system to connect with a volunteer who guides them to their destination in real-time.

Inspired by the concept of the app Be My Eyes.

• Furthermore, the system empowers students with disabilities to effectively manage academic schedules, including exam dates. Users can easily add, update, or delete these dates and receive timely notifications, ensuring they never miss an academic appointment.

Tools and Technologies: Google Places API, Google Maps API, Text-to-Speech (TTS) API, SpeechRecognizer, Firebase, Firestore, Firebase Authentication, Google Duo, Android Studio, use case diagrams, interaction diagrams, class diagrams, and system architecture.

Use Cases:

• The system can assist visually impaired users by audibly announcing, "You have a restaurant located approximately 15 meters to the northeast.
• A hearing-impaired student can utilize the app to order a coffee and have their text instantly converted into speech when the waiter replies, their speech will be converted into text on the student's screen.
• When a student with visual impairment at a large university gets lost, they can utilize the system to connect with a volunteer who guides them to their destination in real-time.

Class Diagram:

Use Case Diagram:

System Architecture:

Functional Requirements

Maps Navigation

• The system should be able to display a map
• The user should be able to search for destination
• The user should be able to choose a location directly from the map
• The system should inform the user by his/her surrounding
• The system should provide a list of nearby places based on category around the user.
• The system should provide the details of the trip to the destination
• The system should be able to provide the fastest and best-fit route to the destination
• The system should direct the user by turn-by-turn instructions by voice and captions
• The system should be able to locate and provide the user location

Dialog for the Hearing Impairment

• The system will receive a real-time speech or audio as input
• The system will display a text corresponding to the received speech or audio
• The user should be able to type text and display it on screen
• The user should be able to turn the written text into audio by clicking play button
• The system should be able to distinguish user A from user B by giving them different colors
• The user should be able to choose text from ready-to-use templates and display it on screen

Emergency Video Call

• The system should be able to direct the user to make video call with assistant on Google Duo app

Calendar

• The user should be able to add, update, delete special events dates
• The system should be able to notify the user based on voice, vibration, caption
• The system should display calendar with all added events

Help Center Call

• The system should provide phone numbers and direct the user to conduct a call to help centers

Real-Time Notification

• The administrator should be able to send a real-time notification to all users of the App
• The system should be able to notify the user based on voice, vibration, caption

Feedback Form

• The user should be able to fill a form and submit it
• The system should be able to send the form to a particular email.

Based on App

• The user should be able to register
• The user should be able to login
• The student should be able to add and update profile information

Home Page

Maps Navigation:

The maps page contains three essential buttons; each provides a specific function.

Explorer:

First, we have Explorer button, which uses Places API to retrieve nearby places in the range of 100 meters. When the user presses the explorer button the application retrieves nearby places and computes the distance from the user location to a particular nearby place. Moreover, it computes the direction to a place relative to the user orientation. Then it displays the information on screen one by one for the ease of reading it by a talkback screen reader, and it turns the text to speech while displaying the information.

My Location:

In addition, we have my location button, which displays the address of the user on screen for ease of reading by talkback screen reader

Search:

Third, we have the search button, which enables the user to search for a location using a regular google maps view. The user can type a place in the search bar; the search bar supports voice search. The speech recognition language is based on the language of the application. When a place is given, a suggestions list will pop up, and the user can select the place he/she is looking for from the suggestions list. Once the user clicks on it, he/she will be directed to trip information page.

At the bottom, we have two buttons; the first is go to button, which enables the user to select any place on the map, then the application will direct the user directly to the turn-by-turn navigation in google maps based on walk mode or car mode.

The second button is University buildings button, which contains a list of places that the user can choose from. We can add any place of our choice to the list. Once the user clicks on it, he/she will be directed to trip information page.

Nearby Places Based on Categories:

Moreover, on the maps page, we have nearby places based on categories. It enables the user to look for nearby places in the range of 1500 meters based on the selected category. The application retrieves nearby places and computes the distance from the user location to a particular nearby place. Moreover, it computes the direction to a place relative to the user orientation. The nearby places will be displayed in the form of a list for ease of reading by talkback screen reader. When the user selects a place on the list, he/she will be directed to trip information page.

Trip Information Page:

The trip information page displays detailed information about a place. It displays the place's name, the type of the place, distance to the place, and direction to the place relative to user orientation.

Once the user enters the trip information page, the application will turn the text to speech. The text to speech will repeat the distance and the direction every 15 seconds. The user can navigate turn-by-turn based on car and walk mode. Once the user chooses one of these modes, then the application will direct the user directly to the turn-by turn navigation in Google Maps application.

Sign Up:

Once the user opens the sign-up form, the user must enter email address, password, name, phone number. The password requirement to be no longer than 8 characters.

Log In:

Once the user opens the application, the user has two options for him to use the application. The user can use the application without registration; the user can sign up with his/her own information. After the user has signed up, the next time, the user will login and use the application. For login, the user must enter the same information entered in the sign-up form. If the user used the application without registration, some of the features will not be available for him.

Personal Profile:

User can add more information and modify his/her profile and save it by pressing the save button. The information that he/she can add beside the sign-up information are student id, type of disability (hearing impairment, vision impairment, physical impairment, other), the degree of disability (light, average, strong), age, and the gender of the user.

Dialog (Communicate):

Two users should use it. The first user types a text and displays it on screen, with the ability to convert the text to speech by pressing the voice button. The second user can speak and then the application will convert the speech to text and display it on screen; while the second user speaks the microphone button must be enabled. The first user's text is distinguished from the text of the second user by different colors and different alignments. The users can use the ready to use templates by pressing the template button on the left bottom corner, which basically is text to use without the need to type them again or speak them again.

Emergency Video Call:

The video call is conducted using the Google duo App. When the user has Google Duo account, the application will directly connect him to the person responsible for receiving video calls.

Calendar:

User can add an event by choosing the date and add information regarding the event, then the event is added with a countdown, and it will be ordered by sooner to older. When that date has come, the user will receive a notification to remind him of the event. The event can be modified by pressing the event, and it can be deleted by swiping or pressing the event and pressing delete.

Contact Us:

The user has two options to contact the support team, either by sending a feedback form or calling the support team using a regular phone call. For sending a form, the user should enter name, subject of the form, description of the form. Once the user fills the information and sends it, the feedback form will be reformed as an email message, and the user should click the send button to send it to the support team email address. For the regular phone call, the user will be directed to the phone dial with the support team phone number.

Real-Time Notifications:

The administrator can log in to the firebase site and send a real-time notification to all application users. To send a notification, first, the administrator should enter cloud messaging, then write the notification description, including notification title, notification text, notification image and notification name.

Additional Features:

The user can choose to use the application without registration. However, the user can only use maps, dialog, emergency calls, and contact us, including feedback form and regular phone call. In addition, at the beginning, the user can either choose to use the application in Arabic or English language. When the user signs up, the next time the user enters the application, he/she will be directed to the home page directly.

The application is created by:

• Aymen Nacer
• Ahmed Alrazouq
• Naif Alobaid