Mobile Based IoT Solution for Helping Visual Impairment Users

Blind people have many tasks to do in their lives. However, blindness gene-rates challenges for them to perform their tasks. Many blind persons use a traditional stick to move around and to perform their tasks. But the obstacles are not detected in traditional stick, it is ineffective for visually impaired people. The blind person has no idea what kind of objects or obstacles are in front of him/her. The blind person has no idea what size the object is or how far away he or she is from it. It is difficult for a blind person to get around. To assist people with vision impairment by making many of their daily tasks simple, comfortable, and organized, they will be able to recognize anything (an obstacle for blind people). A smart stick with mobile application can be used. One of the solutions is a mobile-based Internet of Things solution, which is a stick intended to assist visually impaired people to navigate more easily. It enables blindness and low vision people to navigate and carry out their daily tasks with ease and comfort. A technologically advanced blind stick that enables visually impaired people to move with ease. This paper proposes a system of software and hardware that helps visually impairment people to find their ways in an easy and comfortable way. The proposed system uses smart stick and mobile application to help blind and visually impaired people to identify objects (such as walls, tables, vehicles, people, etc.) in their ways, this can enable them to avoid these objects. In addition, as a result, the system will notify the user through sound from the smartphone. Finally, if he/she gets lost, he will be able to send an SMS with his/her GPS location.


Introduction
Humans obtain 83 % of their knowledge from the world by sight [1], according to human physiology references. At least 2.2 billion people have a near or distance vision impairment, according to WHO statistics.
The walking cane is one of the oldest and most traditional mobility aids for people with visual impairments [2]. There have been different forms of assistive technologies available to blind or visually disabled people in the past. The smart phone, for example, tackles some of the issues that blind and partially sighted people faced in their everyday lives [3].
Visually impaired people face difficulties with their everyday and social lives since their eyes receive most of the information from the physical world, which is then processed by the brain. Blindness, also known as visual impairment, affects many people around the world. This condition degrades vital eyesight. Millions of people around the world have a visual impairment, and many of them are blind [4]. The demand for assistive devices has been increasing and will tend to increase. For visually disabled people, there are a variety of navigation systems and tools available. An identifiable object is necessary for a blind person [5].
In everyday lives, advanced information technologies have opened a new path for innovation. Among these information technologies [6], Internet of Things (IoT) offers enhancements and better solutions for visually impaired and blind people. Sensor-based IoT technology has a great deal of potential to reduce the risks that the visually impaired and the blind face in going about their daily lives [7]. The Internet of Things can be used to assist the visually impaired and the blind in identifying and avoiding obstacles, as well as sending their current location in case they get lost. Effective use of the Internet of Things will help solve the difficulties that blind people with vision impairment face while carrying out their daily activities without fear of falling or losing their way.
The advent of the Internet of Things (IoT) technology, smartphones and the operating systems that depend on them [8], which involve a wide variety of smart apps, and the interconnectedness of these technologies make it easy to address many of the challenges that the visually impaired and blind face [9].
In this paper, we aim to develop a low-cost, reliable system for visually disabled people and those who care for them, provide legitimate, useful mapping information that enables a blind person to make reasonable and appropriate route choices in both indoor and outdoor environments. To provide this type of data, the system should consider all objects in the immediate external environment that may act as "obstacles" to the blind people.
The system should recognize any element in front of the blind user. In the event of an obstacle, the system shall alert the blind user with a sound from the smartphone. The system must be able to receive GPS signals and estimate its location. If they get lost, the system should be able to send SMS messages with their GPS location. This paper is organized as follows: Section 2 provides few more related work

Related Work
Ayesha Ashraf et al. [10] developed an IoT powered sensitive smart stick to support visually impaired people. It is integrated with an ultrasound sensor and buzzer for sensing and alerting in the event of any obstacle. Android app is developed with smart stick to generate important notifications and forward them to recorded phone numbers along with GPS location. The model is designed in a way that helps the disabled person to move more easily and comfortably without any kind of risk and it's designed in such a way that its very light in weight and very easy to carry around. The authors showed that if they used image processing that could support the user in knowing the pattern of the obstacles'' objects and their volumes and then returned with the suggestions on how to avoid or communicate with them.
Waheed-Rehman et al. [11] present smart assistive design for blind peoples In [14], the authors designed a smart walking stick with Proteous software for In [15], the authors proposed a blind walking stick with an ultrasound sensor, water sensor, GPRS, GSM, Arduino and Buzzer. During an emergency, GSM and GPRS modules are used to send alert messages and provide location information. In addition, this system contains RF transmitter and receiver which is exploited to detect the fallen stick from the blind people. They showed that the system can detect the obstacle, digs, water in the ground and enhance battery life to be available 12 hours.

Proposed System
The proposed system is split into two parts: software and hardware. Below the overall processes of our proposed system step-by-step: 1) The user turns on the smart stick and open mobile app.
2) Mobile app start connection to the stick using Bluetooth unit.
3) Mobile app start receiving GPS signals from the stick containing the user's coordinate locations. 4) Smart stick start scanning for any object using ultrasonic sensor. 5) In case of any obstacle object, the stick output a sound using buzzer and send to the mobile app to alert the user. 6) If the user presses lost button in the stick, it sends to the mobile app to send SMS to the assistant's phone number that already saved in the app.
The following figure (Figure 1) illustrates the overall system processes, how the parts of the system (mobile app and hardware stick) are interconnected, and how they are controlled by the user.

Software Implementation
The software part is a Mobile App created with Android Studio program IDE [16]. It is an easy and rich user interface that provides services to people who have visual impairments.
The main form of the app (as shown in Figure 2) contains the Bluetooth address where the stick Bluetooth address will appear when it is online. Then we have the latitude and longitude where the GPS coordinates will be shown. User can also save and edit phone number of the assistant's person. There is a reconnect button that the user can reconnect to the stick if it is disconnected.
After connecting to Bluetooth, the app will start receiving GPS signals if the GPS location is not detected, the app will display a message (location not detected), and when the location is detected, a message will appear with the new location.   To save the assistant's phone number, the user will click in the phone number text box and the number pad will appear, then the user will enter the number that the user will be allowed to enter with no more than 10 numbers after pressing the Save Number button, and a pop-up message will appear (The phone number has been saved). If the user presses the lost button in the app, it will

Hardware Implementation
The proposed system is based on the use of two hardware components: A smart phone device and an ultrasonic sensor. The ultrasonic sensor is operating on the same principle as radar in that it evaluates a target's features by decoding radio or sound waves echoes, they generate high-frequency sound waves and analyze the echo received by the sensor. Sensors will calculate the time between sending a signal and receiving an echo to determine the distance to an object. The embedded system receives this signal. As opposed to other sensors, ultrasonic sensors have a significant advantage in terms of noise immunity. The sound waves are sent forward from the sensors to the obstacle, which can detect a distance of up to 12 feet with a 0.3 cm resolution.
The hardware part is designed in the shape of a stick so it will be easy to use and handle by users, a lot of components have been used in it.
Arduino is an open-source interactive board and platform for creating electronic projects [17]. To produce the ultrasound, Trig should be set to a High State for 10 seconds.
This will generate an 8-cycle sonic burst that will travel at supersonic speeds and be received by the Echo pin. The Echo pin can output the sound wave's travel time in microseconds, e.g., the sound wave can move for around 294 seconds if the object is 10 cm from the sensor and the sound speed is 340 m/s (0.034 cm/s).
Since the sound wave must travel forward and backward, the amount produced from the Echo pin would be twice that. As a result, multiply the travel time value obtained from the echo pin by 0.034 and divide it by 2 to obtain the distance in centimeters (Figure 4 shows how to calculate the distance).
The HC-05 is a sensational module that allows to add bidirectional wireless functionality to wireless hardware projects. It can be used as a communication   In this system, two buzzers are used: one to alert the user from the target, and the other to confirm user feedback. The alert buzzer beeps at different times depending on the distance between the ultrasonic sensor and any object (if any) encountering the user during the move, while the other buzzer beeps for one second when the user presses the lost button.

Results and Discussion
In the proposed system, the working scenario is started by the user when the stick is turned on and run the application on the his/her smartphone, followed by an automatic connection established between the stick and smartphone via Bluetooth, finally the stick starts the scanning process to detect any object meet the user during his/her moving using the ultrasonic sensor. If there is any object, the stick sends a message via Bluetooth to the application on the smartphone and then the smartphone alerts the user through the application to avoid that object. If the user loses his/her way or could not know their current location, they can press the lost button in the stick. The application then sends his/her location to the person whose phone number is stored in the mobile app of the visually impairment person. This can help the visually impairment person to be Ultrasonic waves sent to the air by Ultrasonic sensor and identify the waves that reflected from an object. Moreover, an ultrasound at 40,000 Hz has been sent, and move by the air, and in case there is an obstacle in its way, then it will get back to the module. By taking into account the sound speed and the time of travel, then it can compute the distance.
Below are a few screenshots of the proposed system's results in action. After paring the mobile app with the hardware part and When the user opens the mobile app it will show the user a message that say BT connected and Bluetooth Name and Address will be shown. When the Bluetooth is disconnecting it will show a message the say BT Disconnected and Bluetooth Name and Address will be become empty as show in Figure 5.
After connecting to the Bluetooth, the mobile app will start to receive GPS signals and if the GPS location is not detecting the mobile app will show a message (location not detected), and when the location is detected a message will show with the longitude and latitude of the new location as shown in Figure 6.
To save the assistant's phone number, the user will click in the phone number text box, a number pad will appear, then the user will enter the phone number, and then the user will press the Save button to save the number as shown in    If the user presses the lost button in the hardware part, it sends to the mobile app using Bluetooth unit to send SMS to the assistant's phone number with the location and a message will pop up (SMS Sent) as shown in Figure 8.
There is no need to fear in case of visually impaired person lost the road, the stick can send its location and help to find it more easily. Some related works may be similar to the proposed system of this paper [10] [13], but what distinguishes this paper is that the stick is able to connect to the smartphone and benefit from its services such as Bluetooth, SMS service, and GPS services.

Conclusion and Future Work
An assistant stick for visual impaired persons system has been built to help them to do their daily works in an easy way, provided low cost and convenient stick for visual impaired people and who take care of them, also, provided a real-time and useful navigation information stick that enables a visual impaired people to make appropriate and timely decisions on which route to follow in an indoor space and outdoor areas.
The proposed system was easy for using, it identifies any object in front of the blind person. The system alerts the user by a sound from the smart phone if there is an obstacle. The system receives GPS signals and is able to identify the location of the blind person if he/she lost his/her way. The system able to send SMS message contains their GPS location if they get lost. With this project, there will be no fear for the blind to walk around the house or even out to the street.
There are several changes that could be made in the future, like using machine learning and deep learning algorithms to add a voice assistant within the stick so that it can make voice interaction with the visually impaired or blind to guide and alert them if there is a danger to it well before it happens, as well as obtain and carry out orders from the user.