Suhail: A Deep Learning-Based System for Identifying Missing People

Many people become missing in Saudi Arabia every day, including children, young people, and the mentally ill as well as the elderly with Alzheimer’s. There are many missing people cases that are still unsolved. In Saudi Arabia, people use social media platforms such as Twitter to report missing people cases. The application of deep learning has been successful in a wide range of fields including computer vision and machine vision. In particular, face recognition techniques are effective in saving time and effort, especially when searching for missing individuals. Hence, the goal of this research is to solve this issue by developing a deep learning-based system for identifying missing individuals. This paper introduces a new system called Suhail. The system has been implemented and developed using Android Studio and open-source libraries such as TensorFlow. First, users or governments can report missing persons by uploading photos. Updates and information will then be shared with the rest of the system’s users (volunteers). Once a volunteer discovers a suspect, they scan their face using camera. Then, our application uses face recognition techniques to compare the suspect's photo with photos from the repository. Finally, once a comparison is found, our application contacts the suspect’s family, informs them of his location and then notifies the police that a missing person has been located. By using our application and face recognition systems, we help families and police locate and reach a missing person which saves time and effort. In this study, 759 participants were enrolled to evaluate the performance of the Suhail system. Engagement, aesthetics, and functionality are used to evaluate the user experience. The results of the experiment show that users enjoy the new features of the application and that the system is simple to use. Moreover, the system would help governments and individuals identify missing people faster.


Introduction
Around the world, countless people go missing every day, including children, young people, the mentally ill, and the elderly with Alzheimer's disease. Most of them remain missing. In addition, many children become missing for several reasons (Solbach and Tsotsos, 2017;Almars et al., 2018). Many causes lead to children becoming missing, such as abductions, kidnappings, runaway children, and loss. Over the years, several cases of missing people have occurred in Saudi Arabia. As an example, a Saudi woman raised two boys as her own after kidnapping them from a Dammam hospital in the 1990s. She told them they were born out of wedlock. As a result, finding missing people is a vitial and critical task that can assist governments and individuals in finding lost people.
Deep learning has been successfully applied to a variety of fields, including computer vision and machine vision (Gholape et al., 2021;Solaiman et al., 2022;Kundid Vasić and Papić, 2020). Face recognition techniques are particularly useful for searching for missing people, as they save time and effort. In previous years, facial recognition technologies were used in many ways to tackle this issue. In fact, finding missing persons is the greatest benefit of facial recognition technology (Vinavatani et al., 2022;Kisku et al., 2009). In addition, deep learning for image processing and face recognition has been shown to be effective. Several methods have been proposed to tackle problems in different domains. An example of this is the use of machine learning approaches and deep learning techniques such as support vector machines (SVMs) and Convolutional Neural Networks Konig et al., 2017).
In Saudi Arabia, the number of missing persons is steadily increasing. Missing persons can be found in some cases, but in some critical cases, they cannot be found. Social media platforms are being used to report lost people incidents in Saudi Arabia. Figure 1 shows an example of a user posting a tweet about missing people. However, those platforms provide information that is not official. In addition, the manual process of searching for missing persons is a very lengthy process and takes more time. More time is needed to initiate an FIR (First Information Report) at the police station. Furthermore, the time required to find the lost person is also longer. In the manual process, the amount of time spent on the search for missing people is also less (Nadeem et al., 2022a;Stevenson et al., 2013). To fill that gap, in this project, we utilize face recognition techniques to identify reported missing children by analyzing the child images dataset. An identification system for missing people is useful and important for individuals and governments in order to expedite searches. Therefore, we introduce in this project a deep learning-based system for identifying missing people called Suhail. Initially, the system interface allows individuals or their families to upload photos of the missing people to the system database. A large database is created for the mission people. The system then uses a face recognition algorithm to analyze all photos in the database. In addition, the system adds all missing persons to the Missing Persons list and notifies the government. The system also allows users (volunteers) to cooperate and communicate to find a suspect (missing person). A system scan feature can be used to identify whether a person is missing (available in our dataset) by scanning the suspect's face. In fact, the system will search the database and use a face recognition algorithm (TensorFlow) to make the decision 1=found,0=not found. Finally, if the system identified the suspect, the government and the family of the missing person will receive a notification message with the location and details of the person who found the suspect. Our system offers two main advantages. Firstly, it simplifies the process of finding missing persons in a shorter amount of time. The police are also able to find a missing person more easily when they have their location. The contributions of our paper are summarized as follows:  A new application called Suhail which utilizes artificial intelligence techniques to track down missing people has been introduced. This is the first official system to use face recognition technology to identify missing individuals.
 A face recognition algorithm and cosine similarity were used to find similarities between a familiar person and the photos in the dataset.
 The human element is employed as volunteers in the process of searching for missing people and improving the performance of the proposed system.
 According to the experiment results, users enjoy the new features of the application and find it easy to use. The results show that the total average score of Suhail is 4.80 The remainder of the paper is organized as follows: Section 2 summarises the related work, Section 3 discusses the methodology used in this paper, and Section 4 contains the experimental evaluation of the proposed system. Finally, section 6 concludes the research.

Related Work
Social media platforms are increasingly used in Saudi Arabia to report missing people incidents. It is critical to note, however, that the information provided by these platforms isn't official (Almars et al., 2022). Additionally, searching for missing people manually takes a long time. Initiating a First Information Report (FIR) at the police station requires more time. The process of finding the lost person also takes a long time and the amount of time spent on the search for missing people is also less (Hetal et al., 2018).
Deep learning techniques have gained significant attention in many fields including computer vision and machine vision (Liu et al., 2019;Shelke et al., 2021;Tolosana et al., 2020). Facial recognition is an innovative technology for determining a human face from a digital picture or video frame. Currently, analysts are devising different techniques for facial recognition. The most developed face recognition strategy, which is used to verify clients through ID check administrations, works by pinpointing and estimating human feature highlights from a given picture. Three basic steps are used to develop a robust face recognition system: Face Detection, Feature Extraction and Face Recognition. Computer-based facial recognition systems are a rapidly evolving area of research. Several Computer-based facial recognition systems have been proposed to identify missing people (Pensieri et al., 2020).
Face recognition techniques are particularly useful for searching for missing people, as they save time and effort. The use of facial recognition technologies in previous years has been used in many different ways to deal with this problem (Nadeem et al., 2022b;Almars, 2021;Shelke et al., 2021). A novel framework for detecting missing people in the Internet of Things is proposed. This framework proposes to provide live location information for all areas in a smart city/area. The framework recognizes missing persons and sends live photographs and locations of missing persons to nearby police headquarters when they are caught (Jadhav et al., 2017). Using UAVs, Inata et al. (Inata et al., 2016) propose a system for detecting missing people that is capable of moving quickly and having a wide field of view during disasters. In spite of this, aeroplane-type UAVs cannot always be stable due to weather conditions, rolling, and pitching (Moreno et al., 2016). Alagarsamy et al. introduced a deep learning model to find missing people (Alagarsamy et al., 2022). Based on a predefined pattern, image classification finds similar images and discovers their properties. Matching a specific region of an image to an already present image by segmenting the image. According to the threshold value, pixels are clustered according to similarity.
An Android application was launched in another study to facilitate the task of finding missing persons. Trust members can use this application to locate lost people within a short amount of time through this application (Hetal et al., 2018). Bhanumathi et al. proposed a deep learning-based system to find missing people. The system utilized Convolutional Neural Networks (CNNs) to find missing people (Chandran et al., 2018). Jhanani et al introduced another application based on TensorFlow to detect criminals and missing people (Jhanani et al., 2020). As a result, none of the papers mentioned above has provided enough information to evaluate whether the proposed system is accurate and performs well.
Although several papers have been written to address this issue, the currently available system for finding missing persons is very long and takes a lot of time. This paper introduces an intelligent system that can identify missing people by analyzing images of missing people using face recognition techniques. The development of a system for identifying missing people is useful and critical for individuals and governments to speed up the search process and to develop a community for volunteers to find missing persons.

Method
This paper presents Suhail, a deep learning-based system for identifying missing people. The system interface initially allows individuals or their families to upload photos of missing people. A manual database is created with pictures of missing people for this task. To analyze all photos in the database, the system uses a face recognition algorithm. In addition, the system informs the government and users of all missing persons and adds them to the list. As part of the system, users (volunteers) can also cooperate and communicate to find a missing person (suspect). The system scan feature in our dataset can detect whether a person is lost (available in our dataset) by scanning the suspect's face. This system searches the database and uses a face recognition algorithm (TensorFlow) to decide 1=found, 0=not found. If the system identifies the missing person, the government and the family will receive a notification message containing the location and details of the person who discovered the person.
In the following sections, we first give an overview of the method used to develop our software. Then, we discussed the system design and component design and face recognition algorithm. Finally, we conclude this section by discussing the design of the user interface.

Software Development
In this paper, we used the Agile method to develop our software. The agile process is an iterative approach to software development that helps deliver software in small steps and make changes as necessary (Agi, b,a). Requirements and deliverables can be evaluated and revised during the development process so our team can deliver exactly the software our idea wants. For this study, we chose Agile methodology so that after each sprint, the team can review what needs to be improved and adjust their strategy accordingly. Figure 2 illustrates the main phases of the Agile method. Each phase produces an output that contributes essential information and materials to the following phases:  Requirements: a list of all system and software requirements  Design: a thought-through software architecture design  Develop: a fully developed, proven and well-integrated code for the piece of software  Testing: a debugged version of the software, free of any defects  Deploy software installation.
 The software requirements to develop our system are:  Android Studio is used to develop the application.
 Android Emulator is used to test our application and interact with interfaces.
 TensorFlow is used to implement the face recognition algorithm.

System Design
Design is defined and generated by defining, developing, and designing our system's specific needs and requirements. This section describes the conceptual design, system components, and interface design.

Conceptual Design
As shown in figure 3 we have four types of users using our application: parents, police, volunteers, and government. The application connects to the Firebase server via a wireless network. Firebase authentication layer authorizes users to access Firebase database or storag. All photos of missing persons are saved in a remote database. The system uses face recognition technology to compare the suspicious person with the missing persons in the database to find the percentage of similarity. Section 3.2.3 describes the face recognition model. System components are the processes or elements that help the system run properly and perform its functions.. The main components of the proposed system are listed below:  Login and registration components are responsible for accessing the system. Figure 4 illustrates the Login and registration components.
 The face scan and recognition components allow users to scan a missing person's face, extract features, and compare them to similar faces. The face scan and recognition component IS represented in figure 5.
 The volunteer report component allows volunteers to add details such as pictures, locations, and other details to a missing person case. Figure 6 illustrates the volunteer report component.
 Report components that allow parents to report a missing people case and add details such as pictures, location, and other information. Figure 7 illustrates the volunteer report component.
 The chat component allows users to chat about any case. Figures 8 and 9 show the chat component for the volunteer and parent.  Figure 10 illustrates the model structure. The Suhail system analyzes images of missing people within our dataset using face recognition technology identify missing individuals. Missing persons can be located faster and more efficiently with face recognition systems. Individuals and governments can benefit from the development of a system for identifying lost people to speed up the search process and create a community of volunteers to assist in the search.

Face Recognition Model
Suhail's face recognition mode first receives photos of a suspicious person to identify missing people. Following that, the model utilizes three basic steps to locate the target person:  Face Detection: Digital photographs and videos can be detected and located using face detection technology. The task of this step is to extract the feature vectors for any human face and locate the face locations, similar to object-class detection (Chandran et al., 2018).
 Feature Extraction: Facial features extraction is a crucial step in automated visual interpretation and face recognition. This step detects facial features such as lips, noses, and eyes.
 Face Recognition: This technology identifies faces in digital images or video frames by matching them to a database of faces. It also authenticates users through ID verification services by measuring and pinpointing facial features in an image. To find the similarity of a missing person, we extract the vector representation of the missing person and the photos in the dataset. A similarity algorithm called cosine similarity was then implemented. A cosine similarity (See Eq.1) measure determines how two vectors in n dimensions are oriented regardless of their magnitude (Rahutomo et al., 2012). To calculate this number, two numeric vectors are dot-producted and then normalized by their product, so values close to 1 indicate high similarity. Finally, every photo of a missing person in the dataset is subjected to this process. If a similarity score exceeds 0.8, the model returns" known", otherwise it returns" unknown": (1) Figure 10. Face Recognition Model

User Interface Design
The main interfaces of the Suhail system are described in this section. In the main interface, the user can sign in as a new user or log in as an existing user. Figure 11 shows the registration and login screen where you can join Suhail Journey. On the Login Screen, users can enter their user ID and password to access their home screen. Users can also register by entering details such as their ID, password, name, location, etc. Users' home interface displays information and notifications about missing people at their location. The user can also see the services offered by the application in the form of icons below, with each service represented by a button. Figure 12 shows the home interface.

Profile Interface
A user's profile interface allows them to edit their personal information and specify which type of user they are (parent, volunteer, etc.). Figure 13 shows the profile interface.  Figure 14 shows the Face Recognition interface. Using this interface, users can access the camera and scan the susbisou's face. The system then uses a face recognition algorithm to find similarities within the database. The result is displayed by the system based on the degree of similarity to the user.

Report Interface
On this screen, users can create a missing person report for someone they know, or for someone they find around them. They can also edit it frequently and delete it if needed. Figure 15 illustrates the report interface.  Vol. 16, No. 2;2023 3.4

.5 Chat Interface
A chat icon appears when a user scans the face of a missing person, and the system finds similarities in the dataset. On this screen, users also can interact with the missing person's parents, send pictures, share the location and details of the missing person, and interact with those who interact with incoming reports. The Chat screen is displayed in Figure 16. Figure 17. The number of participants by gender

Experimental Evalution
In this section, several criteria are used to evaluate and test the system's performance.

System Testing
A survey was conducted to evaluate the application procedure, as its usability and accuracy are essential capabilities. A total of 759 participants were enrolled in this study, with 87.9% females and 12.1% males. The total number of participants is illustrated in figure 17.
For assessing the quality of an application, we use the QR method which includes three main categories: engagement, aesthetics, and functionality (Noor et al., 2020). Table 1 illustrates subjective quality items for each category. A total of 10 questions are asked to participants to assess each category independently. Participants answered questions regarding engagement, aesthetics, and functionality, and chose the appropriate scale as follows: 1 = Strongly Refuse 2 = Refuse 3 = Neutral 4 = Agree 5 = Strongly Agree. A high score indicates participants are satisfied with the application experience, while a low score indicates they are dissatisfied.
For each question, the average score is used to assess participants' repsonses according to the QR categories. To illustrate the effectiveness of the system, the overall average score is calculated for all categories. According to Table 2, users are engaging with the system and the application is clear and uncomplicated. Overall, the average score is 4.79. As shown in Table 3, a large percentage of participants in this study found the system to be helpful in finding missing persons faster. Based on Table 4, the overall average score is 4.86, meaning that the system and its functionality are clear and simple to use. The overall average score for all categories is 4.80.

Evalution Discusstion
This study also measured face recognition performance using the confusion matrix. A confusion matrix is used to visualize and summarize the overall performance of the proposed system for detecting missing people. Figure 18 illustrates the experiment result.

Conclusion
There has been great interest in deep learning techniques in many fields over the past few years. In particular, a face recognition system is effective at saving time and effort especially when it comes to locating missing persons. The purpose of this study is to identify missing people by analyzing images of missing people using face recognition techniques. In order to accomplish this, we introduced Shuail, an intelligent system that employs face recognition technology to address the issue of lost people. Individuals and governments can benefit from developing a system for identifying lost people in order to speed up the search process and develop a community of volunteers. A total of 759 participants were enrolled to evaluate the performance of the Suhail system in this study. To evaluate the user experience, three factors are considered: engagement, aesthetics, and functionality. According to the results of the experiment, users are enjoying the new features of the application and it is easy to use. In addition, the system would help individuals and governments identify missing people more quickly. The results of expriemnts show that the total average score of Suhail is 4.80. Future work could focus on improving the detectiong of missing people by utilizing unmanned aerial vehicle (UAV). Furthermore, we will conduct comprehensive tests of the proposed solution in collaboration with the department of lost and found.