YOLOv5 Model Application in Real-Time Robotic Eggplant Harvesting

Deep learning studies in agricultural automation have accelerated in recent years due to its benefits such as increasing product efficiency and reducing labor force. Deep learning is a powerful tool for automation in agriculture with applications ranging from disease identification and crop yield detection to fruit ripeness classification. It helps to automate various processes in agriculture and to perform time-consuming tasks in a shorter time. It quickly processes the data required for robotic harvesting systems and makes it available to the system. In this study, a machine learning study was carried out to be used in the robotic harvesting of eggplant fruit


Introduction
Deep learning is a subfield of artificial intelligence that focuses on the development of neural networks and deep neural architectures to solve complex tasks.This method aims to process and learn data using mathematical model-based systems called artificial neural networks.It has the ability to learn based on large amounts of data and can be used to solve complex problems (Çetiner et al., 2022).It is stated that artificial neural networks consist of multi-layered structures.Each layer of this multi-layered structure can learn features at successive levels by processing input data.Each layer learns more abstract and higher-level features than the previous one.They are called "deep" because they have multiple hidden layers between the input and output layers.Each layer in a deep neural network enables the model to learn complex patterns and representations by incrementally extracting higher-level features from the input data.Deep learning can model complex relationships in data sets by means of the multi-layered structure of artificial neural networks.Each layer of this multilayer structure increases the representation power by transforming the input data into higher-level features.Thus, deep learning models can automatically learn more complex and abstract features and recognize patterns in data sets (Aktaş, 2022).
Deep learning is widely used in various fields and applications.One of the important areas where deep learning is used is medical imaging.It is used in medical imaging, especially MRI analysis.These methods show promise in improving clinical applications and have been applied in tasks such as image segmentation, disease classification, and anomaly detection (Lundervold et al., 2019).Another area where deep learning finds wide application is the automation of industries.Ayadi et al. (2022) emphasized the use of deep learning-based soft sensors to increase automation flexibility in industries.Furthermore, deep learning is used in the field of artificial intelligence, especially in natural language processing (NLP).Deep learning models such as repetitive neural networks (RNNs) and convertors have been successful in tasks such as machine translation, sentiment analysis and text generation.These models have revolutionized the field of NLP by capturing complex linguistic patterns and semantic relationships.Deep learning has made significant contributions to computer vision.Convolutional neural networks (CNNs), a type of deep learning model, have achieved remarkable results in tasks such as object detection, image classification, and face recognition.CNNs enable accurate and efficient analysis of images and videos by automatically learning hierarchical representations of visual data (LeCun et al., 2015).
Deep learning is used in many applications in agriculture.One of the most common applications is the detection of diseases and pests in plants.Deep learning models have been used to develop image recognition systems that can identify diseases and pests in crops based on leaf or plant images (Zhang et al., 2020;Liu et al., 2020).These models can help farmers detect and diagnose plant diseases early, enabling timely intervention and preventing crop losses.Another application of deep learning in agriculture is precision agriculture.By analyzing data from various sources such as satellite imagery, weather data and soil sensors, deep learning models can provide insights and recommendations to optimize crop management practices.This includes tasks such as crop prediction, irrigation scheduling and nutrient management (Ampatzidis, 2018;Jin et al., 2020).Deep learning is also used to increase the efficiency of agricultural operations.It can be used for weed detection and classification, thus, targeted and precise weed control precautions can be taken (Cicco et al., 2017).Another use of deep learning in agriculture is classification.Models have been used for automatic fruit and vegetable classification, allowing faster and more accurate classification according to quality and ripeness (Aji et al., 2019).Deep learning models can enable robots and drones to perform tasks such as autonomous harvesting, crop monitoring, and autonomous spraying (Ampatzidis et al., 2017).These technologies can increase productivity, reduce labor costs and minimize the use of agricultural chemicals.Deep learning is not only used in precision agriculture applications.It has also found a place in livestock management.By analyzing data from sensors and cameras to monitor animal behavior, health, and welfare, it can help farmers detect abnormalities, predict disease outbreaks, and optimize feeding and rearing practices (Umar et al., 2022).In general, deep learning has the potential to revolutionize agriculture by enabling data-driven decision-making, increasing efficiency and reducing environmental impact.However, successful application of deep learning in agriculture requires handling challenges such as data collection and processing, model interpretability, and ethical issues (Dara et al., 2022;Ryo et al., 2022).Eggplant is a vegetable that is widely grown in the world and in almost every region of our country.It is a plant that has a significant share not only as a summer vegetable but also in greenhouse cultivation in our country.Correctly determining the harvesting time of eggplant, one of the important vegetables of Turkish cuisine, will provide ease of marketing and will also affect its shelf life.In this context, determining the harvesting time of eggplant correctly and harvesting at the right time is one of the important parameters.

Material
Eggplant belongs to the Solanaceae family and its homeland is known to be India-Burma and Assam.When vegetable cultivation in the world and Turkey is considered, eggplant (Solanum melongena L.) is among the most produced, consumed and economically high species (Eşiyok, 2012).Eggplant is rich in vitamins, minerals, has a high antioxidant capacity and is rich in phenolic acids.It is called 'egg-plant' because its fruit shape and color look like an egg (Sao et al., 2010).The fruit shape, fruit color, and fruit size show a very large variation in eggplants.Eggplant fruit forms are long, medium long and round.There is a big difference between the harvesting size and the size of the fruit whose seeds have ripened.When a fruit that has reached harvesting ripening is left as a seed, it reaches 4-6 times the harvest size and weight (Uzun et al., 2000, Vural et al., 2000).The harvesting process of eggplant involves several aspects that will affect fruit quality and yield.Researches have shown that the fruit ripening stage at harvesting time also affects fruit quality (Passam et al., 2010).The harvesting criteria of eggplant are affected by several factors such as harvest season, fruit ripeness and environmental conditions.It is known that the harvest season is related to the amount of phenolic acid in eggplant, indicating that harvest timing may affect the nutritional quality of the fruit (Gürbüz et al., 2018).In addition to the developmental stage of the plant and harvest time, fruit type, shape and size are also important items in d recommen generally (Msogoya at   Within the study, the parameters and regulations in the code written below were preferred.
--batch: The number of data point packets to be used by the display card at a time while training the model.
--epochs: The number of times all training data is shown to the trained network and the weights are updated while training the model.
--data: The path to the .yamlfile containing the general path and class information of the file containing the dataset.
--weights: The location of the weight file containing the training coefficients to be used in training the model.
By running these lines of code, the training process of the model was started.The program first checks the YOLOv5 files.The training process is carried out during the determined number of cycles (epoch).

Evaluation Indicators
True Positive (TP) indicates the number of positive images that are correctly categorized as positive.
True Negative (TN) indicates a specific sample number that the model correctly identified a negative sample as actually negative.
False Positive (FP) details the number of samples that a negative sample was incorrectly identified as a positive sample by the algorithm.
False Negatives (FN) indicates the number of samples that the algorithm incorrectly categorized a positive sample as negative. Mean Average Precision: This metric is the precision and recall product of detected bounding boxes.The MAP value scale varies between 0 and 1.The higher the value, the better the result.MAP is found by calculating the average precision (AP) for each class separately and then calculating the average over the class.The result is accepted as true positive if the mAP value is above 0.5. (6)

Research Results
F1 Score, Precision and Recall value graphs were examined according to the error matrix metrics of YOLOv5 algorithms.F1 Score, precision, recall and loss function graphs are given in Figures 3, 4, 5 and 6, respectively.examined which deep learning model was suitable for robotic harvesting systems.The compared models were YOLOv4, YOLOv5 and YOLOv7 deep learning models.They emphasized that the models to be used in such systems were YOLOv5 and YOLOv7.In the modeling study, the best model for eggplant was selected to be used in robotic harvesting systems.When the modeling results were compared with previous studies, it was seen that they showed parallelism according to the study criteria.In all studies, YOLOv5 was found to be the best model.Differences were determined in the sub -models.This difference was due to the structure of the products used, epoch, batch and image processing pixel values.It was determined that the modeling was suitable for robotic harvesting.

Conclusion
The deep learning method has been an important tool in robotic harvesting applications of many products in agricultural automation.Robotic harvesting applications appear as an important method that can contribute to increasing agricultural productivity and reducing labor force.Studies on eggplant harvesting using deep learning methods have a significant potential to increase efficiency and optimize harvesting processes in the agricultural sector.For this reason, in this deep learning model study, it determined which model gave the best results.In the study, it was determined that the YOLOv5m model was the most successful model to be used in robotic eggplant harvesting.All models were trained with 640 × 640 images.Metric values such as "metrics/precision", "metrics/recall", "metrics/mAP_0.5"and "metrics/mAP_0.5:0.95" of the models created with 12 Batch, 110 Epoch were examined.As a result of the comparisons, it was seen that the "YOLOv5 medium" model had higher metric values than the other models.The YOLOv5m model gave the highest score with F1 score of 85.66%, precision of 95.65%, recall of 96.15%, and mAP at 0.5:0.65 of 78.80%.In this study, the potential of deep learning, especially the YOLOv5m model, in automating the robotic eggplant harvesting process was revealed.According to the results, it was contributed to which model would be more efficient in robotic harvesting applications.


Accuracy: This metric is used when the classification problem has a balanced class distribution (similar amount of data in each class).If the class distribution is unbalanced, the problem of capturing the class with a low number of classes may be encountered.Error Rate: It is the rate of frequency of incorrect classifications/predictions in the problem.Precision: It is the success rate of positive class (1) predictions.It indicates how many of the predicted positive classes (classes predicted as 1) are actually positive.Recall: It is the correct prediction rate of the positive class (1).It is the metric value that shows how many of the predicted positive classes have been predicted correctly.F1-Score: It is the harmonic average of precision and recall values.It retains the effect of both Precision and Recall values.

Figure
Figure 6.L