Sorting Raisins by Machine Vision System

The financial support provided by the Research Department of University of Tehran, Iran Abstract In this research, an apparatus for sorting raisins has been designed and fabricated based on machine vision system. This system was composed of conveyor belt, lighting box, controlling and processing system unit and sorting unit. Color feature is the most important parameter in classification and sorting of raisins. In order to carry out image processing and to extract useful features of captured images by machine vision a highly efficient algorithm was developed and implemented in Visual Basic 6.0 environment. The algorithm consisted of background segmentation, raisin selection and feature extraction. The developed algorithm initially extracts the raisins by removing the background from the taken images. It then sorts the raisins according to their Hue, Saturation and Intensity (HSI) color features. By a suitable combination of length and HSI color values raisins were graded it two classes. The final step in the algorithm was the calculation of the center of gravity of each raisin to be later used for automatic sorting and rejection of bad raisins. In order to evaluate the precision of the sorter statistical analysis was carried out. Experimental results indicated the accuracy of the proposed system is about 93%.


Introduction
It is necessary to pay more attention to the export of non-petroleum products specially dried fruits such as raisins due to considerable amount of currency earned.According to the latest statistics, Iran is one of the major raisin exporters among countries in the world and has the second rank.Because of manual sorting, the export value per ton of Iranian raisins is the lowest among the countries exporting this product (Javadzadeh, 2008).Manual evaluation and sorting of raisins is costly and inherently unreliable due to its subjective nature.The poor classification and sorting methodology has caused a reduction of exported product.Automatic raisin sorting system based on machine vision can improve the quality of the product, abolish inconsistent manual evaluation, and reduce dependence on available manpower.Therefore, it is necessity to develop a sorting system for automatic quality assessment of raisins before packaging.The application of machine vision for raisin sorting is promising because it utilizes spectral and spatial information.Many researchers have been applied increasingly for product quality evaluation using machine vision in recent years.Lee et al. (1999) used Robotic weed control system by machine vision for tomatoes Majumdar and Jayas (2000) and Paliwal et al. (2003) classified cereal grains using machine vision and color models.Shahin and Symons (2001).graded lentils by a machine vision system.Shahin et al. (2002) classified apple based on surface bruises using image processing and artificial neural networks (ANNs).Shigeta et al. (2004) used machine vision to determine damaged and undamaged chaff in rice whole crop silage.Yun et al. (2002) and Kumar and Bal (2006) used machine vision to determine the rice quality.Lorestani et al. (2006) applied fuzzy logic as a decision support system to grade golden delicious apples.Cho et al. (2007) developed an automatic grading system for green pepper using machine vision.This system consisted of three main components -a feeding individuation mechanism, an image inspection and processing system, and a discharging system.The green peppers could be graded into four classes (large, medium, small sizes, and high curved shape), based on the measurement of two geometric parameters (length and flexure), by automatically activating air nozzles located at each container of different grades in the discharging system.Omid et al. (2009) developed a hybrid separation system, based on acoustic and ANN techniques, to separate pistachio nuts with closed shells from those with open shells in real-time.Zayas and Flinn (1998), Luo et al. (1999) and Tahir et al. (2006) developed the machine vision to clean and classify the wheat.Therefore the machine vision system can use for quality inspection of agricultural products such as raisin.In the case of raisin detection, it is essential to correctly divide raisin images into regions which are desired (raisins with desired color content), undesired (raisins with undesired color content) and background (surface of convertor).Image segmentation is an important and perhaps the most difficult image processing task.Segmentation refers to subdividing an image into regions exhibiting "similar" characteristics.Subsequent image interpretation tasks, such as feature extraction and object recognition, rely on the quality of the segmentation results.
The objective of this paper was to design an automatic sorting machine and an efficient algorithm for quality inspection of raisin based on machine vision.In the following section the design and implementation of an automatic system is presented and discussed.

Materials and Methods
An apparatus for sorting raisins has been designed and fabricated based on machine vision system (Fig. 1).The sorter is composed of the following parts: Conveyor belt: To transfer raisins under camera location.
Lighting box: A lighting system consisted of three halogen bulbs (220V, 60W) and three CCD color video cameras (PR-565S) installed inside lighting box (Fig. 2).
Controlling and processing unit: In this section there are a PC for image processing (Intel, 3 GHz), frame grabber (PXC200), pneumatic valves operated by an AVR microcontroller and three DC power sources (Switching 24V/13A) (Fig. 3).
Sorting unit: In this section there are 90 pneumatic valves ).The distance between each valve is one centimeter (Fig. 4).
The three color video cameras were employed to capture the images.The cameras were mounted at a height of 40 cm on a custom-made camera boom.The captured images are sent to frame grabber.The output of each camera was routed to PXC200 color frame grabber housed in the PC.The frame grabber had a resolution of 486×640 pixels.Each pixel corresponded to an area of approximately 0.5mm×0.5mm.Images were taken while the conveyor was moving with a forward speed of 15 m/min (250 mm/s) to minimize motion effects.
An efficient feature extraction algorithm was designed and implemented in Visual Basic 6.0 in order to classify the raisins (Abbasgholipour et al., 2006).Basically, the algorithm processes real-time image data and extracts specification features in accordance with the thresholds extracted by algorithm.In the other words, the algorithm classifies desired and undesired raisins by color features, and the location (center of gravity) of each raisin on the belt.Based on these features, each pneumatic valve operates and thus undesired raisins can be rejected, after sending an appropriate signal for opening or closing of valve through AVR microcontroller.The sorting unit is of pneumatic type, consisting of an electronic circuitry, a compressor and 90 pneumatic valves which separate undesired raisins from desired ones.

Data Transformation
By studying and composing physical properties of the machine vision system it can be understood that what is defined as color by human.Color spaces are mathematical perception of these properties.All color spaces are three dimensional right angled coordinate systems that are shown in Red, Green and Blue (RGB) color space as intensities of red, green and blue lights.Intensity dominates the scatter in the pixel data in RGB color space with data points forming cigar-shaped regions along the intensity axis (Fig. 5).This type of distribution does not make simple min-max boundary type thresholding methods feasible for raisins.But the values of pixels in the Hue, Saturation and Intensity (HSI) color space are distributed across the whole space (Fig. 6).Thus, the HSI color space is an ideal tool for developing image processing algorithms based on some of the color sensing properties of the human visual system.We surmised that there existed two cuboids' regions, which defined all desired and undesired raisin pixels.However, background should be the region of pixels, consisting of the belt.Therefore, because only two regions "desired and undesired raisins" could be defined in HSI space by planes, the image data was transformed from RGB into HSI color space, before the implementation of the image segmentation algorithm, by using the following equations (Gonzalez and Woods, 1992):

Design of image segmentation algorithm
In order to carry out image processing and to extract useful features of captured images by machine vision an efficient algorithm and the corresponding program were developed and implemented.Basically, the program receives data in real-time and extracts the required characteristics, i.e., classifies desired (raisins with bright color content) and undesired (raisins with dark color content) raisins by color features and determines the location (center of gravity) of each raisin on the image.The algorithm consists of the following three steps: 1) Background segmentation 2) To find out raisin 3) Feature extraction Each step in the algorithm are shown pictorially in Fig. 7 and described briefly in the following sub-sections.

Background segmentation
The composite color of each pixel is separated into the original H, S and I in the range 0-255 by using Equations ( 1), ( 2) and (3).The histogram studies showed that color changes for background and raisins are regular.Therefore, the following code was implemented for removing the background: , control returns to the middle of the column.At this stage, one unit is added to the number of columns and control transfers to the next column.At this point, the algorithm carries a critical task.Once the control reaches the middle point of upper and lower edges, firstly the lower pixels are examined and the number of a desired (NDP) and undesired (NUP) pixels are calculated.Finally, the upper pixels are examined and the total NDP and NUP for a column are calculated.

Feature extraction
By using the following formulas, center of gravity ( G G Y X , ), and color ratio ( Ratio ) for each raisin is obtained: To calculate the grade of a raisin, the value of Ratio is compared with its respected threshold values, i.e., if ≥ Ratio default threshold, then the raisin is desired, otherwise it is an undesired one.Finally, all the examined raisin pixel values are reset to zero to avoid recalculation.The program then enters to the next raisin.The pictorial explanation of the algorithm is shown in Fig. 7. Based on this algorithm, a Graphical User Interface (GUI) was designed and implemented using Visual Basic 6.0.The following code was implemented for feature extraction:

END
An outline of the proposed algorithm approach is shown in Fig. 11.

Statistical Analysis
A statistical model was devised in order to evaluate the precision of the fore-mentioned apparatus as well as to determine effects of various factors on its sorting accuracy.Analysis of variance was performed by SAS statistical software.The studied factors were percentages of raisin's impurity and density.To determine the effect of these factors and their interaction in detection of sorting accuracy, the model of this was performed as factorial statistical design based on randomized complete block design with two factors as impurity and density and three replications.
To prepare of impurity factor levels, desired and undesired raisins were combined together with pre-determined ratios.For example to prepare level I2, eight to nine kilograms of desired raisins were combined with one to two kilograms of undesired raisins.Similarly, other levels were prepared.Density factor was defined according to the average surface occupied by a grain of raisin.For example, the density of 40% was estimated as 30 mm × 30 mm average surface occupied by a grain of raisin.Similarly, the density levels of 60, 80 and 100 percent were defined as 25 mm × 25 mm, 20 mm × 20 mm and 15 mm × 15 mm average surface occupied by a grain of raisin, respectively.In each experiment, the sorting accuracy was measured using the following ratios: where A D and A U are accuracy of desired and undesired raisin sorting, respectively.R DD and R DU are the number of grains of desired raisins that had been classified as desired and undesired raisins, respectively.R UU and R UD are the number of grains of undesired raisins that had been diagnosed as undesired and desired raisin, respectively.These parameters are shown by using a Venn diagram in Fig. 8.

Evaluation of apparatus performance on A D
The data received from apparatus performance on A D (accuracy of desired raisin sorting) were analyzed by ANOVA technique using the SAS statistical software, and the means were compared by (Least squares means (LSM) of multiple range test (Table 1).Analysis of variance revealed that there are very significant differences for both factors of impurity (I) and density (D) on the A D parameter (p=1%).But interaction effect of these two factors and block are not significant (Table 1).Therefore, the main factors will only be effective in the A D parameter.As shown in Fig. 9(a), levels of impurity factor are classified using the Duncan comparison test.According to this figure I1 will be more effective than I2 and I3 on the A D parameter.Since I1 level of impurity factor has highest mean and its mean difference into I2 and I3 levels is significant.Comparison of density levels has been shown in Fig. 9(b) indicating D1 affects better than D2 and D3 on the A D parameter.Since D1 level of density factor has highest mean and its mean difference into D2 and D3 levels is significant.
The LSM test, on data which were obtained from different treatments and best apparatus performance, was also used for data comparison.As shown in Table 2, treatments of I 1 D 1 , I 2 D 1 and I 1 D 2 placed in a class with the highest means (p=5%) in other word theirs means followed by the 'a' letter.Therefore, best apparatus performance will be obtained on accuracy of desired raisin sorting with these treatments (Table 2).

Evaluation of apparatus performance on A U
The data received from apparatus performance on A U (accuracy of undesired raisin sorting) were also analyzed by ANOVA technique using the SAS statistical software, and the means were compared by LSM's multiple range test (Table 3).
Analysis of variance revealed that there are very significant differences for both factors of impurity (I) and density (D) on the A U parameter (p=1%).But interaction effect of these two factors and block are not significant (Table 3).Therefore, the main effects of factors will only be effective in the A U parameter.As shown in Fig. 10(a), levels of impurity factor are classified using the Duncan comparison test.According to this figure I1 and I2 will be more effective than I3 on the A U .Since I1 and I2 levels of impurity factor have high mean and theirs means difference into I3 level is significant.Comparison of density levels has been shown in Fig. 10(b) indicating that D1 affects better than D2 and D3 on the A U parameter.Since D1 level of density factor has highest mean and its mean difference into D2 and D3 levels is significant.
LSM test was again used for data comparison, which were obtained from different treatments and best apparatus performance.As shown in Table 4, treatments of I1D1, I2D1 and I1D2 placed in a class with the highest means (p=5%) in other word theirs means followed by the 'a' letter.Therefore, best apparatus performance will be obtained on accuracy of desired raisin sorting with these treatments (Table 4).
Commonly the comparison Tables of the mean treatments on A D and A U revealed that I1D1, I1D2 and I2D1 are in a class with highest mean.The overall accuracy of apparatus in sorting raisins according to these tables (2, 4) was 93.3% under mentioned treatments.
As another resulting commonly in the previous studies granule products were not processed distinctly.But the designed algorithm can process distinctly raisins in an image and extract features of every one Therefore the fabricated apparatus able to sort raisins with proper accuracy.

Conclusions
In this paper, a raisin sorter has been designed and fabricated based on machine vision.This system is composed of conveyor belt, lighting box, controlling and processing system unit and sorting unit.The algorithm segmentation scheme described here is a novel and simple approach to robustly segment an image of raisin into desired, undesired and background regions.By using this accurate algorithm we can study all pixels of a digital image and obtain the necessary features.
1) Based on the results presented in this paper, we can state that 2) By a suitable HSI color space values raisins are graded it two classes, 3) The overall precision or correct classification rate of this system was estimated as 93.3 percent.
4) The devised machine vision and algorithm for grading raisins is quite general and can be easily adapted for grading other granule agricultural products.

Figure 4 .RFigure 6 .
Figure 4. Sorting unit If the considered pixel is outside the background range, then the program control is at the first raisin pixel.At this point, the background separation is temporarily stopped and control is entered within the range of raisin pixels.At this point, the program saves the coordinates i Y of entering point and explores the column until the exit point coordinates o

Table 1 .
Variance analysis in accuracy of desired raisin sorting for nine treatments and three replications

Table 2 .
Comparison of treatments effect on A D by LSM test Means followed by the same letter are not significantly different (p = 5%). *

Table 3 .
Variance analysis in accuracy of undesired raisin sorting for nine treatments and three replications

Table 4 .
Comparison of treatments effect on A U by LSM test * Means followed by the same letter are not significantly different (p = 5%).