Designing a Visual Inspection System for Quality Characteristics Dimensions

Industrial inspection is one of the crucial tasks to ensure quality conformance of products. The inspection tasks can be done by using several methods like non-scaled go/not go gauging, measuring instruments, or advanced non-touching tools. In this research visual inspection using a developed optical system is conducted. One of the aims of this research is to design an on line visual inspection system that is capable to test geometrical quality characteristics of 2-D machined products. The design process includes developing an economical optical system to acquire inspected product’s images. Image processing tools are utilized to deal with the product image; and extract features of its geometrical characteristics. A neural network-based methodology is developed and applied to decide whether the product conforms to pre-specified tolerances. The results of the developed methodology are compared to some statistics based visual approaches from the literature. The results show the goodness of the system as an automated visual inspection system and prove its superior performance with respect to other methods.


Introduction
Traditional measurement is used to check and control dimensions' computability.Although it is the common and simple means, however, there are several disadvantages including its long time consuming, inconsistent results between operators (lack of reproducibility), besides human error and fatigue (irrepeatability).Increasing customer requirements regarding better quality products and lower costs are challenges that should be resolved.Automated 100 % inspection can be economically used to satisfy such customer requirements.The visual inspection systems have received considerable industry attention in the recent years (Hassan M.H and Diab S. L., 2010, p363-375, p79-84).They can be easily embedded in the production process to carry out on line inspection.
On line inspection offers many advantages such as increasing inspection speed, accuracy, reliability, reproducibility and repeatability.Although there are many different applications in industrial visual system, however there is no universal system for visual inspection, but almost each application has its unique developed system.
The purpose of this research is to develop a visual inspection system to perform an on-line dimensions inspection that helps segregating non conforming (out-of-tolerance) products from the production line.It is applied to inspect 2-D machined products with multiple characteristics of know tolerances.The system design covers the following aspects to perform quality control:  Developing an active and simple optical system to acquire images of inspected products.


Applying image processing and neural network (NN) tools to decide accepting/ rejecting inspected products.
After transmitting images to the computer, approaches based on image processing are applied to extract their features.Back propagation neural network is used to distinguish between within and out-of-tolerance products; and consequently accept/reject the inspected product.The results will be compared to the statistics-based visual approaches, namely, correlation and mean square error (MSE) introduced by Hassan and Diab (2010, p363-375).

Literature Review
Checking whether the dimensions of a product are within specified tolerances and the products have the correct shapes are ordinary tasks for industrial quality control systems.Such tasks involve characteristics of products in two or three dimensions, namely, the inspection of dimensional quality (E.R. Davies, 1998).
There are several application fields where image measurements can assume a great relevance.Chen and Ventura (1995) introduced a vision-based shape recognition and analysis system of machined parts.Their work involved the development of machine vision algorithms for automated inspection of production parts.The inspection system consists of three parts in series: segmentation, recognition and analysis.The input of this system is a set of ordered boundary data extracted from the product image and the output includes the identity of this product: its pose, dimension, and out-of-profile error.They used computer experiments to show that their proposed algorithms are accurate and fast.Kim et al. (1999) introduced an efficient technique for solder joint inspection.Using three layers of ring-shaped light emitted diodes (LMDs) with different illumination angles, three frames of images are sequentially obtained.From these images, the regions of interest (soldered regions) are segmented, and their characteristic features including the average grey level and the percentage of highlights (referred to as 2D features) are extracted.Using back propagation algorithm of NN, each solder joint is classified into one of the pre-defined types.Hunsicker et al. (R. J. Hunsicker, et al., 1994) introduced an inspection system of screw threads for compliance with manufacturing standards.Edge detection algorithms (based on linear interpolation to the sub-pixel resolution) are applied to detect regions of interest.Each region is matched with multiple models of threads, since the dimensions and positions of the inspected threads are allowed to vary.The system has been tested on the production line and has been shown to perform better than manual measurement.
Active shape models are employed by Hunter et al. (1995).Their inspection approach is based on the identification of characteristic or "landmark'' points (i.e., points that exist in all aspects of the object) in images and on the recording of statistics concerning the relationships between the positions of the landmark points in a set of training examples.
Zeng Zhou et al. (2010) introduced methodologies for locating and identifying the components on a printed circuit board (PCB) used for surface mount device inspection.The proposed method consists of two stages: solder joint extraction and protective coating extraction.Solder joints are extracted by first detecting all the highlight areas, and then recognizing and removing the invalid highlight areas which are mainly markings and via-holes.
An approach to the problem of laying out parts on a stock sheet to minimize scrap Losses is reported in (S.Anard, et al., 1999).An integrated system is developed that incorporates a machine vision module for acquiring the images of irregular (non-convex) parts and sheets, polygonalizing them and storing them in a database of parts and sheets.
A method for dimension measurement of spur gears based on machine vision is proposed by Du et al. (2011).The method concentrates on developing measuring methodologies and algorithms to get the gear's key parameters, such as, outside diameter, root diameter, number of teeth, and estimate the module of gear.The tools used are image segmentation, binarization processing, median filtering, and boundary tracking.An approach to measure the width and depth of keyway and to count the number of teeth is introduced.The measuring process is completed by automatically generating the dimensional drawing of object.
The composing structure of the gear measuring system based on machine vision is introduced by Xiangwei (Xiangwei C, et al., 2005).The software system frame of gear measurement is established, and used in measuring arithmetic of gear dimensions and parameters.It includes the extraction of the edge contour of gear by taking advantage of improved self-adaptation median filter and threshold segmentation and edge detection and label; then it carried on actual measurement to straight-cut gear with keyway by using random Hough transform to get the centre of the gear wheel and analyze the error of measuring.Their experimental results show that fast-accurate measurement for basic parameters of the gear can be realized by using a non-touching measurement method of machine vision, which has great significance to promote the progress and development of gear measurement technology and gear industry.Automated vision inspection system for the size measurement of work pieces is introduced by Lei and Zhou (2005).Based on the fundamental theory of the wavelet transform, an edge detection method is used in the feature extraction of a bearing image.Leta et al. (2006) presented a computer vision system to extract and analyze image features in order to recognize and/or measure it.Computer vision techniques are used to measure parts and overcome the common difficulties of traditional inspection.The parts conformity analysis using a non-contact measurement system has been adapted especially to small objects, where accurate instruments are not so simple to be used.
A system to control quality of press part products based on CMOS web-camera (webcam) in a production line is introduced by Akbar and Prabuwono (H.Akbar and A.S.Prabuwono, 2008).Their objective to develop image processing algorithm which can measure the dimensions of parts and decide on their conformance is realized.The system is divided into hardware and software components; and the dimensions of the part can be obtained by calculating the pixels values.
The research work by Hassan and Diab (Hassan M.H and Diab S. L., 2010, p363-375, p79-84), is an on line, visual, and attribute-based inspection.They utilized image processing techniques to inspect product dimensions with multiple characteristics and pre-specified tolerances.Their approaches in (Hassan M.H and Diab S. L., 2010, p363-375) are based on statistical indices that reflect the dimensions deviations from their nominal sizes.The resulted decision is either to accept/ reject the product.Their research in (Hassan M.H and Diab S. L., 2010, p79-84) includes developing an index for each quality characteristic and the resulted decision shows for each characteristic whether it is in or out-of tolerance.They developed a linear regression model between the index value and the deviation from the nominal size and validated it statistically.
Garcia, H.C. and Villalobos, J.R. ( 2009) presented a methodology that is useful for adapting pre-existing systems to inspect new components, especially when the characteristics of the new components are similar to those of components already inspected by the inspection system.The methodology is based on automated refinement of the inspection algorithms.Dogar (Anamaria Dogar, 2010) presented an integrated quality control system for manufacturing industry.The proposed system combines several components in order to provide an efficient solution for complex real-time measurements tasks.The system is composed of an image-processing technique which can be adapted to specific requirements, monochrome video cameras, and a laser scanner probe, all integrated in a flexible manufacturing cell.Standard visual inspection tasks are performed by the developed vision system.A system with 6 d.o.f.robot mounted laser scanning device is used to inspect 3D products.
In this research work, an on line active and economically optical system is developed that utilizes NN approach and image processing techniques to inspect product dimensions.Unlike the customized system, the developed system can be used for different product having sizes up to 4x4 cm with multiple quality characteristics and pre-specified tolerance.The results are compared with some statistics-based approaches like correlation and mean square error (MSE).

The Developed Optical System and Visual Inspection Techniques
This section describes the developed visual inspection system including an optical system.In addition, the approaches used to extract the features of the products' images acquired by the computer are presented.It is divided into two parts:  Designing the optical inspection system, and  The inspection approaches The scheme of the visual inspection system is illustrated in Figure (1).The scheme includes an optical system to acquire the inspected products' images and transmit them to the processing unit.The output is the inspection results; which include dimension conformance or non-conformance of each quality characteristic to its tolerance limits.
The system is used to inspect the machined product (with thickness 5mm) presented in Figure (2a).The inspected product contains three features; two circles and a slot.The nominal dimensions in millimetres of the product's mechanical drawing are illustrated in Figure (2b).The 100% dimensions inspection of every part has to be performed online and check whether these dimensions fall within specified tolerance bands (7±0.5mm).

The optical system design
The optical system is designed to fulfil the main purpose of this research, that is, to acquire the images of the products with lowest possible distortion, and to transmit them to the computer.The developed optical system consists of the following components: A photograph view of the optical system used in this work is shown in Figure (3).A description of the components of the system is given below.
Webcam camera: The optical system is based on the use of a webcam camera that is of kind area camera to discriminate the geometrical dimensions of products.The camera is the image acquisition mean that transmits the images to the computer.

Telecentric lens:
The telecentric lens provides a constant magnification within certain range of object displacement closer or away from the lens.This displacement often refereed to as depth of field or "telecentric range".The telecentric lens eliminates parallax error and image distortion.In order to get good image, the size of the front optical component of the telecentric lens must be at least as that of the object largest dimension.Also, the front optical component element (chromatic lens) must be as big as the field of view (FOV).This necessitates making such lenses bigger and, thus, more expensive.The basic idea behind a telecentric system is to place a limiting aperture at the back or front focus of a lens (http://www.opto-engineering.com).By definition of a lens focal point, F, this allows only light rays that are parallel to the lens optical axis to enter or leave the system.Sometimes this aperture is called a telecentric stop.Telecentric lens is consists of: Lens 1, which is a type of chromatic lens and Lens 2, which is the lens of webcam.Lens 2 serves as the telecentric stop of the system.The components are arranged as one unit using a tube with two-opened sides.The camera is placed at one side of the tube and the chromatic lens is in the other side.The distance from the aperture of the camera to the centre of the chromatic lens is the focal length of chromatic lens as shown in Figure (4).
Different reflection and shadows may degrade the image quality and affecting the overall inspection process consideration.A uniform light illumination is needed to overcome this problem.The light technique used in this system is annular front lighting technique using LED lights as a light source.Twelve white LEDs are used dependently, and activated and placed in a plate of wood in a circular form.The LEDs are located such that the angle between each consecutive LEDs is 30˚.This ensures that the product to be inspected receives uniform light from all the directions.Figure ( 5) shows the arrangement of white LED lights.

Positioner of the inspected product:
The inspected part is placed in a position such that the optical axis passes through its centre in a perpendicular direction.There is a flexibility to move the product up and down by using two tubes sliding inside each other to control the position of the inspected product.
The developed system has an advantage over the customized systems which is that it can be used for small-size products with complex shape quality characteristics.The system is considered as universal as it can be used irrespective of the product shape.Also, in case of large size product, the optical system can be mobilized using a controlled arm to inspect different quality characteristic that are apart from each other.

Using the developed system
The visual inspection system consists of the developed optical system, processing unit and display unit as shown in the Figure ( 6).
Positioning the product to be inspected, the camera takes a shot to acquire the image.The acquired image is transferred to the computer, which executes the following operations: image pre-processing, feature extraction, inspection approach application, and generation of results.The results are displayed to the operator in a Matlab window (Clay M. Thompson and Loren Shure.2004).Figure (7) shows the process flow diagram of the typical inspection procedure scheme used in this work.

The proposed inspection methodology
The image processing-based inspection methodology include image acquisition, image pre-processing, feature extraction and inspection procedures.
Image acquisition: VP-EYE Image software is used to acquire the product image then the images are stored in the computer.
Image pre-processing: after the acquired images have been stored, the image pre-processing algorithm is executed through Matlab software, which includes three steps: 1) Convert the image from RGB image to grey-level image using Matlab function"Reg2gray".
2) Thresholding process is used to convert the grey image into a binary image.The grey scale value under specified threshold is assigned to '0' and others are assigned to '1'.The binary value can be expressed as by Jain (Anil K. Jain, 1989) as follows: In this research, the thresholding value is calculated automatically through Matlab and is equal to 0.46.
3. Filtering: the filtering is done using morphological operations; the number of pixels added or removed from the objects in an image depends on the size and shape of the structuring element (SE) used to process the image (Rafael C. Gonzalez and Richard E. Woods, 2002).In the morphological operations, the state of any given pixel in the output image is determined by applying a rule to the corresponding pixel and its neighbours in the input image.The morphological operations are used to remove the noise, which called the morphological filter.The filtration is done by removing all connected components (noise) that have fewer than 30 pixels using the Matlab function (bwareaopen).Afterwards another morphological operation closing is implemented using (imclose) function to perform morphological closing on the image.

Feature extraction:
Features to be extracted for each of those characters of the product, namely, the two circles and the slot.Each character is divided into eight divisions.The steps for extracting these characters are image inversion and segmentation.

1) Image inversion:
It means the change binary of image value '0' and '1' into '1' and '0'.It is a necessary step, because the image of the character is wanted not to be the image of background.The image inversion can be expressed as Where B(x, y) is the binary value and B'(x, y) is the inverted binary value.Figure (8a,8b) shows the image before and after inversion respectively.

2) Segmentation
The segmentation of an image into various objects from the background is implemented.Two segmentations are done; first segmentation is executed by separating each character out from the product image.Figure (9) shows the result of the first segmentation of each character using Matlab software.
Each quality characteristic is centred in a box that is dominant to its size.The boxes of the master product image (image with nominal sizes) are similar to their relevant in each inspected product image.
As the characters of the product are extracted and each character has appeared in a box (the features extracted for statistical approach), a second segmentation is executed by dividing each character to eight-equally regions.These regions are performed by determining the character centroid, starting with a horizontal line then drawing other equi-angles (45 o ) seven lines.These lines are the boundary of 8-rgions.Figure ( 10), for example, shows a circle centred in a box and its divisions including portion number1, 2, 3, 4, 5, 6, 7, and 8.
Let the centroid coordinates of any character is (r, c) and the centroid of portion number one is (r1,c1) for example.The feature required for neural network approach is the set distances, d={d 1 , d 2 , …., d 8 }, which is the distances between the character's centroid and the centroids of the eight division.For example, The distance d 1 between the character centroid (r,c) and that of the first division (r1,c1) is calculated from equation (3) as follows: Eq. ( 3) show the sensitivity of the feature to the characteristic dimensions.Figure (11) shows the centroid of the character as a whole and the centroid of the portion number one.For each character there are 8-distance values, meaning that each image have 24 values of distances arranged in one vector called feature vector.The values of this feature vector are sensitive to any change of character dimensions from their nominal values.
The developed NN inspection model: A back propagation neural network is used as a classifier.The network structure is composed of an input layer, two hidden layers and an output layer.The transfer function used is the log-sigmoid function.The number of neurons in each of the hidden layer is 40 and only one neuron in the output layer.Figure (12) shows the network architecture.The network is trained with a set of products with different dimensions.The feature vectors extracted from the product images are submitted to the neural network training.
The network is trained using the backpropagation training algorithm (J.M. Zurada, 1992).Delta learning rule is applied which includes two phases.During the first phase the input vector x is introduced and propagated forward through the network to compute the output value y for each output unit.This output is compared with its desired value do, resulting in an error signal for each output unit.The second phase involves a backward pass through the network during which the error signal is passed to each unit in the network and appropriate weight changes are calculated.
The developed network, training data and desired outputs are processed through the Matlab software, which automatically train the network and create the error reduction progress curve during the training processes.The resulting error value is very small and very close to zero.

The statistical-based inspection models:
The use of statistical indices to reflect a dimension deviation from its nominal value is introduced by Hassan and Diab (2010, p363-375).Correlation coefficient, Cr, and mean square error, MSE, approaches are used to conduct inspection and their results are compared to the neural network results.Comparisons between the reference image f(x,y) -the image of the product with nominal dimensions-and any of the 280 product images w(x,y) are implemented.Experimental work is conducted using the 280 images to find out the acceptable values of the statistical measures (by applying the first segmentation only and centring each character in a box), as each approach has its acceptable values for each character.Then a comparison is done to classify the products to accepted or rejected product.The correlation between two images w and f can be expressed as follow: Where, Cr is the correlation coefficient, w and f are the matrices of the product and the reference images (having the same size).
The application of MSE as a statistical algorithm can be a useful algorithm in the judgment on the products quality.MSE is computed as follow: By experimental work, critical MSE values are computed for the judgment on the product dimensions' compliance.

Results and Analysis
The developed network is trained with 280 product images (180 with accepted dimensions and 100 with rejected dimensions).The network is then tested with additional 191 product images (95 with accepted dimensions and 96 with rejected dimensions).The method used to extract features in the training stage, is used to extract the features in the testing stage of testing images.Then testing the neural network is done.Matlab displays the acceptance/ rejection decision of testing images in Matlab window.If the dimension of one character in the product is out of tolerance, the product as a whole is rejected.In our approach, the acceptance/ rejection decision results are based on the product as a whole not on each character.The trained neural network is then used to indicate the acceptance/rejection decision.The results show that all acceptance/rejection decisions are completely perfect, meaning that the inspection efficiency is 100%.
Comparing to the statistical approaches, the correlation and MSE coefficients are used for product inspection.
The judgment values on the product for each approach are found by using 280 product images.The remaining 191 images were used for testing step.The test procedure is performed based on a comparison between the tested image and the predefined (master) one.
Using correlation coefficient approach the values of correlation coefficient that can be used for judgment on the product to be accepted or rejected depends on the shape of the character.For the slot, the judgment value is 0.9461, which means that the character that has value equal to or more than this value is accepted otherwise is rejected.For clarification, if the similarity between two images increases, the correlation coefficient increase and vice versa.The value of correlation coefficient for the judgment concerned to circle 1 and circle 2 is 0.8751.Of course, if one character or more is rejected the product is rejected.By testing the results, the correlation approaches give 5 products with wrong classification.This gives percentage error of 2.62% and the inspection accuracy is 97.38%.
For the MSE approach, the values of the judgment concerned to the slot is 0.013468; and for circle 1 and circle 2 is 0.057778.The inspection approach based on MSE, means that when the value of MSE decreases, the similarity increases and vice versa.Using the values of the judgment, any character with MSE value equal to or less than these values, the character is accepted and otherwise, the character is rejected.The results give that there are only two products with wrong classification.The percentage of error is 1% giving an inspection accuracy of 99%.This result is better than that obtained using the correlation approach.
Neural network gives good inspection results.The wrong decisions obtained from the statistical approaches are belonging to circle 2. The correlation coefficient values are very close to one, and any numerical approximation may affect the acceptance/rejection decision.
As the differences between the master and the inspected part's images are not enough to affect the statistical values, the solution of this problem is to enlarge the matrices of each character.Enlarging the matrices does not mean increase the resolution of the camera, but means increasing the FOV of the telecentric lens.
The superior performance of the NN approach in comparison to that of the statistics-based approaches can be a result of: a) The testing products are very close to those of the training ones, and that means if we use more testing products with significant variations, the NN results may be different.Beside, the results obtained from the NN approach don't give the quality characteristic(s) that is out of tolerance in case of rejecting a product.
b) Unlike the statistical approaches, the NN approach is not sensitive for noise.

Conclusion
From the results obtained using the visual inspection system, it can be concluded that: Computer vision offers consistency, accuracy and repeatability, in contrast to the subjectivity, fatigue, slowness, and cost associated with human inspection.The advantages of using a machine vision system for inspection include a decrease in the time required for inspection as well as greater accuracy of inspections and better flexibility than the conventional methods.
The developed optical system is effective to inspect 2-D machined multiple-characteristic products with known thickness at low cost.The system components have low costs comparing to other commercial systems.The web camera and the telecentric lens are cheap comparing to advanced cameras used by other picture capturing systems.Developing the system software is based on Matlab coding; and that prevented costing to purchase ready made software.The maximum product size to be inspected using the developed visual system is 40 mm× 40 mm.To inspect products with dimensions larger than 40mm× 40mm, the telecentric lens should be changed to be large in diameter and FOV.
The inspection approaches are used to segregate rejected products from production lines.The user's skill has no influence in the final result and a faster inspection has been realized in real time.
The neural network approach gave 100% error free inspection results that is better compared to statistical approaches.
The problem with statistical approaches is that the number of pixels that are different between the master and the inspected images are very little compared to the total number of pixels.This gives the reason of high of accepting correlation values or low of MSE values.Accordingly, the statistical approaches are sometimes insensitive to small variations.On the other hand, the sensitivity of NN can be increased by increasing the number of neurons of the hidden layers.The methodology of feature extraction utilized in training and testing the NN makes comparisons for low number of pixels in which differences exist and eliminates unnecessary number of identical pixels from comparison.The problem of NN is its requesting for big number of images to be trained.
Zeng Zhou, L of highlight ar a) Webcam camera.b) Telecentric lens.c) Light emitted diods (LED).d) Positioner of the inspected product.