A Review Paper on Vision Based Identification, Detection and Tracking of Weld Seams Path in Welding Robot Environment

,


Introduction
This paper is aimed to review the previous technical works and algorithms on various methods of detection, identified and localisation weld seam path in welding robot environment in vision systems.The paper starts with a block diagram to show the sequence of approaching the literature review.Figure 1 shows the block diagram of the sequence approaching the literature review.The review included discusses the advantages, drawbacks and limitations of approaches from the previous researcher according to weld seam path tracking.In the end of the paper, several recommendations will be stated for future study in identification of weld seams path.

Related Works
Vision system has a tremendous scope in the shop floor, not only in quality control but also in the manufacturing field, especially to enhance the utilization of automation using robots.The integration of computer vision with robot motion is steadily progressed by adding visual feedback.The increasing demand of the welding process in the production shop heavily depends on the skilled operator.A variety of algorithms and methods are used to get the accurate and efficient results.
The welding can consider a higher skilled profession and need many years of experience and training to become an expert.It is also a very dangerous profession but has high demand.Nowadays, most of the welding robot applications are programmed through 'teach and play' type by humans.It need some time and expense to programme paths and refined welding parameter in each new part.To bring the benefit of welding robot, the mass production from low to medium volume manufacturing or repair and maintenance work must be quicker to weld parts manually.Because of robotics welding systems are rigid and cannot adjust to the variation in the weld seam positions therefore the welders may not meet the high requirements of the welding process.The recognition and positioning of start welding position (SWP) is implemented to realize autonomous robot welding (Chen et al., 2010).The system have two step methods namely coarse to fine which is solve the curve functions of seam and work pieces boundaries by fitting and establish a small window that takes the initial value of SWP as centre.In this method a charge coupled device (CCD) cameras in a relative large range is used without additional light.A robust algorithm for weld seam extraction based on the prior knowledge shows the characteristics of the weld seam to recognize the seam edges and calculate the seam representation (Ye et al., 2013).The aim of the algorithm is to develop a method to extract the weld seam from the welding image.
Jian Qin et al. (2011) described an algorithm of weld seam based on crawling robot by using binocular vision.The aim of the research is a welding image process of weld seam using crawling robot.A cascade algorithm composed of medium value filter and homomorphic filter was used to the welding image de-noising, which had been proven effective then adaptive threshold arithmetic for segmented image namely cascade OTSU (Maximum Classes Square Error).
Stereo vision for path correction in off-line programmed robot welding was proposed by Ryberg et al. (2010).The system uses one camera and a weld tool mounted on the robot hand to grab the images of the weld joint from different positions and orientations.Then the system determines the weld joint geometry in 3D using a stereo vision algorithm and a novel camera model.The aim of the research is to correct off-line programmed nominal robot trajectories for advanced welding.Yirong et al. (2010) has introduced image matching for weld seam recognition based on the nonlinear dimensionality reduction algorithm isomap.There are four steps in image matching method which is template construction, selection of first layer, localization by isomap and selection of next layer.
Seam tracking of intelligent arc welding robot was introduced by Daeinabi et al. (2006).The automatic and intelligent control of arc welding systems is important for ensuring weld quality, raising productivity and improving labour conditions.In this system, the recognition of a weld groove is presented.Image processing consists of several step which is grab an image, smooth the image remove noise produced during the grab, generate histogram, determining threshold value from histogram, binarize the image, label each particle, calculate and read the extreme value of image and also determination of groove-edge location.Menno et al. (2010) used trajectory-based control in a real-time seam tracking for robotic laser welding.The seam tracking algorithms can cope with the accuracy demands of robotic laser welding based on trajectory-based control architecture.The trajectory-based control approach can be used for a number of different operations; (1) teaching a seam trajectory, with prior knowledge of its geometry, e.g.from CAD-data, (2) teaching an unknown seam trajectory, (3) real-time tracking a seam trajectory, with prior knowledge of its geometry and (4) real-time tracking an unknown seam trajectory.Kiddee et al. (2014) has developed a recognition method for the initial and end points of lap joints based on the image processing techniques for Welding Robots.The algorithms mainly insists of four steps in order to link the broken edge and find the T-junctions of the image; (1) smoothing the image and extracting edges of the image by using canny operator, (2) linking broken edges, (3) detecting T-junctions and (4) analysing the initial and end points of weld seams.
The weld seam taken from a camera ahead of the machine and the centre is extracted to detect the weld position and the offset between this point and central axis.Adaptive multi step segmentation technique is employed to increase the probable of real edge of the welds and improve the line fitting accuracy proposed by Moradi et al. (2013).The aim is to reduce environment conditions and improve the seam tracking accuracy.Dinham et al. (2014) develop an autonomously identify fillet weld joints regardless the base material, surface finish and surface imperfections such as scratches, mill scale and rust.The adaptive line growing algorithm for robust identification of weld joints regardless of the shape of the seam is introduced in the system.The image capture used two eye-in-hand stereo cameras left and right mounted on the welding torch.Chen et al. (2007) has described recognition of macroscopic seam for complex robotic welding environment without known varying size, material, shape and surface finish.Binocular vision system fixed on end of effectors of welding robot was used in this paper.An algorithm to eliminate effect of complex changing background and filter the unwanted information is proposed.Shen et al. (2007) has implemented a study on vision based real time seam tracking in robotic arc welding.The visual sensor composed of CCD (Charge Couple Device) camera fixed on the robot end joint and captured the image of welding pool in the topside front direction.The recognition of the welded seam algorithms have three steps which is select an area called window, calculate the offset of torch and the time to rectify the robot is accurately calculated.Shi et al. (2007) developed efficient algorithm of weld seam detection for butt joint welding from a single camera.The basic idea of the algorithms is to find a pair of weld seam edges in local area then starting from two endpoints of each edge, search the remnat edge from starting the two endpoints of each edge.Takarics et al. (2008) has proposed a new way to determine welding trajectory reconstruction based on the intelligent space concept.The system uses two cameras and edge detection with other image processing algorithms including epipolar stereo matching and reconstruction of their three dimensional pre-image to find the welding path in the image.
Automatic welding system assisted by a machine vision system to compute the dimension, position and orientation of the work pieces was introduced by (Parakash, 2015).An Image processing algorithms has four steps which is convert to binary image, morphological operations, skeleton of the image and pixel information gathered to detect the dimension, position and orientation of the work piece.Dinham et al. (2013) has implemented an automatic identification and location of welding seams for robotic welding using computer vision.The systems enable the robust identification of narrow weld seams for ferrous materials combined with reliable image matching and triangulation through the use of 2D homography.Dinham et al. (2013) has introduced an algorithm for detecting the weld seam in a butt-joint configuration for robotic Arc Welding of mild steel materials using computer vision.The methods developed use Hough transform to detect the outside boundary of the weldment so that the background can be removed.Chang et al. (2012) has developed a new seam-tracking algorithm through characteristic-point detection for a portable welding robot.The aim in the system is to track the seam for weaving weld path planning using a laser displacement sensor.The algorithm consists of four steps; (1) scanning, (2) filtering, (3) generation of the reference points and (4) path planning.
A weld pool image centroid algorithm for seam tracking in arc welding process was proposed by Ding et al. (2009).The weld pool images are captured by a camera arranged ahead of a welding torch.Image processing techniques are employed to analyze the features of the weld pool and its surroundings and the centroid of weld pool image is extracted as a parameter for seam tracking.Fernandez et al. (2011) developed a low cost system for weld tracking based on artificial vision in which the laser is setup perpendicular to the work piece surface with the webcam on the robot arm oriented towards the projected laser stripe at an angle of 45°.The software tracks the deepest point in the welding seam using algorithms and positions the welding head above it.Kong et al. (2007) has used corner detection technology to recognizing the initial position of the weld in global environment.The works piece image is captured using a CCD camera fixed on the end of the arm welding robot.Then image will be segmented from background using image processing include smoothing, sharpening and region segmentation.By applied Haris operator, the initial position of weld seam will be detected.

Haniff et al. (2013) and Haniff et al. (2014) has used a new correction technique known as Correction of Defect
(CoD) for automatic defect detection by using 2 vision sensors as its core.The approach stresses on the vision algorithm used which mainly focus on the shape matching properties to identify defects occur on the works pieces.In this systems used gaussian smoothing features determining better image results compare median filters.The defects occur provides with information of height (z-coordinate), length (y-coordinate) and width (x-coordinate).

Analysis of Indentify, Detect and Tracked of Weld Seams Path
There are two major parameter used to evaluate the experimental results.They are intrinsic and extrinsic.The widely used parameters are accuracy, effectiveness, robustness, smoothing, efficiency, realistic and illumination.Intrinsic parameter is related to the camera's coordinate to the idealized coordinate system and extrinsic parameter is related to the camera's coordinate to a fixed world coordinate system.
Computer vision can be used to detect and identify the weld seams and provide the new paths according to its algorithms to robot to weld parts automatically.However in vision system it has some challenges especially in welding environments such as poor contract, reflection from metallic surface, shadow from parts and imperfection on the workspace like rust, mill scale and scratches.This would be one of the major drawbacks in welding robot.
The previous researchers implemented two CCD cameras with references angle (Chen et al., 2010) and one CCD camera (Ye et al., 2013) parallel to workspace area to get the input images.A CCD cameras is fixed on the end-effectors or welding torch of the welding robot.There is several joint type of welding where butt, lap, fillet and Tee weld joints are.The work piece also have a different type of weld seams which is S type, line type, zig zag type, space curve, line with groove and S type with coarse surface.
In literature reviews implementing the weld seam identification, there are many algorithms involved depending on the researchers.Least mean-square error rule is used to calculate the best fit to recognize initial start welding position (SWP) (Chen et al., 2010).The results show that the guiding error is less than 1.1mm which meets the requirements of practical production when there are no occlusions and light reflections for the typical planar "S-shape" and spatial arc seam.Even though the results meet the practical production, the data fitting and corner detection in local range cannot solve the problem of the irregular curve seams.
Other approaches to detect welding seam location are based on prior knowledge based on characteristic of weld seam (Ye et al., 2013).The system can be used to extract seams of different deviations from noise-polluted images efficiently and also extract the weld seam even when the image is noisy, and it is quick enough to be applied in seam tracking technology.However it must need the references feature value from prior knowledge.The results are accurate under different welding condition with running time less than 10 ms, quick enough for seam tracking applications.A cascade algorithm composed by medium value filter and homomorphic filter was used in welding image de-noising, which had been proved effective as an algorithm has overcome the problem of fuzzy edge and has removed burr (Qin et al., 2011).However the system lacks edge detection and thinning.
Isomap method described the hidden structure of the image data manifold, which in the case of weld seam recognition is the relative location of the seam edge (Yirong et al., 2010).The result of matching proves that the method has a potential to solve similar problems of image matching and image clustering.However the isomap method can only find meaningful low-dimensional structure hidden in their high-dimension.By comparing various interference sources such as rust, stain, scrapes and oxide films using an image, the groove edge is smooth and continuous while the other interference sources are curved edges with discontinuities (Daeinabi et al., 2006).In this method a local small window is used to consider a straight line of the groove edge.However the algorithm has recognized the weld-groove image taken under mild environmental conditions.Meanwhile, Graaf et al. (2010) presented the two trajectories which is teaching and tracking.The time seam tracking procedure is well suited for welding of moderately curved seam trajectories and time teaching procedure is well suited for accurate welding of heavily curved seam trajectories.Accuracies are 0.1mm for a line trajectory to 0.5mm for the curved sine trajectory.However the system has been performed using a line trajectory and a curved sine trajectory.In Kiddee et al. (2014), authors proposed the T-junctions where the conditions of angles are checking and coordinate comparing will be detected as initial and end points of weld seam.The technique applied in this work is simple and effective can be used to recognize the initial and end points of the weld seam of lap joint.However there are many parameters to be assigned.This will lead to the lack of flexibility.Moradi et al. (2013) proposed the accurate and stable system in real time NDT testing machines with the average tracking error of 1.5 pixels, approximately 0.25mm.It's based on natural light and does not need any auxiliary light.By using the adaptive threshold segmentation technique, the environmental lighting condition will be decreased.However the output of adaptive threshold cannot be used directly for feature extraction to increase seam tracking accuracy and decrease executing time and the seam must be thinned.Adaptive line growing algorithm is capable of automatically identifying the weld joints for typical fillet joint configurations for aluminium and mild (Dinham et al., 2014).The method is capable of adaptively detecting fillet weld joints of any shape without prior knowledge of the shape or position of the joint in the image.The mean position error is within 1.3 pixels with the resulting localisation errors within ±1 mm which is acceptable in most industrial robotic arc welding applications.However the algorithm is based on the assumption that the weld seam is darker than the surface of the work pieces.

Another work in
In Chen et al. (2007) to recognize seam from complex environment directly is more difficult.Background eliminating are selected according to the different situation is one of the proposed method to eliminated the environment effect and will be recognize seam in different environment.However the method only recognizes two different shapes and material seam.Meanwhile, authors proposed seam tracking algorithms by consider the calculation of torch offset (Shen et al., 2007).It was effective where the error of seam trajectory less than ±0.2mm could meet the requirement of production.However the performance and reliability of the system still needs future improvement.
Another approach to detect weld seam is by finding a pair of weld seam edges in local area and search the remnant edge from starting the two endpoints of each edge (Shi et al., 2007).The approach offers the solution of weld seam detection where it can be applied in most types of weld seams in butt joint welding and it can detect both the weld seam and endpoint in a single image.It is suitable for most types of weld seam in butt-welding and it has initial welding position accurately if adjust the angle view of robot.However the algorithm is insensitive to variance of background image.It cannot work when the reflection of the work-piece is strong to submerge the weld seam in the image.Meanwhile, in Takarics et al. (2008) authors presented an image processing algorithms including epipolar stereo matching and reconstruction of three dimensional pre-image to determine the welding path in the space and to give the welding trajectory of the robot.The average distance between the measured and the reconstructed welding plane is 1.96 mm and the average vertical distances between each point of the splines is 1.29 mm.However it need image processing and the transformation interface fast enough.
The flexible system can adapt itself to the changes in the position and dimensions of the work piece introduced in Prakash (2015).However the system is simpler and effective in improving the quality of the weld and also increases the welding efficiency.In Dinham et al. (2013) can provide a 3D Cartesian accuracy of within ±1 mm which is acceptable in most robotic arc welding applications.The matching technique is more reliable than NCC as it is not affected by local pixel intensities.The seam can be identified without prior knowledge of the geometry or location of the seam from a single image.It works well even in the presence of imperfections on the surface of the steel such as scratches and mill scale.However it is difficult to estimate the homography transformation using RANSAC as forming correct matches is unreliable in welding images.
Another work proposed using Hough transform to detect the outside boundary of the weldment (Dinham et al., 2012).The algorithms is capable of not only detecting straight line weld seams but can be used for curved, saw tooth and other various shapes of weld seams.It detects both straight and curved welding seams without prior knowledge of the location of the seam.Meanwhile, the method has its ability to detect not only the seam but also the three-dimensional configuration of a given profile and its potential for application to a compact sensory system (Chang et al., 2012).It has be end enveloped to work on tasks with very irregular profiles and uneven bead surfaces.However the algorithms cannot recognize characteristic points of circular patterned butt welding or fillet welding.

Summary of Weld Seam Path Identification
From the previous literature concerning the identification, detection and tracking weld seam extraction, we have noticed that this issue has been investigated widely and in different ways.However many researcher have sought to work on the different ways how to get the input image from camera either integrated with external light source or control the welding environment to reduce the noise.On the other hand, the method and algorithms that has been developed is not too robust where it only suitable to apply at certain type of welding which is in butt-welding.We feel that further investigation and improvement can be undertaken to ensure the systems is more robust by exploring some of the approaches in the literature reviews.
Most of the previous approaches investigated in indentifying, detected and tracking the weld seam; however the approaches cannot detect and locate the position of the starting point and end point of weld seam except (Chen et al., 2010;Prasarn et al., 2014;Shi et al., 2007;Kong et al., 2007).Because of that the further investigations can be taken to improve the approaches that have been developed in the literatures especially in (Chen et al., 2010;Prasarn et al., 2014;Shi et al., 2007;Kong et al., 2007).The other aspect of interest here is the use works piece material and different type of weld seam.Most of the previous approaches discuss the material must be not too light because of the reflection and background should be considered also.The type of weld seam normally use S type, line type, zig zag type, space curve, line with groove and S type with coarse surface.From the literature they have limitation of the approach where some of it can detect successfully 3 or 4 type of weld seam.It was required to propose a new approach to improve the result where it can detect most type of weld seams.

Conclusion
From the literature review, the identification, detection and tracking weld seam path is not new research problem but detection of the location and position from start point and end point of welding needs further to investigation more.Nevertheless there are plenty of studies involving the step to solve the problem of welding environment, reflection material, blurring image with noise or unwanted information, background selection, work piece type of weld seam and filtering selection.
For the seam identification, there are no preferred technique that has be chosen as basic seam identification approaches and combining with the improvements of various techniques could be robust and fast enough.In term of detect and recognition of the weld seam path, most of the previous algorithms adopted the multi-image and single image from CCD camera or stereo camera mounted on the welding torch.A robust approach needs a faster algorithm that works with various types of welding and works pieces.The best way is to use algorithms that can offer faster processing and reduced memory requirements.

Figure 1 .
Figure 1.Block diagram of the sequence approaching the literature review