Radial Projection Fourier Transform and its Application for Scene Matching with Rotation Invariance

Scene matching is used for image registration in many fields. There are usually translation and an arbitrary unknown rotation angle between reference and template images. The corresponding scene matching algorithm costs far more computing time than that with small rotation angle and translation. Conceptions of generalized vector image, gray-scale image rotation transformation, gray-scale image point transformation, nature of shift invariance and rotation invariance are proposed to form the foundation of this paper. Radial Projection Fourier Transform (RPFT) is proposed and its rotation invariance is formal proved in this paper. It is applied to the Algorithm of Scene Matching with Rotation Invariance (ASMRI). Calculation on reference and template images can be done separately. Some works can be done before the template images are required. This can improve the matching speed at the cost of more memory. A program to implement the proposed RPFT and ASMRI based on RPFT is coded by means of Visual C++ 6.0. The results prove that ASMRI based on RPFT is not only rotation invariant, but also more accurate and faster than traditional methods. The program can be carried out with hardware.


Introduction
Scene matching refers to an image processing technology which can find corresponding region of template images in reference images or find the correlation between them (Zhao, Fengwei, 2002, P.110-113).The images are of the same scene taken at different times, from different perspectives, or by different sensors (Barbara Zitova, 2003, P.977-1000).
In scene matching area, image matching with translation and arbitrary unknown rotation angle is fundamental and important.There have been many researches on rotation invariant image matching, such as Mean Square Difference (MSD) matching algorithm (Pang S N, 2004, P.519-527) based on the minimum average square difference, Normalized Product (NPROD) correlation matching algorithm (Su, kang, 1997, P.1-7) based on regional gray association, etc.However, those methods can perform well only when reference and template images differ in a small rotation angle.To have a better effect for scene matching with rotation of arbitrary unknown angle, Farhan Ullah (Ullah F, 2005, P.201-209) presented orientation codes for rotation invariant template matching, but the method costs much computing time which may leads to less practicability; Wang Jingdong (Wang, Jingdong, 2005, P.6-10) presented a circular projection matching algorithm, but the correct matching probability of this method is not high enough for practice; Sun Bojiao (Sun, Bojiao, 2008, P43-48) presented a matching method based on normalized cross-correlation, but the selection of step has great effect on image registration.
Conceptions of generalized vector image, gray-scale image rotation transformation, gray-scale image point transformation, nature of shift invariance and rotation invariance etc. are proposed to form the foundation of this paper.
Radial Projection Fourier Transform (RPFT) is proposed and its rotation invariance is formal proved in this paper.It is applied to the Algorithm of Scene Matching with Rotation Invariance (ASMRI).It involves eight steps as follows: step1: extract the edge of original reference image to form reference edge image; step2: extract Subsets of Suitable Matching Points (SSMP) from reference image which includes original reference image and reference edge image; step3: calculate Radial Projection Fourier Transform (RPFT) of each point in SSMP of reference image and save it into memory; step4: extract the edge of original template images to form template edge images; step5: extract SSMP from template images which includes original template image and template edge image; step6: calculate RPFT of each point in SSMP of template image; step7: match the result of MDFT of each RPV utilizing proper comparability measurement to confirm the corresponding points; step8: calculate the rotation angle between reference and template images.
In most scene matching, the reference image is obtained earlier.Therefore, the computations on reference image can be fully implemented offline.In this paper, the result of the computations on reference image was saved into memory.Then the matching process can be quickly done after reading in the saved data when the template images are acquired, which may greatly improve the matching speed.This paper is organized as follows.Section 2 describes generalized vector image in detail.Section 3 describes gray-scale image point transformation and its nature of shift invariance and rotate invariance.In section 4, RPFT is proposed and its nature of rotation invariance is formal proved.In section 5, ASMRI based on RPFT is presented.In section 6, experiments of ASMRI based on RPFT are presented and compared with that of tradition methods.Finally, conclusions are drawn in section 7.

Generalized Vector Image
In order to interpret RPFT better, the formal definitions of concepts of gray-scale image, gray-scale image shifting transformation, shifting image, gray-scale image rotation transformation, set of rotation transformations of gray-scale image, rotating image, rotation corresponding point, generalized image, -dimensional N generalized vector image, vector image shifting transformation, vector shifted image, vector image rotation transformation and vector rotated image are provided first basing on the describing method of Mathematical Logic and Set Theory (Geng, Suyun, 2002, Shi, Chunyi, 2000) Definition 1: A gray-scale image is a map from 2 R to R , where R is the real number domain.In other words, a gray-scale image is a real function of two variables, it can be noted as ( , ) means the grey value of the pixel with the coordinate of 0 0 ( , ) x y .The set of all gray-scale images is noted as GI .
Definition 2: If a map from GI to GI satisfies Proposition 1, the map can be referred to as a gray-scale image shift transformation with the shift of ( , ) dx dy .It can be noted as Definition 3: If ( , ) f x y and ( , ) g x y can be noted as a shifting image of ( , ) f x y with the shift of ( , ) dx dy .That is to say, ( , ) g x y is the result of the shift of ( , ) dx dy on ( , ) f x y .
Definition 4: If a map from GI to GI satisfies Proposition 2 or 3, the map can be noted as a gray-scale image rotation transformation with the rotation of 0 0 ( , ) x y and the angle of α .It can be noted as 0 0 , , x y ROT α . )] . (3) Definition 5: A set of rotation transformations of gray-scale image is defined as: In other words, set of rotation transformations of gray-scale image is a set of gray-scale image rotation transformations with the arbitrary rotation of ( , )  x y or with any rotate angle ofα .
Definition 6: If ( , ) f x y and ( , ) )] ( , ) g x y can be noted as rotating image of ( , ) f x y with the arbitrary rotation of 0 0 ( , ) x y and the rotate angle of α .
Definition 7: x y and 1 1 ( , ) x y satisfies x y can be noted as rotation corresponding point of point 1 1 ( , ) x y based on 0 0 , , x y ROT α .
Definition 8: A generalized image is a map from 2 R to S , in which S is a set.Most transformation of image can be described by the conception of generalized image.The form of S can be different in different applications.
Definition 9: N -dimensional generalized vector image is a kind of generalized image, "generalized vector image" for Vector Space, which is a set of all -dimensional N vectors.For example, a gray-scale image is a one-dimensional generalized vector image; a color image is a three-dimensional generalized vector image; Radial Projection Vector Field expatiated in this paper is a 512-dimensional generalized vector image.That is to say, -dimensional N generalized vector image is a vector function of two variables.It can be noted as ( , ) x y f , i.e.
( , ) x y g can be noted as vector shifted image of ( , ) x y f with the shift of ( , ) dx dy .
Definition 12: If a map from N VI to N VI satisfies Proposition 5 or 6, the map can be noted as a vector image rotation transformation with the shift of ( , ) dx dy .It can be noted as 0 0 , , [ ( , )] x y x y α ROT f .

( , )]
x y x y xy Proposition 6: )] x y x y xy x y f and ( , ) x y g satisfies 0 0 , ,

( , ) [ ( , )]
x y x y xy x y g can be called vector rotated image of ( , ) x y f with the rotation of 0 0 ( , ) x y and rotate angle of α .

Gray-scale image point transformation
The formal concept of gray-scale image point transformations is proposed first, and then its transformation invariance and rotation invariance is defined. ] Definition 17: A gray-scale image point transformation possesses the nature of rotation invariance, if the rotation of input image leads to the same rotation in output image.That is to say, ]

The definition of RPFT
The definition of Radial Projection Transform, Radial Projection Vector, and Radial Projection Vector Field is given first, and then Radial Projection Fourier Transform is proposed based on DFT and modulus operator.
With regard to -dimensional , and the nth element is noted as x y , then RPV of point A in grey-level image ( , ) f x y can be noted Where . radius is the radius of the rotundity neighborhood of point A .
That is to say, RPV of a pixel point is to project grey-level value of its rotundity neighborhood.According to the method in formula 15, grey-level value of different angle is projected to vector of specifically dimensions.
As is shown in Fig1, (a) is grey-level image ( , )  f x y , in which, a specified point A is pointed by a white line;

Definition 19:
Radial Projection Transform (RPT) is a kind of GIPT , it belongs to GIPTSet . : RPT: Calculate corresponding RPVs all points in gray-level image.
Radial Projection Vector Field (RPVF) is the result of RPF on each point of gray-level image ( , )  f x y .It can be noted It is easy to prove that RPF is shift invariant, but not rotation invariant.

Definition 22:
Radial Projection Fourier Transform (RPFT) is the composite map of RPT and PMDFT.It can be formally described as: ( , ) According to Definition 15 about RPV, In conclusion, Formula 19 is proved.

Lemma 2:
If there is a circular shift between -dimensional N vector y and x , then the result of MDFT of y equal to that of x .It can be formal described as: if Proof: According to the property of circular shift (Alanv. Oppenheim, 1998), The MDFT of rotation corresponding points can keep invariance.
Calculate the MDFT of RPV of point A and B .
According to formula 22, if According to the randomicity of A and B , and .
In conclusion, Theorem 1 is proved.

Implement of arithmetic of RPFT
Implement of arithmetic of RPFT consists of three steps.
step1: set up polar coordinates on each point of gray-scale image ( , ) f x y , and uses a reverse transform method [12] to calculate grey level in polar coordinates corresponding to Cartesian coordinates; step2: calculate RPVF of image ( , )  f x y ; step3: calculate PMDFT of image ( , )  f x y .
Overview of this method is illustrated in Figure 2.

Inverse transform in polar coordinates
To find the correspondence between Cartesian coordinate system and polar coordinates, the general method is to transform Cartesian coordinates to polar coordinates: calculate the distance and angle ( , ) ρ θ mapping from the position of given pixel ( , )  x y .
Polar coordinates system (Wang, Qi, 2006, P.6-7): arctan(( -) ( -)) x x y y y y x x where 0 0 , x y refers to the coordinates of a point(noted as K ) in Cartesian coordinates which is the origin of polar coordinates.
Combination at the shortcomings of conventional methods, in this paper, reverse polar coordinates transform algorithm (Wang, Qi, 2006, P.6-7) is presented, which is able to overcome the above disadvantages.Specific ideas are: utilizing polar coordinates of pixel ( , ) ρ θ to calculate its corresponding value of each axis in the Cartesian coordinates ( , )  x y .That is, Generally, location of corresponding points in Cartesian coordinates has a decimal value, it locates between four pixels.Based on the idea of interpolation (Karl C. Walli, 2005, P.8), the original Cartesian coordinate system can construct out of an "optimal equivalence" of pixels.Its concrete steps are as follows: 1) The reciprocal of distances from the four surrounding pixel points to the actual coordinates can be defined as the weight proportion of each surrounding pixel points respectively.The distances are noted as 1 2 3 4 , , , d d d d .
2) Calculate the value of ( , ) After these transformations, each point of polar coordinates has an "optimal" corresponding grey value in Cartesian coordinate, which can guarantee that each polar coordinate point has only a corresponding value.

Radial Projection Vector
Set up polar coordinates with the origin of arbitrary point A in image ( , ) f x y .According to Definition 18, RPV of where and radius is the radius of neighborhood area of point A .
512-dimensional vector is adopted to denote the RPV of each point in this paper.Because DFT is designed to meet the precondition that points of signal N satisfies x RPV (30) where As it is shown in Figure 5, (b) is an image of the size 300 250 × pixels which is a subscene of an image which only differs in an unknown rotation angle with (a).( c) and (d) is respectively the RPV of rotation corresponding points A and B .As can be seen, there is circular shift between the corresponding RPV of points A and B .

Modulus Discrete Fourier Transform
The computation of DFT of -dimensional .To speed up the implementation process, FFT is used to implement DFT in the experiment.
According to Lemma 2, if the RPVs differ in a circular shift with the shift of m , the result of MDFT maintain invariance.
As is shown in Figure 6, (a) and (b) is respectively MDFT of RPVs in Figure 5(c) and Figure 5(d).

ASMRI based on RPFT
According to rotation invariant of RPFT, it can be applied to ASMRI.
Offline calculation on reference images involves three steps: step1: extract the edge of original reference image to form reference edge image; step2: extract SSMP from original reference image and reference edge image respectively; step3: calculate RPFT on each point from SSMP of reference image and calculate corresponding MDFT.
Calculation after acquiring template images consists of five steps: step1: extract the edge of original template images to form template edge images; step2: extract SSMP from template images, select several points from SSMP of original template and template edge images with adjacent coordinate for matching; step3: calculate RPFT on each point from SSMP of template image and calculate MDFT on each point; step4: match the result of RPFT of each selected points in template images with that of reference image utilizing proper comparability measurement to confirm the corresponding points; step5: calculate the rotation angle between reference and template images; Overview of those steps is illustrated in Figure 7.

Extraction of edge images
The edge of the images reflects step change of grey level of adjacent pixels.In this paper, the operator of Prewitt with orientation is adopted (Lu, Zongqi, 2006, P192).The result of this processing keeps the form of gray-level images.Templates of Prewitt with orientation are shown in Figure 8.
To each pixel, calculate the gradient of eight orientations, and choose the maximum of them as the output value.
As shown in Figure 9, (b) is the grey level edge image corresponding to (a), and (d) is the grey level edge image corresponding to (c).

Extraction of SSMP
For some points which are poor in grey level changes with surrounding pixels, the mistake matching probability may be unacceptable.To enhance the matching accuracy and reduce the matching time, SSMP are extracted from images.In this paper, method based on Harris corner extraction method (C.Harris, 1988, P.147-151) is adopted.
Extracting SSMP in template images involves four steps: 1) Filter to each pixel in the image utilizing horizontal and vertical difference operator to confirm x I and y I , which stands for horizontal and vertical changes, and calculate matrix 3) Calculate corner value of every pixel.4) Extract corner which satisfies "corner value is not less than 0.6 multiply the maximum corner value of all pixels" and "its corner value is the maximum of its neighboring pixels".
Extracting SSMP in reference images also involves four steps.It is the same as the method above from step 1 to step 3. Instead, step 4 is to extract corners which satisfy "corner value is not less than 0.30 multiply the maximum corner value of all pixels" and "its corner value is not less than 0.60 multiply the maximum of its neighboring pixels".
The steps of extracting SSMP are illustrated in Figure 10.This can insure that SSMP from template and template edge images can find its corresponding points in reference and reference edge images.

Determination of corresponding points
After extracting SSMP and calculating their RPVs, matching can be performed to determine corresponding points.
As shown in Figure 6, the value of the both ends of MDFT is much larger than the others.The algorithm is easy to degenerate to be the algorithm of find the point of similar sum of the surrounding grey level, because for MDFT, is respectively noted as XM and YM .Combined with the method of the similarity of multi-dimensional vector, similarity measure function (Du, Jie, 2007, P11) is defined as: It is the same principle in determination of corresponding point in reference edge image and template edge images.

Determination of rotation angle
After acquiring corresponding points M and ' M of reference and template images, rotation angle exists between them can be determined.

Calculate corresponding PRV of , '
M M separately, noted as According to the offset when relevance of both vectors is the maximum of all, angle of rotation can be determined.
Angle of rotation between reference edge and template edge images can be determined using above methods.

Experimental Results
A program to implement the proposed RPFT and ASMRI based on RPFT is coded by means of Visual C++6.0.In this section, three scene matching experiments are presented.For each experiment, reference images, template images, matching accuracy and matching speed are shown.Experiments are done on the PC platform with a CPU of 2.8GHz.In those experiments, raidus is 20.The interface of the program is shown in Figure 11.

Experiment 1: rotation between reference and template images
The first experiment demonstrates robustness of the method to scene matching with rotation between reference and template images.Rotation angle varies from 5 o to 355 o with interval of 5 o during taking photos, totally 71 images.Four pairs of images are considered in this experiment as shown in Figure 12.

Experiment 2: rotation and grey change between reference and template images
The second experiment demonstrates robustness of the method to scene matching with rotation and grey change between reference and template images.Rotation angle varies from 5 o to 355 o with interval of 5 o .Four pairs of images are considered in this experiment as shown in Figure 13.

Experiment 3: rotation and Gaussian noise between reference and template images
The third experiment demonstrates robustness of the method to scene matching with rotation and Gaussian noise between reference and template images.Rotation angle varies from 5 o to 355 o with interval of 5 o .Four pairs of images are considered in this experiment as shown in Figure 14.
Four pairs of images are considered in those experiments as shown in Figure 12, 13, 14. (b), (d), (f), (h) is respectively one of the 71 images of corresponding images of (a), (c), (e) and (g).The experimental results are shown respectively in table 1, 2, 3.
As can be seen from Table 1, 2 and 3, matching accuracy rate of this algorithm is higher than the circular projection algorithm (Wang, Jingdong, 2005, 6-10).When the template images have a change in brightness or are interfered by Salt-Pepper noise, matching accuracy rate of circular projection algorithm is not high, while the algorithm proposed in this paper can still achieve a high matching rate.

Conclusion
In scene matching, there is usually a difference in rotation angle between reference and template images, which may lead to mistake matching.The algorithm proposed in this paper maintains a very high matching accuracy with arbitrary rotation angle and keep robust with rotation and gray change, rotation and Gaussian noise in template image.The analysis of the reference image have been completed earlier, together with the RPV is only operated by 1D-DFT, so the matching process needs only less computation.Combination at the different character of original image and edge image can enhance the matching accuracy.
In all, the algorithm proposed in this paper possesses the nature of rotation invariance, has a high matching rate, matching speed and strong applicability in scene matching with rotation invariance, it will surely has a wide application.

Definition 14 :
A gray-scale image point transformation is a map from GI to N VI .It can be noted as GIPT , A set of gray-scale image point transformations is a set of all gray-scale image point transformations, it can be noted as GIPTSet .Definition 16: A gray-scale image point transformation possesses the nature of shift invariance, if the shift of input image leads to the same shift in output image.That is to say, if GIPT GIPTSet ∈ is shift invariant, and ( , ) Point-by-point Modulus Discrete Fourier Transform of vector (PMDFT) is the result of calculating MDFT of each point in a generalized vector image, it is a map from N VI to N VI .It can be noted as of rotation invariance of RPFT Definition 23: Circular shift (Alanv.Oppenheim, 1998) on -dimensional N vector x with the shift of m is still a -dimensional N integer, i.e. , m n Z ∈ .That is to say, there is a circular shift with the shift of m between m % x and x , i.e. m % x is the circular shift vector of x with the shift of m ., then there is a circular shift with the shift of m between , shown in Figure3(Wang, Qi, 2006, P.6-7, Karl C. Walli, 2005, P.8).
I x refers to the maximum integer less than x .Inverse transform in polar coordinates is shown in Figure 4. (a) shows input image and Cartesian coordinate with the origin of point K (240,300), (b) shows inverse transform in polar coordinates with the origin of K .

2 tN
= .If a RPV is the circular shift vector of another one, RPV of 360 elements can not satisfy the nature of circular shift when 2 t N = after extension.According to Lemma 1, if there is a rotation of unknown angle between reference image and template image, RPVs of rotation corresponding points differ in circular shift.
which could not meet the needs of practical applications.The computation of FFT of -dimensional N Gauss smoothing filter to four elements in matrix M , and update M (a)  and (b) shows original reference image and template images, (c) and (d) shows reference edge image and template edge images, (e) and (f) shows SSMP extracted from original reference image and template images, (g) and (h) shows SSMP extracted from reference edge images and template edge images.
is respectively the number of points of SSMP in template and reference images.

Table 1 .
Summary of the experiment 1

Table 2 .
Summary of the experiment 2

Table 3 .
Summary of the experiment 3