The Fusion of Remote Sensing Images Based on Lifting Wavelet Transformation

The fusion of remote sensing images has become one of the new hotspots in recent years. It can not only improve spatial resolution effectively, but can keep the integrity of the multi-spectral image. In this paper, we take the Hangzhou area as an example and put forward a new image fusion based on lifting wavelet transformation, and carry out the qualitative and quantitative comparison to the spatial characteristics and spectral characteristics with traditional fusion methods, such as IHS transform, PCA transform and Brovey transform. The results show that by lifting scheme wavelet transformation the characteristics and information of the fusion image have been preserved greatly, and the spatial quality of the original image has been better improved. Furthermore, this method is much simpler and easier.


Introduction
The remarkable characteristic of modern remote sensing is to use multiple sensors, multiple resolutions, multi-spectral and multi-temporal technology together to obtain the remote sensing image with different spatial resolution, temporal resolution and spectral resolution (Zhu S L, Zhang Z M. 2000.).Spatial resolution and spectral resolution are the two highly related factors for optical sensors.Usually, the sensor with higher spectral resolusion has lower spatial resolution, though it can capture more radiation information from different object styles in various electromagnetic spectrums, and it can not obtain optimum resolution (María González-Audícana. 2004.).
Image fusion is an important image enhancement technology, which can fuse high spatial resolution and multi-spectrum image.So far, we have developed various remote sensing image fusion technologies, such as Brovey transform (Vrabel, Jim. 1996.),Principle Component Analysis (PCA) ( Haydn R. 1982.),IHS transform (Chavez S, Sides C, Anderson A. 1991.),high-pass filtering (Chang S G, Yu B. 2000.),multi-resolution analysis (Aiazzi B, Alparone L, Baronti S, et.al. 2002.) and wavelet transformation (Chipman J, Orr M, Lewis N. 1995.) ( Li H, Manjunath B S, and Mitra S K. 1995.) ( Kingsbury G. 2000.).Brovey transform, IHS transform, PCA transform are most widely used in many remote sensing commercial softwares.But they have two defects.Firstly, it requires three low-resolution images in Brovey transform and IHS transform, or even more for PCA transform.Secondly, compared to low-resolution image, there will be a general spectral distortion in fused images.During the fusion of remote sensing image, the spectral and detailed information should be both taken into consideration.Wavelet transformation is a famous algorithm in recent years, which can retain effective original spectral information while enhancing spatial resolution.First, original image is decomposed into approximate and detailed image by wavelet transformation, and then fuse the image on each characteristic fields by high frequency substitution.Though wavelet transformation has multi-resolution and more advantages in the image fusion with different image resolution, it has larger computational complexity.
In this paper lifting scheme (Calderbank R, Daubechies I, Sweldens W. 1998.) and a new wavelet transformation based on lifting scheme is introduced.To prove its effectiveness, the experiment with ETM+ multi-spectral image and panchromatic imagery based on the Hangzhou area is conducted, and the spectral quality and spatial quality are also analyzed and compared with the traditional fusion such as IHS transform, PCA transform and Brovey transform.

IHS transform
IHS transform refers to a transformation between RGB space and IHS space, it can transform the color space from red (R), green (G), blue (B) to hue (H), saturation (S) and intensity (I).The former describe the color from physics while the latter from human subjective vision.Firstly, the spatial characteristic (I) and spectral characteristics (H, S) is separated.Secondly, the high-resolution image is stretched according to I component, and then I was substituted by the high-resolution image.At last, IHS inverse transform is carried out to improve resolution and enhance spectral characteristics.IHS forward and reverse transformation formulas are as follows (Zhang W, Ding X W, Chen H L. 2007.).

PCA transform
Principle Component Analysis(PCA)is a multi-dimensional orthogonal linear transformation based on computing the statistical character of the imagery, which is called K-L transform in mathematics (Zhang W, Mao Q M, Zhang X C, et.al. 2005.).It aims to compress and integrate the multi-band imaginary information into one image, using main components information to substitute the information of multi-band image, so that the information of different bands could be furthest displayed in new and irrelevant images.PCA transform first requires PCA forward transformation to obtain n (n 3) components and stretches high resolution panchromatic band and other data to obtain the same mean and variance with the first principal component, and substitutes the first principal component with stretched high resolution image and fuse the image by PCA reverse transformation.

Brovey transform
Brovey transform is a simpler but widely-used RGB color fusion.The algorithm decomposes the phase space of the multi-spectral image into color and intensity, which essentially substitute the I component of multi-spectral image with high resolution image.It simplifies the image transformation coefficience to furthest reserve the multi-band image information, and all the intensity information is transformed into high resolution panchromatic image.The formulas of Brovey transform are as follows: (1) In this formula, R stands for the red band of multi-band image, G stands for the green band of multi-band image, B stands for the blue band of multi-band image, and PAN stands for the intensity of high spatial resolution image.
In addition, Brovey transform need better pretreatments and noise filtering in order to reduce redundant data and non-spectral information.

Lifting wavelet scheme
Lifting wavelet transformation is a new wavelet construction proposed by Sweldens based on lifting scheme.Lifting scheme consists of split, predict and update.This reversible wavelet transformation is applied widely in the image process and becomes the core part of JPEG2000.Besides, lifting method avoids FFT operation of a convolution and requires only shift, addition and subtraction operations, which makes the hardware implementation easier.
The frame of lifting format is as follows: split: suppose there is a source data set s 0 , and split the data set s 0 into two subsets s 1 and d 1 .The simplest way is to split s 0 into even sequence d 1 and odd sequence s 1 , namely split (s 0 ) = (s 1 , d 1 ) (2) predict: according to correlation between image data, a prediction operator p which is irrelevant to data structure is adopted.First act p filter on even signal to obtain the prediction value p(s 1 ) of odd signal.In practical application, the prediction value p(s 1 ) is quite close to d 1 , which can be substituted by the difference between d 1 and p(s 1 ): This difference has less information than d 1 .
update: some characteristics o f the coefficient subset s 1 (such as mean) and s 0 are different, so the update operator u is introduced to update s 1 and preserve the integrity of signal s 0 : So far the subset s 1 and d 1 substitute the original dataset s 0 .When the recursion of the previous process repeats, s 1 will be divided into subsets s 2 and d 2 .d 2 is substituted by difference between d 2 and p(s 0 ), and s 2 is substituted by the sum of s 2 and u(d 2 ).When the recursion goes on the original dataset s 0 is substituted by a more compact expression .
The decomposition and reconstruction of lifting scheme wavelet show as Fig.
This paper uses (4,2) lifting scheme to decompose and reconstruct image data: (5) Formula ( 5) is the forward wavelet transformation based on lifting scheme, Formula ( 6) is the reverse wavelet transformation based on lifting scheme.

Wavelet transform based on lifting scheme
Lifting wavelet transformation is a new wavelet construction based on more efficient and simpler-to-inverter lifting scheme, which needs less operation storage space and has better expansibility.Lifting wavelet transformation fuses the multi-spectral and panchromatic bands of ETM+ images.The concrete process shows in Fig2.
In this paper, we first match PAN image (high resolution image) and MS image (multi-spectral image), and resample MS image to get the same pixel as PAN images, and then stretch PAN image according to mean and variance in the single band image.
Lifting wavelet transformation of PAN image can be conducted to obtain four components by formula (5), which are HH , LH , HL and LL .And among them, HH , LH and HL represent spatial detailed information of high resolution image of "P" resolution, LL represents approximate image of "P/2" resolution.Lifting wavelet transformation of MS single-band image can be similarly conducted to obtain Mi-HH , Mi-LH , Mi-HL and Mi-LL .
Substitute LL with Mi-LL and reverse transformation by formula (6) to obtain Mi .Till then, the fusion of PAN images and MS single-band images has been completed.
This process is repeated until the images of other bands are fused and the spatial detail information of high resolution images are conducted to various bands of multi-spectral images, so that the original images have kept both spectral information and spatial detail information.

Experimental Results
In order to prove the effect of lifting wavelet transform, the Landsat-7 ETM+ multi-spectral image with the resolution of 30m and panchromatic image with the resolution of 15m are fused.First, the 7, 4 and 3 bands of ETM+ image are resampled to obtain the same pixel as panchromatic image.The size of the 7, 4 and 3 bands and panchromatic band of ETM+ image is 512×512.
From Fig. 3 we can see that all the spatial information of fused image is better than the original image by visual interpretation.And among them, Brovey transform leads to larger spectral distortion.But it is difficult to identify which method can keep better spatial detail information or less spectral distortion.Therefore, the quantitative evaluation of the fused images is separately carried out from the spatial quality and the spectrum quality.

Evaluation of spatial quality
In this paper, the entropy and average gradient as the two indicators to evaluate the enhancement effect of the fused images.Among them, entropy represents the abundance of image information, the higher the entropy is, the more 1 1 , , , , information the image has.Average gradient shows the fine contrast, texture characteristic and clarity.Higher average gradient has more image levels and the image is clearer.The formula of the entropy and average gradient is as follows: (7) H is entropy, P i is the grey value which is equals to the probability of i and L are total grey level.
(8) Among them, is average gradient, and are the gradient on row-column direction, M and N represent the row number and column number of image f.
According to formula (7) and formula (8), the entropy and average gradient of original ETM+ and fused images are shown as tab2.We can see that compared to Brovey transform, IHS transform and PCA transform, the entropy and average gradient of the image fused by lifting wavelet transformation are larger, which means this method can hence the spatial detail information much better.Analyzed from algorithm principle, this method preserves the high frequency information by low frequency substitution in wavelet domain, which corresponds to the detail information the multi-spectral image, so the entropy and average gradient of the fused image are larger.

Evaluation of spectral quality
In order to evaluate the spectral information, the correlation efficient and deviation index are adopted to measure the fusion quality.The correlation efficient shows the similar degree of the two images and the alternation of spectral information, while the deviation index shows the offset degree of the two images.The formula is as follows: (9) Among them, R(f 1 , f 2 ) is the correlation efficient of f 1 and f 2 , and f 1 (i, j) and f 2 (i, j) are the grey values of f 1 and f 2 on the point (i, j).u 1 and u 2 are the average grey values of the two images.
(10) Among them, D is the deviation index of compared to f, M and N are the row number and column number of f, and and f (i, j) are the grey values on the point (i, j) of the fused image and original image.
We can see that compared to Brovey transform, IHS transform and PCA transform, the image fused by lifting scheme wavelet transformation has the highest correlation coefficient and the lowest deviation index, which means that this method keeps more spectral information of the low resolution images.Analyzed from algorithm principle, Brovey transform fuses high resolution panchromatic images and multi-spectral images directly by multiplication, while IHS transform substitutes I component with panchromatic image and results in spectral distortion.PCA transform substitutes PC1 component with panchromatic image, which results in the loss of spectral information.The method proposed in this paper makes full use of the thoughts of wavelet transformation and fuses the images on the sub-band to preserve spectral information.

Conclusions
In this paper, we introduce the lifting wavelet transformation fusion of low resolution multi-spectral image and high resolution image.This method decomposes the panchromatic image with high resolution and the multi-spectral image with low resolution by lifting wavelet transformation.It fuses the low-frequency component of the high resolution image into the low-frequency component of the multi-spectral images, which further reserves spectral information of original image and enhances spatial resolution of fused image.The results show that compared to Brovey transform, IHS transform and PCA transform, the lifting wavelet transformation fusion is a better fusion method. ln

Figure 1 .
Figure 1.Decomposition and reconstruction of lifting scheme wavelet

Table 2 .
Statistic parameters of spatial performance between original ETM+ images and fusion images

Table 3 .
Statistic parameters of spectral performance between original ETM+ images and fused images