Panoramic Images Automatically Stitching Algorithm Introduction

The panorama is more effective image-based rendering one of the methods. Papers on the panorama generation process, and key technologies on one of the most panoramic picture of the stitching algorithm is classified, summarize and compare.


Introduction
Panorama is a kind of full-horizon and high-resolution images, in robotics, computer vision, virtual reality, military and commercial on a wide range of applications.Web-based Panorama in the real estate and tourism are widely Application, the user can interact homes to enjoy the exotic customs.Use the full range of imaging sensors and cameras compatible with the composition of panoramic cameras, can be a one-time access to a wide range of images -a panoramic picture, but the method of hardware devices require a higher, more expensive, and the need to better shooting skills, to be universal.Image Mosaic approach will be adopted by ordinary camera with the image sequence automatically splicing, is the structural panorama one of the most commonly used method in order to make ordinary photo mosaic to Panorama, to maintain perspective on the visual effects and the consistency necessary to Images that re-projection to a surface.That is commonly used spherical surface, the cube and cylindrical projection, cylindrical projection of the most widely used.Cylindrical panorama in the direction of the level of 360 ° to meet the look, it has the advantage of cylindrical image can be launched into a Plane, greatly simplifies the cylindrical projection and display images quickly.

Panoramic image
Based on the graphic image mapping technology is a new graphic image technology.IBR existing technology probably can be divided into four categories: Based on the panoramic view of the method, based on Morphing methods, image-based methods of the depth of information based on formation the way light field.At this stage, a relatively mature technology is the first panoramic view on the way.In fact, the panoramic view can be simply interpreted as: the perspective used in a fixed camera or video camera in accordance with a certain way (in accordance with the uniform point of view is usually around 360 rotating Degrees) image acquisition, image acquisition after the importation of computer image mosaic, integration processing, generation seamless panoramic images, after the final re-use computer display projector out and provided a partial limited roaming capabilities.Although the panoramic view has its own limitations, such as a single viewpoint, can only be realized within the scenes, such as robots, but because the technology is extremely workable, but also relatively mature, has become the most widespread application of the IBR technology 1. Panoramic view of the current application in the main: virtual environment, game design, film special effects, virtual museums, etc.In commercial areas relatively well-known Apple in the early 1990s, launched QuickTime VR system.

The general panorama generation process
The general panorama generation process including panoramic model of choice, image acquisition, image mosaic, image and suture Panorama show here five steps 2.2.1 Panorama model of choice: According to Panorama projection display of different ways, can be divided into three main modes: cube model, cylinder model, spherical model.This is the three models were the stitching has been good panorama projection to the cube / Cylinder / sphere of the inner surface.There are other display modes, such as a polyhedron is spherical approximation to the method.

Image Acquisition:
There are two methods used to shoot panoramic photography equipment or by an ordinary camera image mosaic again.Ago a way to relatively easy to collect images, but this approach often means that the purchase of expensive photographic equipment, Impact of its generality.Latter, with ordinary cameras in fixed-point shooting pictures and then splicing generate panorama of it is more active, and the panorama generated by the core technology -image mosaic algorithm is the focus of the study.
2.2.3 Image mosaic: the existing panoramic image mosaic generation algorithm can mainly be divided into three categories: Based on the current method, based on the characteristics of the method and based on the phase approach.
2.2.4 Suture image: a good image by stitching, the images also need to deal with some overlap, in order to achieve a seamless mosaic image.Now often used in a simple image suture technology is linear interpolation.2.2.5 Panorama show: get a 360-degree panoramic image, but also to the image projection to the model chosen by the inner surface impressions, and to provide simple browsing.

panorama of the classic generation algorithm
Panorama from the concept to the present, there are many scholars who have the panorama of the Algorithm done in-depth study, McMillan and Szeliski, who is particularly prominent.McMillan and Bishop made a panorama function model, the algorithm based on the camera around 360-degree rotating taken by the camera image sequence for each parameter, which conducted panorama stitching, but the algorithm is not suitable for cameras, because it requires between the two images have more than 2 / 3 of overlap, so If a scene with the camera to collect images on the need to take a lot of pictures, which apparently has increased the difficulty of image acquisition, but also increase the amount of computation error and the amount.McMillan and through the cylinder is trying to achieve geometric constraints automatically match point, but the very core of geometric constraints on the basis of error of over-sensitive matrix, and therefore lead to practical results do not match.Szeliski and Shum in their series of articles in the proposed use 8 parameters of the two-dimensional projection model for projection matrix M:(figure 1-2) Set up two match points overlapping images in their respective coordinate system in the homogeneous coordinates for X (x, y, 1) and X '(x', y ', 1'), Formula One (1) has given them Spatial relationship between the mathematical expression.Assumption that X 'and X' (the brightness of a point (or color), respectively I '(x', y ') and I (x, y), then two overlapping regional counterparts Points between the brightness (or color) and the square for the poor:(figure 3) To minimize, is actually for a non-linear least-squares problems, with L-M can be solved projection matrix M, but the method of calculation and large there are limitations.Need a good initial value, otherwise Solving the time is likely to get the wrong match.Szeliski and Shum improved the 8-parameter model for the 3-parameter model, so although to a certain extent, reduce the volume of operations, but this improvement is within the parameters of the camera correction obtained at the cost of, And the calculation method is still greater.

panoramic image mosaic algorithm classified and compared
The existing panoramic image mosaic generation algorithm can mainly be divided into three categories: Based on flow method, based on the characteristics of the method and based on the phase approach.
(1)Based on the current approach: the area also called on the way the method is by comparing the two images brightness (or color) differences, and the minimum to find the best match point.Description of the above two classic Algorithm that is part of the way, this approach were: Duffin and Barrett in Szeliski on the basis of the restoration of a nine-parameter model of the algorithm.Method based on the area of the shortcomings is that it's obviously too large amount of computation.
(2)Based on the characteristics of the method: This method is the main ideas from one image to extract certain characteristics, such as: points, lines, and so on the edge and use characteristics of this template for the match in the second image in the search.The Ways to increase computing speed, but the right image feature extraction more difficult.How to extract the appropriate image features, many scholars have done in this study, Kim and others used as the outline of objects taken from features, and the bell of overlapping images, and other people to use the brightness of a two (color) or brightness than the (color) Poor characteristics as a template, Zhang Peng, who value the use of a gray-scale information extraction characteristics of a rectangular region as a template to match.Need to pay attention to another issue that is collected in the overlapping part of the image is not a simple plane displacement transformation, and there are still Stretching deformation, and so on, so it also increases the feature extraction more difficult.
(3)Based on phase approach: the use of Fourier Transform, the first wavelet transform, and so transform the image and then use transform the image of some of the features match.But if there is space on the local changes will lead to The method have a greater error.

Improved algorithms
The algorithm proposed in [4] can be summarized as the follows.
Step 1: Camera orientation estimation.Using an internally precalibrated camera, the extrinsic orientation parameters of camera can be determined from an aerial instrumentation system (GPS/INS) and bundle adjustment techniques.
Step 2: Image rectification.An image rotation transformation is applied to each frame in order to eliminate the rotational components.
Step 3: Slice determination.Determine the fixed lines in the current frame k and the previous frame k-1 by the left slit window distance dy/2, and "ideal" straight stitching lines by their 2D scaled translational parameters.The locations of the stitching lines are in the middle of the two fixed lines.Therefore, two overlapping slices in the k and (k-1) frames are obtained.And each of them starts from the fixed line and ends a small distance away from the stitching lines in opposite directions.
Step 4: To stitch the left stereo mosaics and the right stereo mosaics respectively by match and ray interpolation, triangulation and warping mentioned.
In order to satisfy the increasing demands for the remote sensing applications, the UAVRSS was integrated by UAV platform, GPS/INS, RS sensors (including Charge Coupled Devices (CCD), CCD TV camera and Synthetic Aperture Radar (SAR)), mobile ground station, and the RS data processing center [3].The UAV used in RS is designed according to the manned aircraft standards and RS mission initially.The CCD TV camera is mounted on the UAV to acquire the image sequence, which is transmitted to the ground receiver in real-time.We need to create the stereo mosaics in real-time to satisfy the applications.Therefore, according to the speciality of the UAVRSS, the following modified algorithm is proposed based on the parallel perspective stereo mosaics.
Step 1: Camera orientation estimation.The CCD TV camera is precalibrated in laboratory to get its intrinsic parameters.The extrinsic orientation parameters in each camera point can be determined from an aerial instrumentation system (GPS/INS) in theory.However, it is difficult and time-consuming to estimate the extrinsicorientation parameters for all images.Therefore, an interpolation approach is proposed to estimate the extrinsic orientation parameters for each image.
Step 2: Image rectification.In [4] an image rotation transformation is applied to each frame in order to eliminate the rotational components.The result sequence will be a rectified image sequence as if it was captured by a "virtual" camera undergoing 3D translation (Tx, Ty, Tz).However, the sequence after that rectification still remains the height deviation.In order to simplify the latter mosaics, we introduce the rectification to compensate the aircraft flying height deviation.Therefore, after eliminating the rotation and height deviation, the camera's motion can be seem as 2D translation.This will make the latter mosaics easier and faster.
Step 3: Slice determination.This step adopts the same method in [4].And in order to get different oblique viewing directions, we suggest that to choose different dy to produce several pairs of stereo mosaics.So, it is possible to view the same area in different viewing directions.
Step 4: To stitch the left stereo mosaics and the right stereo mosaics respectively.

Conclusions
Image Stitching is an increasingly popular field of study, it could provide Realistic Panorama, is to create virtual reality scenes and virtual roaming the foundation.In this paper, the algorithm used to independently choose the baseline characteristics of the block thinking, and the image of Precise stitching, a pyramid-layered match the idea and then making the Image Stitching fast.Therefore, this algorithm is both good and robust has good practicality.