Panoramic Mosaic Stitching

Project Report

Yupu Zhang, Guoliang Jin and Tuo Wang

1. Goal

        To combine a set of images into a larger image by registering, warping, and blending them together.

2. Methodology

2.1 Taking photos

Using Canon SX100 IS(tag 6418204024, with Kaidan head helping to place the camera optical center on the rotation axis) to take 18 photos at Chapel Hill apartment. The parameters for the class camera are:

                     Camera                            resolution      focal length                k1                              k2

Canon SX100 IS(tag 6418204024)   480x640         676.21154                -0.22032               0.20938

2.2 Removing image distortion

2.3 Warping each image into cylindrical coordinates

Computing the inverse map to warp the image into the cylindrical coordinates. For these x and y which are not integer, using 2-D linear interpolation method to sample the pixel value.

2.4 Computing the alignment of the images in pairs

        Firstly we used the SIFT program to extract the feature points from images. Then we modified the match program from the SIFT package to reject outliers, and finally we applied the RANSAC algorithm to find the homography matrix.

2.5 Stitching the resulting aligned images

As for the blending, we use a simple feathering function as the weighting function. The weight of every pixel is determined by its distance to the edge of the overlap rectangle. Assuming  yHmin denotes the minimum column of target image (which should be mapped to the reference image coordinate), yrmax denotes the maximum column of reference image, and y denotes the yth column that is being processed. Then the weighting function works as the following:

OutImg(x,y,:)=((y-yHmin)/(yrmax-yHmin))*RefImg(x,y,:)+((yrmax-y)/(yrmax-yHmin))*TargetImg(x,y,:);

The VS2005 blending needs a special file “mask.png”. It’s the result of warping a white image to the cylindrical coordinates. We have included this in the hand in folder.

2.6 Cropping the result image to generate a final output.

3. How to run the program

Before you run the program, please create four folder named ‘images#’, ‘cylimg#’, ‘disimg#’ and ‘result#’ respectively. ‘#’ is the number of your input images if you have several groups of images to be processed.  Put the input images in the ‘images#’ folder, then the program will automatically store the cylindrical warped image, undistorted image and final result before cropping in the ‘cylimg#’, ‘disimg#’ and ‘result#’ folders. You can also modify the variables f, k1, and k2 in the main.m file.

We handed in two sets of code, one with Matlab blending and one with VS2005 blending. As the size of the input image is hard coded in the VS2005 blending, it can only work with the 480x640 pictures.

The VS2005 blending uses the gil library, the handed in directory does not include this.

4. Interesting points

4.1  In our program, x axis is the same as image height direction and y axis is the same as image width direction.

4.2 Usually the input image should be size of 640*480 (height*width). Interestingly, our program gets a much better result with input image size of 480*640 (height*width).  We still cannot figure out the reason.

5. Results

        5.1 Chapel Hill Apartment

Input image

Matlab Panoramic Image

VS2005 Panoramic Image

Matlab Result in Interactive Viewer

VS2005 Result in Interactive Viewer

5.2 Chapel Hill Outdoor Parking Lot

Input Image

Matlab Panoramic Image

VS2005 Panoramic Image

Matlab Result in Interactive Viewer

VS2005 Result in Interactive Viewer

6. Favorite Result

Matlab Result with Inpute Image Chapel Hill Apartment

7. Who Did What

        Matlab Code: Image Warping, image undistortion, and blending are completed by Tuo Wang and Yupu Zhang. Yupu Zhang is also responsible for the SIFT algorithm and RANSAC algorithm.

        Cpp Code: Completed by Guoliang Jin.