CS 766 - Fall 2008

Project 2: Panoramic Mosaic Stitching

Mayank Maheshwari / Blayne Field

Implementation details

Step 1: Images

We took 2 sets of 18 images for the project. We used the Canon Powershot SX100IS tag 6418203978 with resolution 480x640 focal length 682.05069 and the parameters K1=-0.22892,K2=0.27797for the pictures. If you are interested in seeing the panorama from the other set, please email us.

Step 2: Warping into cylindrical coordinates

The Matlab program cylinder_projection.m takes in the set of input images and outputs the warped image projected on cylindrical coordinates.We remove the radial distortion in the original image by applying inverse warping of the distortion model.

Step 3: Computing the alignment

We use the SIFT program sift.m to find the set of keypoints in the image. The output of the sift program is in the form of 3 matrices:

• image matrix: the image array in double format
• descriptor matrix: a K-by-128 matrix, where each row gives an invariant descriptor for one of the K keypoints. The descriptor is a vector of 128 values normalized to unit length.
• location matrix: K-by-4 matrix, in which each row has the 4 values for keypoint location (row, column, scale, orientation). The orientation is in the range [-PI, PI] radians.

We use the sift features and implement RANSAC algorithm in the Matlab program ransac.m which matches the features based on the angle between the unit vectors of descriptor matrix to be below a certain threshold. The match implemetation is borrowed from match.m. The other part of ransac.m implements the RANSAC algorithm by selecting a random feature pair and calculating the translational homography between this pair. It then calculates the squared sum distance between the other feature pair matches and if it falls below a certain threshold (in our case, it was 0.6), it is recorded as an inlier. This is iterated for K=20 random feature pairs and the homography is calculated with the least squares error for the largest set of inliers.

Step 4:Stitching and cropping the images

To do this, we use a simple feathering technique for blending the images. This is done by stitch.m which basically creates an image large enough to store the panorama, and then inserts each image in the correct location based on the translations calculated by the RANSAC algorithm. This step also relies on stitch_channel.m which does this process for one of the R, G, or B channels.

Comments



• There are a few places that could probably be improved in our project. One might be more careful setup of the camera. Things looked right, but they still may have been slightly off.
• In the RANSAC algorithm, we are running the iterations for K=20 and we observed that we don't get any significant gain if we run RANSAC for more iterations.

Work load
Taking images: Mayank and Blayne
Cylindrical project: Blayne
RANSAC: Mayank
Stitching: Blayne


Source code files
cylinder_projection.m
ransac.m
shifts.m
stitch.m
stitch_channel.m
panorama.m


Files from SIFT
sift.m
match.m
appendimages.m
showkeys.m


Running the code :
Usage:
run:
rgb = panorama(files, f, k1, k2)
where files is like: {'file1.jpg', 'file2.jpg', ..., 'file_x.jpg'}
and f, k1 and k2 are the camera parameters

Results

Source images: Result












Test Image result