CS766 Project 2 Report

Project done by : Sundeep Vaddadi , Alok Deshpande

GOAL

To make a panoramic image from a set of images.This involves aligning the images, blending and stitching them together.

Algorithms implemented

1. Warp the images to remove distortion.
2. Wrap the images onto a cylinder.
3. Align the consecutive images using feature based method followed by RANSAC algorithm.
4. Blend the consecutive images into each other to form panorama.
5. Apply linear warp on the whole panorama so as to match last image with first image, crop the result to get final panorama.
   

Different image sets used to get panorama image

We took 3 sets of images, first in graphics and CV lab, second inside camp randall stadium and last one in one of the CS labs. We chose these locations so that there are no moving objects involved in the scene. We adjusted the Kaidan head (mounted on tripod) so that it is prefectly horizontal and we took around 18 images to cover entire 360 deg span. The last set of images was for creative panorama where we took the pictures with same person appearing in different images.

Approach

• Image Undistortion : After taking the images, undistortion is done on them using the camera distortion parameters k1 and k2 and focal length. We undistort by reverse warping. Assumption is done that we have a distortion free image(buffer), and we model this particular image cordinates to have distortion. Now, we take the pixel values from these locations and fill up the buffer.
• Converting to cylindrical coordinates: Each image was wrapped onto a cylinder with radius equal to focal length, getting the consecutive images into one plane near the boundaries. This was done with inverse mapping.
• Feature based method for image alignment: we find SIFT features for both the images to be aligned using the SIFT program provided. The SIFT program returns a descriptor and location for each point of interest. A matching agorithm is written to find pairs of correspondence between the SIFT features using exhaustive search and remove outliers to an extent by checking how different the best matched feature is from the second best. Any remaining outliers after this are removed using RANSAC algorithm. The parameters important in the RANSAC algorithm performance were 'no. of sample points randomly selected for calculating the exact model','no. of iterations','threshold to determine whether other points agree with the calculated model','size of consensus set to check whether a determned model is good or not'. Output of RANSAC algorithm was the translational vector between two images.
• Forming the Panorama: Using the translational vectors (returned from RANSAC), images are aligned to form the panorama.For Blending, a weighting function is used where the weights are calculated based on the squared distance from each pixel to its corresponding image center. So, given img1 and img2, the resulting pixel would be ( 1/r1^2 * p1 + 1/r2^2 * p2 ) / (1/r1^2 + 1/r1^2) where r1 and r2 are the distances from the center of img1 and img2 to p1 and p2 respectively, and p1 and p2 are pixel values from img1 and img2 respectively. (The weights only vary across columns. This was done because the images mostly have horizontal translation).
• Solving the drifting problem: To remove any vertical drift between the first and last image a linear warp is done on the mosaic. The warp is of the form y' = y + ax, Cropping: Result is cropped to get the final panorama.

Instructions to run the code

• The folder 'mosaic_testimages_final' contains the following scripts : 1. final_startup.m 2.cylin.m 3. ComputeHomographySimple.m (there are other .m and binar files which are used for finding SIFT features.)

• Step 1: Get the input images which are in the folder 'input_images' and copy them into the folder 'mosaic_testimages_final'. Run the script final_startup.m. The program will generate undistorted gray scale images wrapped onto the cylinder and will write them in the current folder. These images will be used for feature detection and matching purposes. Finally, the program will write the panoramic image named 'pano_lab_new.jpg'.

Result of matching algo.(followed by RANSAC) showing translational vectors

Image1

Image2

O/P showing translational vectors

INPUT PHOTO SET WE TOOK : showing 5 of the 18 images we took

Panorama output(CS lab):

Panorama output(Camp Randall Stadium):

You can see ghosting of an image in the right half. This image was not having good features because of which matching was bad and we could not get correct translation vector.

Panorama output(CV and graphics lab):

Panorama output(Autostitch result on test images):

Panorama output(our result on test images):

　

FAVORITE ARTIFACT APPLET

References

[1] Richard Szeliski and Heung-Yeung Shum, Creating Full View Panoramic Image Mosaics and Environment Maps

[2] Wikipedia - for RANSAC

Individual Contributions

Contributions would be : Alok : 60% (matching algorithm,RANSAC,blending,panorama) Sundeep : 40%(undistortion,cylindrical) Taking images: Joint contribution