Show full image
Show in panoramic viewer
In this project, I implemented a Panoramic Mosaic Stitching program. The source code for the program is located in the source folder. The executables are located in the bin folder. The artifact I have chosen is located here. My Nikon D50 camera with a Nikon 50mm f/1.8 was used for this project. The camera calibration info using the toolkit. The focal length of the lens attached to the D50 is about 6898 pixels for taking images at 3008 x 2000 pixel resolution. The images I used for calibration are located here. Below is the data Matlab computed:
Calibration parameters after initialization:
Focal Length: fc = [ 6643.95154 6643.95154 ]
Principal point: cc = [ 1503.50000 999.50000 ]
Skew: alpha_c = [ 0.00000 ] => angle of pixel = 90.00000 degrees
Distortion: kc = [ 0.00000 0.00000 0.00000 0.00000 0.00000 ]
Main calibration optimization procedure - Number of images: 13
Gradient descent iterations:
1...2...3...4...5...6...7...8...9...10...11...12...13...14...15...16...17...18...19...20...21...22...23...24...25...26...27...28...29...30...done
Estimation of uncertainties...done
Calibration results after optimization (with uncertainties):
Focal Length: fc = [ 6896.48068 6899.99554 ] ? [ 152.09003 163.66536 ]
Principal point: cc = [ 1404.46566 1092.15302 ] ? [ 182.38104 220.95792 ]
Skew: alpha_c = [ 0.00000 ] ? [ 0.00000 ] => angle of pixel axes =
90.00000 ? 0.00000 degrees
Distortion: kc = [ 0.25656 -1.91732 0.00833 -0.00463 0.00000 ] ? [
0.13387 2.44959 0.01685 0.01321 0.00000 ]
Pixel error: err = [ 1.50278 1.91124 ]
The image library used is the gil library. The algorithm I used to perform the mosaic image stitch basically performs the following steps:
1. Undistort the image using the distortion coefficient and the focal length. The image are constructed using reverse image warp interpolating the nearest neighbors.
2. Project the images to their cylindrical projection using the focal length. The image are constructed using reverse image warp interpolating the nearest neighbors.
3. Save the cylindrical images as a grayscale PGM format and run the SIFT program on each images to create the list of feature keys.
4. Compute the list of feature key matches by keeping matches where the (1st min SSD / 2nd min SSD) < 0.3
5. Run the RANSAC algorithm to compute the translation motion between each pair of images. The number of iteration was calculated by using a probably of .25 just to be sure enough iterations are ran. One random pair of feature key match is chosen, then the number of inliers are counted where SDD < .1. This is repeated x number of times and the pair with the max inliers counts is chosen.
6. The whole panoramic image size is calculated and an image is creating using the RGBA format. I will use the alpha channel later to crop the image.
7. Each image is added to the large panoramic image. The first image is also added at the end so that the image can be warped later on. I blend the overlapped areas using pyramid blending (extra credit). I first extrapolate the region of overlap from the two images. The pyramid blending algorithm creates the laplacian pyramid P1 and P2 for the two extrapolated images, which have the same image size. I create a third laplacian pyramid P3 by combining the two previously created pyramids P1 and P2, where middle vertical line of the pyrammid is equal to the average of both the middle vertical lines of both P1 and P2, the left side of the middle vertical line is equal to P1, and the right side is equal P2. I rebuild the image from P3 which creates a blended image of the overlapped region and insert this image to the panoramic image.
8. Crop half of the first and half of the last image from the panoramic image.
9. I warp the panoramic image so that the first y pixel is at the same height location as the last y pixel of the image using reverse image warp interpolating the nearest neighbors.
10. I traverse through all the rows and remove any rows that contains pixel with alpha values of 0.
Program usage:
The program takes as many pictures as the user wants. Here is a list of requirements:
-Images must all be the same size
-Images must be less than 1800 in any dimension which is a requirement by the SIFT program
-Images must be specified in order in clockwise or counterclockwise
-Images must be in same folder as the executable are located since for some reason I can't get the SIFT program to process images located in a different location
Command arguments : ImgStich.exe <focal length > <distortion karpa1> <distortion karpa2> <image1 path> <image2 path><image3 path> ...
Example1: ImgStich.exe 595.0 -0.15 0 100NIKON-DSCN0008_DSCN0008.JPG 100NIKON-DSCN0009_DSCN0009.JPG 100NIKON-DSCN0010_DSCN0010.JPG 100NIKON-DSCN0011_DSCN0011.JPG 100NIKON-DSCN0012_DSCN0012.JPG 100NIKON-DSCN0013_DSCN0013.JPG 100NIKON-DSCN0014_DSCN0014.JPG 100NIKON-DSCN0015_DSCN0015.JPG 100NIKON-DSCN0016_DSCN0016.JPG 100NIKON-DSCN0017_DSCN0017.JPG 100NIKON-DSCN0018_DSCN0018.JPG 100NIKON-DSCN0019_DSCN0019.JPG 100NIKON-DSCN0020_DSCN0020.JPG 100NIKON-DSCN0021_DSCN0021.JPG 100NIKON-DSCN0022_DSCN0022.JPG 100NIKON-DSCN0023_DSCN0023.JPG 100NIKON-DSCN0024_DSCN0024.JPG 100NIKON-DSCN0025_DSCN0025.JPG
Example2: ImgStich.exe 1765.888 0 0 DSC_0093.jpg DSC_0094.jpg DSC_0095.jpg DSC_0096.jpg DSC_0097.jpg DSC_0098.jpg DSC_0099.jpg DSC_0100.jpg DSC_0101.jpg DSC_0102.jpg DSC_0103.jpg DSC_0104.jpg DSC_0105.jpg DSC_0106.jpg DSC_0107.jpg DSC_0108.jpg DSC_0109.jpg DSC_0110.jpg DSC_0111.jpg DSC_0112.jpg DSC_0113.jpg DSC_0114.jpg DSC_0115.jpg DSC_0116.jpg DSC_0117.jpg DSC_0118.jpg DSC_0119.jpg
Here is the result for the example images provided:
Show full image
Show in panoramic viewer
Here is the result for the sets of images I took (images are located here):
Show full image
Show in panoramic viewer
What I learned is that the feature match does not work well when the images to be combined contains mostly repeating patterns such as the siding of a house. Also you have to either make sure there are no close objects near you when taking the pictures or make sure the focus is set to manual; because the first time I took images, the camera kept on changing its focus. I first implemented the mosaic stich program first using the example images provided, which left me little time to take pictures so I just took a paronamic view of my alley. If I had more time, I would have taken more images and probably tried to see if I can find a way to remove the brightness fluctuation and try bundle adjustment as well.