Introduction

The purpose of this assignment was to composite two partially overlapping images together to form one complete image. This mosaic was created by registering, projective warping, resampling and then compositing the images.

Image Registration

Four points in each of our two images are required to properly register our images. This registration allows us to find the perspective projection necessary to warp one image so that it is aligned with the other image. The points used in the registration should be correlated. Registration was done by hand.

Computing the Projection

Given two input images, ImageA and ImageB, and four registration points in these images, x₀, . . . , x₃ and u₀, . . ., u₃, it is possible to find a 3-by-3 perspective matrix P such that
u = xP
(see Heckbert, 1989). This projection was found using the poly_map code supplied in class.

Image Warping

Projection
With the projection matrix P, it is now possible to map ImageA into the space occuppied by ImageB. This will be done using a backward mapping. Before we begin this mapping it is useful to find the bounding box of our resultant image. This was done by finding the inverse of P, denoted as Q. We map the original four corners of our image to the space occuppied by ImageB by using Q. This then tells us the size of our resultant image.

Resampling
Now that the size of the resultant warped image is known, it is possible to color the pixels of the warped image using backward mapping:

for every pixel p_w in the warped image:

find the corresponding pixel p_o in the original image by using: p_o = p_wP.

if p_o is not contained in the original image, set p_w = black

else p_w = Interpolate(p_o)
Interpolate is defined using bilinear interpolation. Let I(i) be the image intensity at pixel i, i = [i_x, i_y]. The backward mapping gives us an image location in the original as p_o. Let f = [floor(p_ox), floor(p_oy]. Then the resultant shade is found:

shade =

(1 - a) * (1 - b) * I([fx, fy]) + (1 - a) * b * I([fx, fy] + 1) + a * (1 - b) * I([fx + 1, fy]) + a * b * I([fx + 1, fy] + 1)

where: a = p_ox - f_x, and b = p_oy - f_y

Compositing

Create an space ImageC and color every pixel p_w in ImageC such that:

if p_w is in both warpedA and ImageB: I_C(p_w) = w_AI_A(p_w) + w_BI_B(p_w)

else if p_w is in warpedA: I_C(p_w) = I_A(p_w)

else if p_w is in imageB: I_C(p_w) = I_B(p_w)

else I_C(p_w) = black

Figure 1: The distances used for defining the weighting scheme of overlapping regions

Results

I used two images from a sculpture garden in the Netherlands, originally taken by Victor Zandy. I show the images in figure 2. The red cross-hairs are the points used to find the projection maps.

Figure 2: The original images. Registered points are shown in red

Resultant composite images are shown in figure 3.

Figure 3: The composited images

Discussion of Results

Note that at there is a noticeable seam at the overlapping regions. This is because of the lighting changes that occur when rotating a camera. The purpose of the weighting given above was to provide a smooth transition between the overlapping and non-overlapping regions. Unfortunately, this weighting scheme does not allow for a continuous blend at the overlapping region borders. Instead, a better weighting scheme is given below, and demonstrated in figure 4:
w_A = d_B2 / (d_A2 + d_B2), and w_B = d_A2 / (d_A2 + d_B2)

Unfortunately, I haven't had a chance to test this technique to see if things improve.

Figure 4: The distances used for the improved definition of the weighting scheme of overlapping regions

Hooray! Brand new, working results

Now that I finally learned exactly what the hat function was, I went and corrected my mistake from earlier. So here are the new results (see figure 5).

Figure 5: The well-blended mosaiced images. Click on the figure to see the full version