In this project we developed a Matlab program to reconstruct the three-dimensional surface of an object based on a series of photographs. The camera and object remain completely stationary across all photographs, but the single source of lighting is moved to different locations. In order to determine preicsely where the light was located for each photograph, a series of calibration photographs taken of a chrome sphere are required. The reflections on the chrome sphere in these reference photographs are used to find the lighting directions. Once the position of the light source is known, the light and dark areas in the various photographs can be used to determine the subject's shape.
As mentioned above, the first step is to determine the lighting directions for each photograph. This requires finding the location of the center of the chrome sphere as well as the center of the specular highlight. Given this information, the lighting direction can be calculated through use of the Euclidean distance between the centers of the two points.
Once the lighting vectors have been determined, we must find the normals of the object at each pixel location. With the light intensity at each pixel for each image as known values, we have an overdetermined system which can be solved in the linear least squares sense with SVD. The objective function to minimize is Q = sum_i (Ii^2 - IiLi*transpose(g))^2, where I is the intensity at a pixel, L is the lighting direction for the image in question, and g is the albedo times the normal. After taking the partial derivative with respect to g, we can assemble the A and b matrices required for SVD. The discovered g values can be divided by their length to produce the normal vectors for each pixel.
With the complete set of normal vectors known, it is possible to fit a surface to the object. The difference in the normals between any two neighboring pixels will define a tangent vector to the surface. This tangent vector should be perpendicular to the pixel normal. Given this information, we can create a system of equations to solve for the unknown z (height) values at each pixel location. This overdetermined system of equations is once again solved using SVD.
We implemented the algorithm in Matlab in a total of three functions: FindLightingVectors.m, FindNormals.m, and FindHeightMap.m. FindLightingVectors.m takes in a file directory/name and number of files and determines the lighting direction used in each, as explained above. The files passed in should be those of a chrome sphere (with lighting positions identical to those used to take pictures of the subject.) FindNormals.m takes in a set of lighting vectors, a file directory/name, and a number of files present in that directory and returns the normal and albedo of the subject at each pixel. Finally, FindHeightMap.m takes in a set of normals, a file directory/name, and number of files and returns the final height map of the subject.
We ran our program on the Buddha, Gray, and Owl image sets.
Buddha:
RGB-encoded normals:
Albedo map:
Views of the reconstructed surface:
Gray:
RGB-encoded normals:
Albedo map:
Views of the reconstructed surface:
Owl:
RGB-encoded normals:
Albedo map:
Views of the reconstructed surface:
DISCUSSION OF RESULTS HERE