Photometric Stereo

by Steven Kappes

Description:

I implemented photometric stereo in Matlab. The algoritm included these steps: find the position of the lights, find the normals of the object, calculate the per channel albedo values, and calculate the depth values of the object.

Finding the light positions was accomplished with the use of chrome sphere images. Since the spheres are specular objects, the location of the light can be found by looking at the highlight and using the properties of specular reflection. Using the equation that models specular reflection, L = 2(Normal dot Eye) - Eye, the light position can be found. This equation is only true where the light is directly reflected into the eye, which is at the specular highlight. The normal at this point is can be calculated with the sphere radius like so:
N(1) = radius * (highlightPos(1) - center(1))
N(2) = radius * (highlightPos(2) - center(2))
N(3) = normalize ( sqrt(radius^2 - (N(1)^2 + N(2)^2)) )
The eye position is located directly above the image naturally: (0, 0, 1). Therefore, L can be calculated based on the specular hightlight since all of the necessary variables are known.

Calculating the normals of the object is similar to calculating the light positions, except diffuse refection is assumed instead of specular. Diffuse reflection depends on the light position we calculated earlier. The normal can then be solved at each pixel with the following equation:
Intensity = kd (Normal dot LightPosition)
This can be simplified to Intensity = LightPosition dot G, where G = kd * Normal. Since there are 3 unknowns, at least 3 images are needed to solve for the normal. More images will make the system over-determined and yield a better result. I used the grayscale value for the intensities of the pixel from each image to solve this. I also added a weighting factor to both sides of the equation: w = 2 * i if i <= .5 or 2 * (1 - i) if i > .5. This weights the pixel intensities which are not at the extremes, which can help with issues like self shadowing in which the pixel intensity would have the incorrect value if you consider diffuse reflection alone.

Finding the per channel albedo values depends on the normals. This can be solved the same equation above (diffuse reflection) for each color component independently instead of using the grayscale value for the intensity. This can be simplified to kd = (I dot J) / length(J) where J = L dot Normal.

Finally, a relative value for the depth of each pixel can be calculated based on the normals. Using the fact that the normals are always perpendicular to the surface, we can add a constrain to the depth value of each pixel:
Nz * Z(x+1, j) - Nz * (Z(x, j) = -Nx
This is based on the fact that the vector to the adjacent pixel location will be on the surface of the object (or close enough since the pixels are very close). Therefore, the dot product between this vector and the normal must 0. This constraint was setup to the pixels right neighbor. More constraints can be added to all adjacent pixels. Only two constraints are needed though to overly-determine the system.

Overall, the project taught me more about Matlab than anything else, particularly because I have never used it before. As a result, my program is probably not the most efficient that you can write in Matlab. I also learned about deriving information about an image based on its lighting. I found the method of fitting a surface to an object based on only its normals (using a space matrix since their are so many constraints) to be particularly interesting.

Program Usage:

My program can be run from Matlab as follows: PhotometricStereo(chrome images file, input images file, use true lights). The images file are the images files given in the test images. Use true lights can be set to 1 if the given light values of the test data should be used. This can be used to compare how well the light positions are calculated. Matlab cannot read targa images which the test data includes. I converted the buddha images to bitmaps and included them. My program can be run on those images with the script I included by just typing 'run' in Matlab. My program will show 4 images after it runs: the normal vector map, the normal vectors encoded into RGB, the albedo map, and the surface map. The normals encoded to RGB and albedo map are also saved to the Matlab directory.

Results:

My results can be seen here. These include all of the outputs of my program on the six test sets. In general, the results appear to be accurate. The one issue I did notice was with the gray sphere set. The normals do not calculate exactly right as seen by where the center of the image is in the needle map. As a result, the surface is slightly elongated in one direction. I'm not exactly sure of the cause but when using the true lights, the result is slightly better. The other noticeable singularity is with the owl's eye. This has a higher depth value than the surrounding area since this part of the owl is not really diffuse. It is dark in every image. Other than that the surfaces look smooth and correct.