Sample original and reconstructed images.
A sample reconstructed 3D model in SGI Inventor format, viewable using ivview available freely from SGI Dino.iv
I am working on an extension of a 3D reconstruction algorithm developed by Seitz and Dyer. Their technique, Voxel Coloring, divides the 3D world scene into small voxels and uses a calibrated set of cameras to reconstruct positions and orientations of surfaces in the scene. Voxels are traversed in a specific order such that at each voxel a decision is made as to whether it contains a surface or not. Since they considered only Lambertian objects, a comparison of pixel intensity values in each image is enough to determine consistency with a surface being at a particular voxel. Lambertian surfaces reflect uniformly regardless of viewing direction. Therefore, those voxels containing surfaces should map to pixels of similar intensity in each camera image. Once each voxel is determined to contain a surface or not, surfaces are fitted over the voxels to get a complete model of the scene.
My research consists of extending this algorithm to allow the possibility of non-Lambertian surfaces (http://www.cs.wisc.edu/~tamaram). Relatively little work has been done in this area of computer vision, though it has been studied thoroughly in computer graphics for semi-physical cases. In this case we cannot simply make a direct comparison of intensity values, but must determine consistency according to the surface's bidirectional reflectance distribution function (BRDF). This introduces many more parameters into the algorithm as we must be able to estimate not only the surfaces in the scene, but their BRDF's as well and do this all concurrently. Fortunately, most methods of modeling a surface's reflectance function, model the reflectance as a linear weighted sum of it's specular and Lambertian parts. Therefore, we are able to use the original Lambertian assumption to get an approximate outline of surface positions. From this approximation we can then calculate a reflectance function and additionally a better model of the surfaces.
Advances in vision and modeling will someday allow us to visit a museum, swim under the ocean, or wander the surface of Mars from the comfort of our living rooms. I want to be part of creating the systems that will make this possible.