CS766 Project 3 Report

Submitted by : Sundeep Vaddadi

¡¡

GOAL

Construct a height field of a diffuse object using a series of images taking under different light sources.

¡¡

Algorithms implemented

1. Calibration: Estimating ligh directions using the set of chrome ballsli>
2. Obtaining surface normals usig the above lighting directions
3. Least square surface fitting.
4. Also implemented are:
5. Implementation of color albedo by simple projection as given in projecct description.
   
6. Estimating surface normalsand lighting directions without using chrome balls using Hayakawa paper method

Approach

• Calibration: Both the center of the program and the center of the highlight are calculted by centroid method. For radius calculation, firstly the image is passed through high pass filter to get boundary pixels and then radius is the average distance between the center pixel and each boundary pixel.From the vector joining the center of the sphere and the center of the hightlight, we can get the theta and phi information about the highlight 3D point on the surface of the sphere. Since the refleting direction is Z axis, we can get the lighting direction by law of reflection.
• Screen shots showing lighting direction superimposed on corresponding calibration image
• Surface Normals: The suface normals were calculated using the above lighting direction vectors using a vrery slighly modified objective function given in the project description: Q = w*(I_i – L_i * g' )^2 . here 'w' is a triangular weighinf function. This was used to bring the pixels in the shadowed and very bright regions towards the mean value value pixels by approprate weighing.
Shown below are the RGB encoded results and a quiver plot showing surface normal orientatiion. Normals were mapped to RGB values by assigning x to R, y to G and z to B. However, the image is not rendered immediately due to the fact that pixel values range in [0,1] and any component of a unit vector may be in [-1,1]. I therefore scaled x and y to ensure that all possible valueshad some equivalent rendering. z is not scaled, as at no pixel of the object does the normal vector point away from the camera on the z axis (its value is always within [0,1]).(For the RGB encoded results, we looked at Jakes project report )

Results of surface normals for Buddha

Quiver plot for channel 1

RGB encoded result channel1

• Albedo map
Albedo map was obtained from solving the objective function Q = w*(I_i – L_i * g' )^2 and also was implemented using the alternative methosd suggested in the description.In both cases, the map was same.
• Albedo map for buddha

  Channel1

  Channel2

  Channel3
  
• Least square surface fitting:The matrix formulation has been done as per the details on the projects webpage. Since the matrix is pretty huge, it is stored in sparse format and we have used MATLAB's in built 'pcg' (preconditioned conjugate gradient method) to solve the linear sustem of equations to get the surface which has this calculated set of normals in the least square error sense. The results are pretty satisfactory.

  Reconstructed surfaces without albedos

  Image1

  Image2

Image3


Estimating surface normals and lighting vectors using hayakawa method

• We have also implemented the algorithm from the paper"Photometric stereo under a light source with arbitary direction"
The exact algorithm was followed. We encountered a problem here in fullfilling the constraint"Finding 6 pixels whose relative value of reflectance is constant or known". We choose to go with constant. We found the algorithm was very sensitve to the pixels we pick. The RGB encoded map doesnt look great and we are sure of doing a beter job, if we could have picked the right 6 pixels or more.
• Overall, the purpose of finding lighting vectors without chrome balls was fullfilled, though not greatly with the extension.
We are showing the RGB encoded map obtained using hayakawa method below:

RGB encoded result channel1

• results

CAT images output

Quiver plot for channel 1

RGB encoded result channel1

Reconstructed surfaces without albedos

Image1

Image2

Image3

• Albedo map for cat

  Channel1

  Channel2

  Channel3
  
• results

ROCK images output

Quiver plot for channel 1

RGB encoded result channel1

Reconstructed surfaces without albedos

Image1

Image2

Image3

• Albedo map for ROCK

  Channel1

  Channel2

  Channel3
  

  General description and Instructions to run the code

  • 1.Open MATLAB. Change working directory to 'chrome'. Run the script 'compute_lighting_dir.m'. There will be 12 figures opening up each showing image of chrome sphere (with highlight) and direction of surface normal and lighting vector superimposed onto it. X and Y components of both the vectors are shown, red for surface normal and green for lighting vector. Note that as the highlight approaches the center of the sphere, the length both the vectors diminishes. This is because their Z component (which is the camera view direction) become stronger and stronger. The lighting vector (size 3 X 12) is stored in 'lighting_vect.mat'.
  • 2. Copy the 'lighting_vect.mat' from 'chrome' folder to 'buddha' folder. Change working directory of MATLAB to 'buddha'. Run the script 'normal_albedo_map.m' It will generate the two outputs: normal_map.mat and albedo_map.mat normal_map.mat - size (# of pixels within the mask) X 5 X 3 Each row indicates the row no, column no, and the x, y and z components of the surface normal at a particular pixel. The third dimension is basically for each color channel. It was only used to check whether normal map is significantly coming different for every color channel. albedo_map.mat - size (# of pixels within the mask) X 3 Each column indicates the albedo values for all the pixels within the mask of buddha image for a particular color channel.
  • a. For viewing RGB encoded normal map and albedo_map, run the script 'showRGBencodednormals.m' It will display RGB encoded normal maps calculated for each plane of 'normal_map.mat'(each plane of 'normal_map.mat' corr. to normal map calculated from each color channel of all original images.)
  • b. We have also implemented the alternative method to get albedo map per color channel once we have the normal map. Make sure you have 'normal_map.mat' in your working directory. Run the script 'albedo_map_alternative.m' It will store the albedo_map per color channel in 'albedo_alt_map.mat' (format same as 'albedo_map.mat')For viewing RGB encoded normal map and this alternative albedo_map, run the script 'showRGBencodednormals_alt.m'. Note: you will also see the normal map in quiver format for each color channel.
  • 3. Least Squares Surface Fitting: run the script 'least_squares_surface_fit.m' It will display the surface plot of the 3D buddha model with default viewing direction. You can view the model from any desired direction using 'view' command.The 3D model is stored as 'model_3D_buddha.mat'. (the program takes a long time, around 6-7 mins to build up the huge sparse matrix and solve the linear system of equations using preconditioned conjugate gradient method.)
• Running hayakawa method: Go into project 3 submission\buddha\hayakawa : Run 'hayakawa.m'. then, 'albedo_map_alternative.m' then 'showRGBencodednormals_alt.m'to see the RGB encoded map. If you want to see the 3D reconstruction, run 'least_squares_surface_fit.m'.
• The above same process is to be followed for the other folders 'cat', 'rock'.
• If you want to see quiver plots for all channels, they are avilable in the respective folders( buddha,rock..) with explicit names like buddha_quiver1 for channel 1, buddha_quiver2 for channel2 .

References

[1] Hayakawa, Journal of the Optical Society of America, 1994, Photometric stereo under a light source with arbitrary motion.

Individual Contributions

Contributions would be :
• Sundeep: 50% ( Surface normals using objective fn. from project description), Hayakawa method for surface normals and lighting vectors
• Alok: 50% Lighting vectors from chrome balls,Color Albedo , Least square surface fiting