CS766 Project 1 Report

Project done by : Sundeep Vaddadi , Alok Deshpande

GOAL

This project aims to recover high dynamic range radiance maps from a series of photographs taken with different exposures and use these radiance maps to construct a High Dynamic range image and view it by tone mapping it.

Algorithms implemented

1. Median Threshold bitmap method for Image allignment (bonus)
2. Recovering high dynamic range radiance maps(one for each channel of RGB) from series of digital images taken at different exposures using Debevec's algorithm. This is obtained by an intermediatery step from which we obtain the response curve of the camera used
3. Output the high dynamic range radiance maps as .hdr file and tonemap it to display the final HDR image.
   

Image Set used to get HDR image

We took one photo set,but this was done with due care to ensure that we would require one( We hope). The image set contains 17 images taken with various exposures. We used a high intensity lamp , which was kept behind a plant as source of illumination. we kept a few objects infront of the lamp . The lamp is placed so that, at high exposure we loose information of the plant and objects in front of it, at low exposure these objects show etail, but we loose information of sorrounding objects which are not illuminated by the lamp. The objective is to get a HDR image which shows details of all objects within and outside the direct illumination range of lamp. Another objective is to capture the lamp detail(like the structure of dial of the lamp) itself which will be lost at higher exposures.

Approach

• Image Allignment : For the HDR creation algorithm to function correctly, it is assumed that the irradiance values corresponding to all pixels remain fixed across all pictures. Also the camera should not move at all while taking the pictures. But practically, as we change the exposure settings manually and press the click button to take picture, camera moves a bit in the process giving slightly shifted pictures of the same scene. We need some kind of alignment method which is robust to exposure variations in the images. Ward抯 MTB (median threshold bitmap) algorithm is a good candidate for such requirements Implementation: The algorithm was implemented in MATLAB and tested on shifted images.It could accurately predict the offset for the 2nd image with 1st image as reference. care was taken to ensure that after shifting the image by appropriate offset, the exposed border pixels are cleared to zero. As the images we took were almost perfectly alligned we show the result of applying this algorithm on a set of images which we knew to be misalligned.The test images used were from OHTA sequence, downladed from middlebury university site. http://bj.middlebury.edu/~schar/stereo/data/tsukuba/

• Getting the camera response curve and radiance maps: We got the camera response curve by imlementing Debevec's algorithm. Then we got the log irradiance value for each pixel in each image by mapping the pixels values using the response curve. Though ideally, the log irradiance should remain constant across exposures, we use a triangular weighing function for robustness. The log irradiance and by extension the irradiance values for each pixel will be a weighted average of pixels from all input images. All this is done in MATLAB.

• Getting the formatted .hdr file frothe obtained irradiance maps: we then import the irradiance maps obtained above as textfiles into C++ workspace and write it as .hdr file using gil image library.

• Tonemapping and displaying: we then import the hdr file into matlab and tonemap and show it. The algorithm used in MATLAb to tonemap HDR images is from the paper "A visibility matching tone reproduction operator for HDR images" by Ward. As the tonemapped image is very good in detail but displaying toned down colors, we use the Adjust saturation parameter provided with the tonemapping algorithm in MATLAB to increase saturation and display colors more vivdly.

Instructions to run the code

• The folder 'radiance_map_MATLAB_code' contains the following scripts : 1. startup.m 2.gsolve.m 3. irradiance.m

• Step 1: Get the input images(which were resized (with the correct aspect ratio) so as not exceed memory usage) which are in the folder 'input_images' and copy them into the folder 'radiance_map_MATLAB_code'. Run the script startup.m. This will produce the required response curve for each channel and will be saved as mat files.

• Step 2: Run irradiance.m . This script takes in the mat files stored in above step by default and produces text files which contain the irradiance maps for each channel. The text files are named as 'E_blue.txt' 'E_green.txt' 'E_red.txt'

• Step 3: Import these text files into the folder 'HDR_cpp_code'. Run 'HDR.exe' to produce .hdr result . The folder 'HDR_cpp_code' already has the required text files (saved by us), so one would just need to run HDR.exe to get an output hdr file.

• Step 4 : Import the HDR image into MATALB. Read it using 'readhdr' and tonemap it using 'tonemap'. We used a saturation level of 5 to display our final result. Even without adjusting saturation, the output displays a high detail and achieves the objective of showing details of all objects in the field of view of camera. Another objective is to capture the lamp detail(like the structure of dial of the lamp) itself which will be lost at higher exposures.

• All the images taken were taken with care to see that all of then would be alligned.We set a specific exposure, then keep a timer of 10 sec to take image. The timer is to ensure that any vibration caused by manual handling gets damped over time and the camera returns to equilibrium. Therefore one need not use the Image allignment program. But we give the execution of the program here for completeness step1: The scripts for image allignment is in the folder 'Allignment'. The scripts are named as 1. startup_code.m 2. GetExpShift.m step2 : Copy input images from folder 'input_images' to 'Allignment'. step3: Run the script startup_code.m . The alligned images will be written with image name affixed with '_new'.

  Example of Image allignment code run on Ohta images which we knew to be misalligned

  reference image

  2nd image

  Alligned image

  INPUT PHOTO SET WE TOOK : showing 5 of the 17 images we took

  The tone-mapped output:

  

  Response curve of the camera powershot SX100 is like this:

  

　

FAVORITE ARTIFACT

It's the tone mapped outputI listed above, or you can get it from



Individual Contributions

Sundeep vaddadi : Recovering HDR maps(implementing Debevec's algorithm), Making the HTML report

Alok Deshpande: Implementing MTB algorithm for allignment of Images

Joint Contributions: Making .hdr files, Taking the image set of different exposure images, Tonemapping.

References

[1] Paul E. Debevec, Jitendra Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997

[2] Greg Ward, Fast Robust Image Registration for Compositing High Dynamic Range Photographs from Hand-Held Exposures, jgt, 2003.