CS766 Computer Vision, Fall 2007

Project 1: High Dynamic Range Imaging

Assigned: Sept 11, 2007

Due: Sept 26 noon, 2007

The instructor is extremely thankful to Prof Yung-Yu Chuang for allowing us to use this project which was developed in his Digital Visual Effects class.

Project Description

High dynamic range (HDR) images have much larger dynamic range than traditional images' 256 brightness levels. In addition, they correspond linearly to physical irradiance values of the scene. Hence, they have many applications in graphics and vision. In this project, you are expected to finish the following tasks to assemble an HDR image.

1. Taking images. Taking a series of photographs for a scene under different exposures. As discussed in the class, changing shutter speed is probably the best way to change exposure for this application. For that, you need a digital camera that allows you to set exposures. (Note that not every camera allows a user to manually set exposures.) You can use your own camera or borrow a Canon PowerShot A640 from us. If you need to borrow an A640, please check out the schedule and send TA an email. We suggest that you use a tripod when you take pictures. Again, you can use your own or borrow one from the TA. We would also provide flash memory. However, you should bring your own batteries (4x AA Batteries, Alkaline Ni-MH) to operate the camera. 

One thing to note is that you should avoid moving your camera during this process so that all pictures are well registered. For some Canon camera models, there are programs, for example, AHDRIA, which allows you to set exposures and release shutter remotely from a PC or a laptop. Cameras of other brands might have a similar solution for remote capturing, but you have to discover on your own. You are welcome to write down your findings for that matter in your report.

If you decide to manually change exposures, it would help if you align your images before proceeding to the next step. Photomatix has a free version for HDR creation and image alignment, Photomatix Pro 2.5.2. RASCAL also has a utility called imageAlignment for aligning images. You can download from their website. You are also welcome to write a program for alignment as bonus points. A good candidate is the median threshold bitmap we discussed in the class.

2. Write a program to assemble an HDR image. Write a program to take these captured images as inputs and output an HDR image as well as the response curve of the camera. We provide a C++ image class called gil which supports I/O for many image formats for both unsigned-char images and float-point images. Click here to download a sample code (including the gil library), showing you how to use the gil. To compile and build the code in MS Visual Studio, you may need to change the "Include" and "Library" directory settings for the gil library. We recommend that you output your radiance map as a Radiance RGBE image (.hdr), which is implemented in gil already. If you prefer to use linux version, here is a description about how to set it up

You will use Debevec's method. Please refer to Debevec's SIGGRAPH 1997 paper. The details of this algorithm has been discussed in class. The most difficult part probably is to solve the over-determined linear system. You can use these packages, GSL, Boost, LAPACK, LAPACK++, ATLAS, or write your own solver. (I recommended you to use LAPACK.) You should not use Matlab for this project since Matlab has already implemented the Debevec's HDR algorithm. You can refer to Chapter 2 of Numerical Recipes if you want to write your own linear solver.

If you want to test your program before taking pictures of your own, you can download test images from here, from Debevec, or from Photomatix.

3. Develop your radiance map using tone mapping. Load your radiance map into HDRShop and use Reinhard's tone mapping algorithm to develop your radiance map into a usual image. Reinhard's algorithm has been implemented as a free HDRShop plugin. Optionally, you are welcome to write other tone mapping algorithms, preferably in HDRShop plugin format. You can refer to this page to learn how to write a plugin for HDRShop. Implementing tone mapping will be counted as a bonus since it is not required.

Here are several other options for tone mapping: Photomatix (algorithm unknown) and logview. The following two tone mapping programs are for Linux only: Fast Bilateral Filtering and PFStmo (seven algorithms implemented).

Bonus Points

Students will get extra points if they implement any of the following extensions. Please note that the basic requirement for this project is pretty simple. It is strongly recommended that you implement at least one of these extensions.

• Image alignment methods for HDR imaging. A good suggestion is Ward's MTB algorithm.
• Other HDR creation methods such as Mitsunaga and Nayar's algorithm. Please refer to their CVPR 1999 paper, Radiometric Self Calibration. The main difference between this method compared with Debevec's method is that this method assumes that the response curves are high-order polynomials. RASCAL uses this method. Another popular method is Robertson's method. It is more robust.
• Tone mapping. Any tone mapping algorithms other than Reinhard's are counted as bonuses. More tone mapping algorithms are listed in the reference at the bottom of this page.

You are welcome to do any other extensions or develop algorithm related to high dynamic range images. The bonus depends on how useful and difficult these extensions are.

Submission

• You have to turn in your complete source, the executable, the original pictures you have taken, at least one recovered HDR image and its tone-mapped image, instructions to run your program, and a report in html format and hard copy put in TA's mailbox before the due time. The report could contain a description of this project, what you have learned from this project, description of the algorithm you implemented, implementation details, results (either good or bad), and what extensions you have implemented. Note: you do not need to submit the library you used. However, you need to indicate which library you used and where you got it
• You also have to submit your favorite artifact generated by the program you have implemented (not the reference software). For the artifacts for this project, submit the tone-mapped image. All class participates will vote for the top three artifacts. The creators for these artifacts will get extra bonus point.
• Please submit your result to the handin directory by the following:
  
  • First add the following line to your .cshrc.local file (located in your home directory):
    
    set path = ($path /s/handin/bin)
  
  
  • Then run the following command at the shell prompt:
    
    source ~/.cshrc.local
  
  
  • Finally,to handin your files, enter the following command
    
    handin -c cs766-1 -a PA1 -d directory
    
    where directory is the path to the directory where all your files are located.handin will go through the specified directory and hand in the tar ball. Please note that the handin script will copy everything in the directory and you should only put all files related to this project in that directory. Note : don't put the debug inforamtion in your source code.
    

Reference software

• Photomatix: a high-quality commercial program to create HDR images and tone mapping. It has a free version with basic operations.
• HDRShop: a program to create, view and manipulate HDR images.
• AHDRIA & AHDRIC: a program to automatically create HDR images including camera support.
• RASCAL: a HDR creation program with image alignment capability.

References

1. HDD
   • Paul E. Debevec, Jitendra Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.
   • Tomoo Mitsunaga, Shree Nayar, Radiometric Self Calibration, CVPR 1999.
2. Alignment
   • Greg Ward, Fast Robust Image Registration for Compositing High Dynamic Range Photographs from Hand-Held Exposures, jgt, 2003.
3. Tone Mapping
   • Jack Tumblin, Greg Turk, LCIS: A Boundary Hierarchy for Detail-Preserving Contrast Reduction, SIGGRAPH 1999.
   • Sumanta Pattanaik, Jack Tumblin, Hector Yee, Donald Greenberg, Time-Dependent Visual Adaptation for Fast Realistic Image Display, SIGGRAPH 2000.
   • Raanan Fattal, Dani Lischinski, Michael Werman, Gradient Domain High Dynamic Range Compression, SIGGRAPH 2002.
   • Fredo Durand, Julie Dorsey, Fast Bilateral Filtering for the Display of High Dynamic Range Images, SIGGRAPH 2002.
   • Erik Reinhard, Michael Stark, Peter Shirley, Jim Ferwerda, Photographics Tone Reproduction for Digital Images, SIGGRAPH 2002.
   • Michael Ashikhmin, A Tone Mapping Algorithm for High Contrast Images, EGWR 2002.
   • F. Drago, K. Myszkowski, T. Annen, N. Chiba, Adaptive Logarithmic Mapping for Displaying High Contrast Scenes, Eurographics 2003.
   • Erik Reinhard, Kate Devlin, Dynamic Range Reduction Inspired by Photoreceptor Physiology, IEEE TVCG 2005.
   • Hwann-Tzong Chen, Tyng-Luh Liu, Tien-Lung Chang, Tone Reproduction: A Perspective from Luminance-Driven Perceptual Grouping, CVPR 2005.
   • Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel, Lightness Perception in Tone Reproduction for High Dynamic Range Images, Eurographics 2005.