High Dynamic Range Imaging

Chaman Singh Verma and Mon-Ju

Introduction:

Human eyes have extremely high visual range that can easily differentiate between very bright and dark regions in the same scene. High Dynamic Range Imaging is a technique to emulate this behavior with the low end commercial cameras available in the market by taking images of the same static scene at various exposures and then merging the images.

In a simple static setting, the process of HDR starts from image acquisition at various exposures of the same scene at quick succession so that effect of changing light conditions can be ignored. Despite using tripods for greater control, some perturbation is inevitable, therefore, image alignment is often necessary.

In this class project, we have implement Debevec algorithms to calculate the radiance map.

Tone mapping goes in reverse direction, the HDR values calculated from the step three, cannot be used on devices which has limited dynamic range. The idea of tone mapping is to reduce the dynamic range in a way that preserves the features in the HDR images and present colors which are pleasing and appreciable to human eyes.

Implementation Details:

The radiance equation from the Devebec paper is given as:

debevec

Two important issue in solving this equation are:

Sampling: As described in the paper, we need not select all the pixel available in the image, but only to satisfy N(P-1) > (Zmax-Zmin). We have chosen about 100-200 pixels randomly from middle exposed image in the range of (5-250). Presently, for each color channel sampling is done independently.
Lambda : The second term in the equation is the penalty term and proper selection of lambda ensures that the resulting response curve is smooth.

Results and Conclusions:

Test and Benchmarking: There are many commercial and public domain software available that can do high dynamic range imaging. One of the publicly available such tool is PFSTools with which we compared our results using Reinhard tone mapping. The final color depends on many parameters and it is hard to judge why our implementation produces better results ( it has more yellow and red colors on the top which is closer to real Madison Capitol building).


HDR with pfstools	Our Implementation

Dataset result

Response Curves:

For each red, green and blue channels, radiance map is calculated from the Debevec equations. Ideally, the radiance map must be smooth and monotonically increase, but because of errors introduced in solving the equations and possibly imperfect imaging devices, we get little perturbations in the calculated maps. Polynomial fitting could be used to get the smooth curves, but this is not done in this study. It is also likely that near the saturation ends ( 0 and 255 values ), the curve show some erratic behavior, but this can be handled by truncating or clampiing the graph.

(Z, lnE) radiance map of Red, Green and Blue colors.

Tone Mapping and Perceptual Image Quality:

HDR Image with Reinhard02 tone Mapping

HDR with Fast Bilateral Tone Mapping

As we can see, both Reinhard algorithm preserves the color better than Durand's Fast Bilateral tone mapping as can be seen near the upper arch, where Reinhard has more yellow color preserved. But Durand's algorithms preserves the sharp edges far more better than Reinhard algorithm.

Performance Results

The results were taken on HP-9308NR notebook with 1GM RAM. For a dataset of 30 images, the table shows that the maximum time is taken in loading the images and extracting the colors. The HDR process is independent of image size and take fraction of overall time.

Action	Execution Time(sec) 1024x768	Execution time (Sec) 2048x1536
loading images	25	36
Extracting {R,G,B} Color	150	180
Selecting random points	03	18
Generating Matrix	03	03
Solving least square problem (Lapack)	36	36
Generating HDR image.	02	02
Total	219	275

Appendix

Datasets:

These dataset are from beautiful Madison Capitol building.

Data Set	Image
MadisonCapitol-1
MadisonCapitol-2
MadisonCapitol-3
UWMechDept

Software and User Guide:

The entire software UWHDR is written in standard C++ on Linux running ubuntu 9.04. There are some public domain software that we have used in developing this code and they are:

CLAPACK : solving least square matrix.
ImageMagick++ : For image loading, exposure time attribute extraction.
RGBE : Stores 3*nx*ny floating point data in Radiance *.hdr format.
pfstools 1.8.1 : Create tone mapping on a given *.hdr file. It support many tone mapping algorithms, but we use only Reinhard02 and Durand's Fast Bilateral Tone mapping algorithms. pfstools depends on openEXR software, which has its own dependencies on other public domain software. Installing everything and making it work wasn't straightforward.

Software Usage:

Assuming that all external software are installed on the machine then use makefile to compile the code. Create one directory that contains all the images ( tested with JPEG ). Make sure that only the relevant image files are in the directory. Then use

uwhdr <directory_name> <lambda>

where directory_name is the directory where images are stored. The value of lambda depends on the image set which is used to smooth out the radiance map. Start with small values (~0.1 ) and increase the value till a smooth monotonically radiance map is obtained. It is however likely that the near end points the radiance map may be erratic and therefore, care must be taken to clamp the radiance map rather than changing the "lambda" values. For most of the cases, the lambda value (1.0-5.0) was good enough.

This will generate "scene.hdr" file. which becomes input for tone mapping.

pfsin scene.hdr | pfstmo_reinhard02 -s | pfsout scene.png

Besides generating "scene.hdr" uwhdr also generates red, green and blue radiance maps. They are stored in lnE0.dat, lnE1,dat and lnE2.dat.

To visualize the scene.png image:

display scene.png

Mon-Ju has written similar code in Matlab HDR.m.

Reference

Recoveing High Dynamic Range Radiance Map from Photographs: Paul E. Debevec and Jitendra Malik.