Project 1 - High Dynamic Range Imaging

24 Sept 2007

Project Description

The intent of this project was to implement an algorithm to produce a High Dynamic Range image from a series of source images. These source images were taken at various known exposures Δt, and compiled under an algorithm to determine the irradiance E as given by f(X_i,j) = E_iΔt. This irradiance value for each pixel yields a high dynamic range image, or .hdr file, which uses 32 bits to represent the value at each pixel instead of the standard 8. This image is then mapped to a standard representation (.JPG, .BMP, etc.) using a tonemapping algorithm, resulting in an image that shows both areas of high light and of low light at comprehensible levels.

What I learned

I'm better with numbers than images. This project took me an inordinately long time, partially due to a lack of usable libraries for the Macintosh - I spent a good half a day trying to install libraries manually on my Mac, only to find that gil ultimately wouldn't work without far more tinkering than I was willing to put in. I had to then figure out how to use Visual Studio (which I suppose counts as something I learned) and link everything appropriately. I also learned some new numerical hacking techniques.

More relevantly to the project, I learned how to translate an algorithm from a paper into code, and how to work with nonstandard libraries; I learned how to do (some) configuration on manual installs for said libraries. I also learned some very interesting theory on high dynamic range imaging, which I (despite the headaches of actual implementation) have very much enjoyed. I also learned effective use of pointers and how to (write and) speed up C++ code.

On top of all this, I learned a good deal about the methods and concepts behind High Dynamic Range imaging. My only prior exposure to the concept was in the form of Half Life 2 Episode 2, and it was extremely edifying and fun to have exposure to the technical side of that development. I also learned about the structural representations of images in computers, something to which I had previously had no exposure.

Algorithm Implemented

I implemented Debevec's HDR method (SIGGRAPH 97). This method recovers film response using the above equation (f(X_i,j) = E_iΔt), and eventually minimizing the function:

Σ&Sigma{w(Z_i,j)[g(Z_i,j)-ln(E_i)-ln(Δt_j)]}^2 + λΣ[w(z)g''(z)]^2

From this minimization we obtain values of g(Z_i,j), and recover ln(E_i) = g(Z_i,j)-ln(Δt_j). This gives the irradiance value for pixel i.

Implementation Details

In my implementation, I used 19 images (hard-written into the code) of differing f-stops and about 20 randomly chosen sample pixels for the minimization. The images were read and manipulated using the gil library, and the mathematical functions I used were from the GSL library. This includes matrix and vector storage, as well as the singular value decomposition and solution used to create a linear fitting to minimize the above equation.

My algorithm interprets the data strangely, however; I found that the ln(E_i) values were not in fact negative - perhaps a function of the very low f-stops I used - and therefore the exp(ln(E_i))=E_i values were not between 0 and 1, as they should be in a FloatImage in the gil library. So I instead merely normalized the values by finding the maximum value in each channel and dividing all other values in that channel by it. This yielded a range of values in (0,1] and resulted in a passable image.

My implementation deals with each color channel (RGB) independently, and normalizes only by forcing g(128)=0. I chose not to implement RGB->Luv->Iuv->RGB filtering, with the fitting being performed on I, due to time constraints. The unknown scaling factors relating relative radiance to absolute radiance for each channel are not calculated in this implementation, the results of which are discussed below.

Results

My resulting images ranged from completely blank at one point to a psychadelic sky (below) to the final image which I have submitted. This image is, to my eyes, a successful high dynamic range image in all but one point - the extreme dark values have skewed in color, near the base of the trash can in the lower left corner. These artifacts are a result of the scaling factors mentioned in the previous section, as evidenced by a slight spill of magenta around the striping.




It's not HDR, but I think it looks cool.

The final image may have been corrected completely with the implementation of Luv filtering, but as mentioned previously, time constraints prevented this from occurring.

Extensions Implemented

None - project basics took too long.