Sunday, November 13, 2011

High Dynamic Range Imaging - A Technical Overview

             Due to the digital revolution, analog cameras are rarely used these days. Digital cameras provide us sophistication in the sense that one can review the shot picture immediately apart from making any changes to the picture. Cost and expertise required for one to use digital cameras is minimal. These benefits of digital cameras over the analog cameras have made them ubiquitous in the imaging. Digital imaging has become the most wide used term at present and no one bothers much about analog imaging.

           Real world scenes have a range of brightness levels so high that common digital capture devices are unable to capture the entire range. This drawback is mainly due to the restriction of capacity of the each sensor element present in the camera sensor array. Analog cameras can however capture higher range of brightness levels than common digital cameras, as the brightness levels are recorded in a continuous form through burning of oxides in the film.

There are digital cameras which can capture high dynamic range (HDR) scene information in a single snapshot such as Spheron VR camera. However, such cameras are very expensive at present. The challenge is to enable common low dynamic range (LDR) digital cameras to capture the HDR information. We shall now look at the new imaging paradigm called HDR imaging which enables one to capture higher dynamic range even with the common LDR capture devices. The advantage with such an approach is that the cost is less compared to re-designing the imaging sensor.

A real world scene which has both brightly and poorly illuminated regions is said to be a HDR scene. If we take multiple images of the scene using a digital camera by changing the exposure times, we would be able to capture all the details of the scene. Auto-Exposure Bracketing (AEB) feature in common digital cameras let one capture this sequence. However, the details of the scene are now distributed across all these multi-exposure images. We need to appropriately weigh the intensity values of these images to capture all the details in a single image. This process requires us to remove any non-linearity introduced by the digital camera before weighting the intensity values.

Many commercial software products perform this task. Some of them are HDRsoft Photomatix, pfstools,  and Adobe Photoshop (CS2 onwards).  Even recent releases of Matlab has functions such as hdrread, hdrwrite, makehdr, and tonemap to achieve basic HDR image operations. HDR images are encoded in formats such as Open EXR (.exr) and Radiance RGBE (.hdr)  which store a floating point value at each pixel location. The generated HDR image is then subjected to a process called tone mapping for display in display devices which can understand only LDR content. 

Problem in multi-exposure image capture arises when one needs to capture a dynamic scene. Real world scenes are dynamic (objects in the scene are in motion). Even an expert photographer does not have any control over the movement of objects in the scene while capturing multi-exposure images. If the scene changes are not detected in the multi-exposure images, the final HDR image will have artifacts called ghosts. Several de-ghosting algorithms have been developed which eliminate such artifacts due to motion in the scene.

The real challenge for HDR imaging in future is the development of fast algorithms which can produce artifact-free HDR image corresponding to both static and dynamic scenes. Real time tone mapping operators are also a challenge which can then enable HDR images to be visualized in common LDR displays. Inverse tone mapping techniques, which enable existing LDR content to be visualized on HDR displays, are also of much interest to the research community.

Reconstruction of High Contrast Images for Dynamic Scenes


        Our recent work titled "Reconstruction of high contrast images for dynamic scenes", published in The Visual Computer journal, provides a novel solution to the high dynamic range (HDR) imaging problem for dynamic scenes. The journal articles are accessible to all without subscription till Dec 31st 2011. So you would be able to download the article from the link The Visual Computer article.  I have posted a review article, "High Dynamic Range Imaging - A Technical Overview" which I wrote for IIT Bombay magazine called "Background Hum". The review article would motivate one to read this recent journal article and understand it better apart from providing a glimpse of the research I have been doing for the past five years.