Overview of Haze Removal Methods

Matteo PedoneMachine Vision Group, University of Oulu, Finland

Overview of Haze Removal Methods

1. Description of the problem

2. Overview of current approaches found in literature

3. Strengths and weaknesses of present methods

4. Description of our method

The Atmospheric Scattering Model

• A particle scatters incident light

• The nature of scattering depends on material properties,

shape and size

• The exact form and intensity of the scattering pattern varies

dramatically with particle size

The Atmospheric Scattering Model

Haze: •constituted of aerosol (small particles suspended in gas)•Main sources: volcanic ashes, foliage exudation, combustion products, sea salt…•Haze particles are larger than air molecules but smaller than fog droplets.•produce a distinctive gray or bluish hue and affects visibility.•Extends to altitudes of several Km

The Atmospheric Scattering Model

Fog: •Same origins as haze, associated with an increase in relative humidity of an air•Size of water droplets increases •Haze can turn into fog (transition state: mist)•Reduces visibility more than haze•Extends to altitudes of few hundred meters.

The Atmospheric Scattering Model

The Atmospheric Scattering Model

Important physical mechanisms:

•Attenuation: radiance of a scene-point falls as

its distance from the observer increases

•Airlight: Atmosphere behaving like a source of

light. Due to multiple scattering. Increases with

distance.

The Atmospheric Scattering Model

Mathematical Model:

• I(x) is the observed radiance at x

• J(x) is the original scene radiance at x

•A is the airlight

•t(x), scalar called transmission: describes how the radiance of a point in the

scene is attenuated according to its distance d from the observer

•Note that I, J, A are (R,G,B) triplets

The Atmospheric Scattering Model

Mathematical Model:

• In order to remove the effect of haze, one must recover J(x)

• Quantities A and t are typically unknown

•I(x) is known

The Atmospheric Scattering Model

References

Narasimhan & Nayar, “Vision and the Atmosphere”, International

Journal of Computer Vision, 2001

Overview of Haze Removal Methods

1. Description of the problem

2. Overview of current approaches found in literature

3. Strengths and weaknesses of present methods

4. Description of our method

Current Methods for Haze Removal

• Can be grouped into several categories

1. With multiple images

2. With one image + depth-map

3. Single image

• Subcategories of the ones above are:

1. Requires user interaction

2. Fully automatic

• We are mostly interested in Single-Image methods

Multiple Image Approaches

Assume 2+ images of the same scene are taken:

Under different weather conditions [1]

or

With different polarization filters [2]

[1] Narasimhan & Nayar, “Vision and the Atmosphere”, 2001

[2] Schechner et al. 2003, “Polarization-based vision through haze”, Applied Optics

42

Multiple Image Approaches

Narasimhan & Nayar’s method

•Assumes 2+ bad weather images are given

•Uses geometric constraints to estimate A

•The airlight component [1-t(x)] is estimated from corresponding pixels of the two

bad weather images

Multiple Image Approaches

Narasimhan & Nayar’s method, RESULTS

(a),(b) Foggy images (c) Dehazed image, (d) Clear weather images

One Image + Depth + Texture

Kopf et al. Method: Deep Photo project from SIGGRAPH 2008

•Assumes a 3D model of the scene is given (e.g.: from Google Maps)

•Assumes textures of the scene are given (from satellite or aerial photos)

•Requires user interaction to align the 3D model with the scene

•Very accurate results

Single Image Approaches

• Do not require information extracted from additional images

• Do not require depth-information

• Typically rely upon statistical assumptions, and or the nature of the scene (e.g. part of the sky is visible)

• Sometimes they require user interaction

• Most relevant:• Fattal’s ”Single-Image Dehazing”, SIGGRAPH 2008• He’s ”Single Image Haze Removal Using Dark Channel Prior”, CVPR 2009

Single Image Approaches

He’s Method (with Dark Channel Prior)

• Assumes a portion of the scene is dominated by airlight

• STATISTICAL ASSUMPTION: ”in most of the non-skypatches, at least one color channel has very low intensity atsome pixels. In other words, the minimum intensity in sucha patch should have a very low value”

• The 1st assumption is used to estimate airlight, the 2nd assumption is used to estimate the transmission

Single Image Approaches

He’s Method (with Dark Channel Prior)

• Dark Channel:

• Airlight is estimated by picking up the pixels of the image corresponding to the 0.1% brightest pixels in the dark channel, and then choosing the one with maximum intensity.

Single Image Approaches

He’s Method (with Dark Channel Prior)

• Dark Channel:

• He shows that the transmission can be estimated by calculating:

Single Image Approaches

He’s Method (with Dark Channel Prior)

• Dark Channel:

• He shows that the transmission can be estimated by calculating:

Dark channel of the image divided by the airlight color

Single Image Approaches

He’s Method (with Dark Channel Prior)

• Airlight and transmission are sufficient to invert the model and retrieve the original radiance of the scene.

• Dark channel is computed on square neighborhoods Block artifacts and halos are reduced by using a soft-matting algorithm.

Overview of Haze Removal Methods

1. Description of the problem

2. Overview of current approaches found in literature

3. Strengths and weaknesses of present methods

4. Description of our method

Summary

Multiple image methods

•require special equipment (polarizers) or same scene under

different weather conditions.

•They don’t necessarily produce better results than single-images

approaches

Summary

One Image + 3D model + textures

•Accurate and does not require special equipment

•Requires a considerable amount of special information (3D model,

and aerial photos of the scene)

•Requires user interaction

Summary

Single-Image methods

•do not require special equipment, nor extra information

•They either make assumption on the nature of the scene, or

require little interaction by the user

Summary

Single-Image methods

•It is known what are the consequences of a bad estimate for the

transmission haze is not completely removed, or it is removed

where there is no haze (overboost contrast)

Summary

• None of the aforementioned authors shows what happens when

the airlight estimate is inaccurate (motivation of our work)

Overview of Haze Removal Methods

1. Description of the problem

2. Overview of current approaches found in literature

3. Strengths and weaknesses of present methods

4. Description of our method

Estimation of Airlight• Collect statistics of airlight colors from 100+ natural images (daylight and twilight hazy scenes)

• Manually select 32x32 pixel patch with “full haze”

• Airlight colors are scattered around a 29.8784 degrees line in hue-saturation plane, and most are close to the origin (=> low saturation).

Estimation of Airlight• Extract patches of 13x13 pixels from hazy image according to the following criteria:

1. The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes)

2. The pixels in the patch do not have too low or too high transmission (avoid degenerate cases)

Estimation of Airlight• Extract patches of 13x13 pixels from hazy image according to the following criteria:

1. The patch contains pixels with same transmission and hue but with different shades (=> same direction for R but different magnitudes)

2. The pixels in the patch do not have too low or too high transmission (avoid degenerate cases)

3. Pixels in the patches do not have too low saturation (hue would not be reliable).

4. Pixels in the patches are not too dark or too bright in average, and variance should not be too high (noise) or too low (homogeneous areas with no shades).

Estimation of Airlight• Solve minimization problem: find an airlight color vector that enforces:

1. Perpendicularity to patches albedos2. Closeness to natural airlight hue line3. Low saturation

• ni : normal to the plane containing the [R,G,B] values of the pixels in the i-th patch extracted

• nsky : unit-vector in RGB space having direction corresponding to the statistical hue of airlight in natural images

• w : unit vector [1,1,1]/31/2

• c(ni ): scalar associated with the i-th patch (based of residual error, see paper)

• : weight parameters (respective default values: 3, 0.02)

Estimation of Airlight

Results with artifical haze (Middlebury dataset)

Estimation of AirlightResults with real images

THANK YOU