Image Display & Enhancement

Lecture 2

Prepared by R. Lathrop 10/99

updated 1/03

Readings:

ERDAS Field Guide 5th ed

Chap 4; Ch 5:137-153;

App A Math Topics: 459-469

Analog-to-digital conversion process

A-to-D conversion transforms continuous analog signal to discrete numerical (digital) representation by sampling that signal at a specified frequency

Discrete sampled value

Continuous analog signal

Radiance, L

dt Adapted from Lillesand & Kiefer

Digital Images

• Digital Number (DN) or Brightness Value (BV) - the tonal gray scale expressed as a number, typically 8-bit number (0-255)

• Dimensionality - determined by the number of data layers (bands)

• Measurement Vector of a pixel - is the set of data file values for one pixel in all n bands

Digital Image

0

255

8 bit DN

Multiple spatially co-registered bands, can be displayed singly in B&W or in color composite

Band 1

Band 2

Band 3

Image Notation

• i = row (or line) in the image

• j = column

• k, l = bands of imagery

• Bvijk = BV in row i, column j of band k

• n = total # of pixels in an array

Rows = i = 4

Columns = j = 5

Calculating disk space

[ ( (x * y * b) * n) ] x 1.4 = output file size

where:

y = rows

x = columns

b = number of bytes per pixel

n = number of bands

1.4 adds 30% for pyramid layers and 10% for other info

Digital Image Storage Formats

• Band sequential (BSQ) - each band contained in a separate file

• Band interleaved by line (BIL) - each record in the file contains a scan line (row) of data for one band, with successive bands recorded as successive lines

• Band Interleaved by Pixel (BIP)

Summarizing data distributions

• Frequency distributions - method of describing or summarizing large volumes of data by grouping them into a limited number of classes or categories

• Histograms - graphical representation of a frequency distribution in the form of a bar chart

Summarizing Data Distributions: Histograms

0 255Digital Number

# of pixels

Measures of Central Location

• Mean - simple arithmetic average, the sum of all observations divided by the number of observations

• Median - the middle number in a data set, midway in the frequency distribution

• Mode - the value that occurs with the greatest frequency, the peak in a histogram

Measures of Central Location

0 255Digital Number

# of pixels

Mode

Mean

Median

Measures of Dispersion

• Range - the difference between the largest and smallest value

• Variance - the average of the squared deviations between the data values and the mean

• Standard Deviation - the square root of the variance, in the units of data measurement

Measures of Dispersion: Range

0 255

Digital Number

# of pixels

Min = 60 Max = 200

Example: Range = (max - min) = 200 - 60 = 140

Covariance & Correlation Matrices

• Provide a useful summary of data relationships

• High variance suggests a higher information content for that band

• High correlation suggests a substantial amount of redundancy

• Low correlation suggests that each band provides information not found in the other

Covariance Matrix

Covariance matrix 1 2 3 4 5 6 7

1 232.3 139.1 237.2 -35.3 191.9 42.0 182.62 139.1 89.9 153.4 -4.6 142.1 24.1 122.03 237.2 153.4 273.1 -26.9 249.1 46.4 219.44 -35.3 -4.6 -26.9 341.1 216.1 -38.1 25.35 191.9 142.1 249.1 216.0 555.2 33.5 305.36 42.0 24.1 46.4 -38.1 33.5 31.22 40.67 182.6 122.0 219.4 25.3 305.3 40.6 227.6

Diagonals represent band variances. Example, variance for Band 3 = 273.1Off-diagonals represent covariances. Example, covariance of Band 1 and 4 = -35.3; same as covariance of Band 4 and 1. Negative covariance: as one band increases, the other decreases.

Image Display

Computer Display Monitor has 3 color planes: R, G, B

that can display DN’s or BV’s with values between 0-255

3 layers of data can be viewed simultaneously:

1 layer in Red plane

1 layer in Green plane

1 layer in Blue plane

Image Display: RGB color compositing

Red band DN

Blue band DN

Red band DN = 0

Blue band DN = 200

Green band DN

Green band DN = 90

Blue-green pixel (0, 90, 200 RGB)

Landsat MSS bands 4 and 5

GREEN RED

Landsat MSS bands 6 and 7

INFRARED 2INFRARED 1

Note: water absorbs IR energy-no return=black

• combining bands creates a false color composite

• red=vegetation• light blue=urban• black=water

• pink=agriculture

Rutgers

Manhattan

PhiladelphiaPine barrensChesapeake BayDelaware River

MSS color composite

Primary Colors

Red Green

Blue

Subtractive Primary Colors

Yellow (R+G)

absence of blue

Cyan (G+B)

absence of red

Magenta (R+B)

absence of green

Color Additive Process

R G

B

M

Y

CW

Black background

Y C

M

R

G

BB

Color Subtractive Process

White background

Why is this image SO magenta colored?

TM 4-5-3

R-G-B

Additive Color Processcolor R G B

white 255 255 255

black 0 0 0

grey 100 100 100

red 255 0 0

yellow 255 255 0

cyan 0 255 255

magenta 255 0 255

orange 255 100 0

dark blue 0 0 100

Image Spectral Enhancement

Image spectral enhancement

0 255Digital Number

# of pixels

Min = 0 Max = 255

Image display devices typically operate over a range of 256 gray levels. Ideally the image data ranges over this full extent.

Image spectral enhancementHowever, sensor data in a single band rarely extend over this entire range, resulting in a loss of contrast. The objective of spectral enhancement is to determine a transformation function to improve the brightness, contrast and color balance and thereby enhance image interpretability.

No data No data

0 255

Digital Number

# of pixels

Min = 50 Max = 200

Image spectral enhancement: lookup tables

• Image file values are read into the image processor display memory. These values are then manipulated for display by specifying the contents of the 256 element color look-up-table (LUT). By changing the LUT, the user can easily change the output display without changing the original file DN values.

Data FileGreen band DN = 100

LUTGreen band DN = 190

Enhanced Green pixel

Display DN = 190

Input LUT Output

Image spectral enhancement: Lookup tables

• Since the same transformation function is used for all the pixels in the image, it is calculated for all possible DN before processing the image. The resulting values of DN are stored in a lookup table (LUT).

• All possible values are computed only once - computationally efficient.

• Each pixel’s DN is then used to index the LUT to find the appropriate DN’ in the output image

LUT Input-Output relationship: ideal

00

255

255

Output

DN

Input DN

Input = 127

Output = 127

From ERDAS Imagine Field Guide 5th Ed.

1-to-1 transformation function

Transformation function

00

255

255

Output

DN

Input DN

The steeper the transformation line -> the greater the contrast stretch

LUT Breakpoint Editor for ERDAS Imagine

Image spectral enhancement: NO contrast stretch

0 255

60 108 158

Image spectral enhancement: Min-max linear contrast stretch

0 255

60 108 158

125

Linear transformation function

The steeper the transformation line -> the greater the contrast stretch

00

255

255

Output

DN

Input DN

Input min = 60

Output min = 0

Input max = 158

Output max = 255

Image spectral enhancement: Min-max linear contrast stretching• Linear stretch: uniform expansion , with all

values, including rarely occurring values, weighted equally

• DN’ = [(DN - MIN)/(MAX - MIN)] x 255

• Example: DN = 108DN’ = [(108 - 60) / (158 - 60)] x 255

= [48 / 98] x 255 = .49 x 255 = 125

Example from Lillesand & Kiefer, 2nd ed

Image spectral enhancement: Std. Dev. linear contrast stretching• If data histogram near normal, then 95% of

the data is within +- 2 std dev from the mean, 2.5% in each tail

0 255

Overstretching: too much of a good thing

Image spectral enhancement: Histogram stretching

• Histogram stretch: image values are assigned to the display LUT on the basis of their frequency of occurrence

greatest contrast near modeleast contrast in histogram tails

0 255

108 158

38

60

Example from Lillesand & Kiefer, 2nd ed

Histogram stretching

00

255

255

Output

DN

Input DN

Input min = 60

Output min = 0

Input max = 158

Output max = 255

Nonlinear function in tails of distribution

Image spectral enhancement: Contrast stretching

• Special stretch: display range can be assigned to any particular user-defined range of image values

Example from Lillesand & Kiefer, 2nd ed

0 255

15860 92

Special piecewise stretching

00

255

255

Output

DN

Input DN

Different sections of the input data stretched to different extents;

I.e. different pieces of the transformation function line with different slopes

Adaptive Filtering

• Image stretching represents a global operator – i.e. applies the stretch equally across the entire scene and doesn’t take into account local differences in image brightness or other characteristics. Not always the best approach.

• Adaptive filters work by adapting the stretch to a smaller region of interest, usually the area within a moving window.

Multisensor fusion

• Various techniques have been developed to merge low spatial resolution (but high spectral resolution) with high spatial resolution (but low spectral resolution, e.g., panchromatic) imagery example: TM and ETM+ PAN

• Multisensor fusion will become more common as the new high spatial resolution PAN imagery becomes more widely available

One meter Pan-sharpened Multispectral IKONOS imagery (simulated)

Tennis courts in Washington Park, Denver, CO

Quickbird image example: Barnegat Bay, NJ 10/18/2004

Panchromatic: 0.61-1m

Multispectral (color): 2.5-4 m

Pixel size for this merged Pan-Multi image is 0.7 m

What’s going on here?

Example: IHS Color-space transform

• RGB to IHS: transform fro Red-Green-Blue color space to Intensity-Hue-Saturation

• Low and high resolution images are co-registered and resampled to same GRC

• 3 bands of the multispectral image converted to IHS space then PAN band substituted for the Intensity component, then back-transformed into RGB color space

• A disadvantage is that only 3 bands may be transformed simultaneously

Intensity, Hue & Saturation color coordinate system

Saturation

Hue

Intensity

0

255

0255

redgreen

blue

255,0

Example: PCA Spectral domain fusion

• Low and high resolution images are co-registered and resampled to same GRC

• PCA of multispectral image• Substitution of PAN image for 1st PC, often the

“brightness component”, then backtransform to image space

• This technique can be used for any number of bands

• Generally a good compromise between limited spectral distortion and visually attractiveness

Example: High Pass Filter (HPF) method

• Capture high frequency information from the high spatial resolution panchromatic image using some form of high pass filter

• This high frequency information then added into the low spatial resolution multi-spectral imagery

• Often produces less distortion to the original spectral characteristics of the imagery but also less visually attractive

Example: Brovey Transform fusion

For each spectral band i

[DNBi / (DNB1 + DNB2 + DNB3)] x (DN high res. Image)

Brovey transform was developed to increase contrast in the low and high tails of the image histogram for visual interpretation- doesn’t preserve the original scene radiometry.

Other methods: Multiplicative

Spherical Coordinates

Wavelets

Simple Image Segmentation• Simplifying the image into 2 classes based on

thresholding a single image band, so that additional processing can be applied to each class independently

• < DN threshold = Class 1• >= DN threshold = Class 2• Example: gray level thresholding of NIR band used

to segment image into land vs. water binary mask

+