computational photography opti 600c jim schwiegerling 2014

61
Computational Photography OPTI 600C Jim Schwiegerling 2014

Upload: louisa-noreen-robertson

Post on 15-Jan-2016

231 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Computational Photography OPTI 600C Jim Schwiegerling 2014

Computational PhotographyOPTI 600C

Jim Schwiegerling

2014

Page 2: Computational Photography OPTI 600C Jim Schwiegerling 2014

Computational Photography

• Many definitions, many concepts have been performed for years, but just not called computational photography.

• Conventional photography captures a 2D projection of a scene and has limited flexibility in adjusting its display (exposure, contrast, color balance).

• Computational photography incorporates “current” technologies in lighting, optics, sensors and post-processing to create images beyond what conventional photography can provide.

• Digital images make this process easier.

Page 3: Computational Photography OPTI 600C Jim Schwiegerling 2014

Class Outline

• Software for manipulating data such as Matlab (e.g. Octave free alternative).

• Images will be provided for various assignments, but if you have access to a DSLR a better variety of images will be available to play with.

• Class grade will be based on homework assignments. I am mainly interested in you trying the techniques even if the results aren’t great.

• Be creative!

Page 4: Computational Photography OPTI 600C Jim Schwiegerling 2014

Traditional Photography

http://www.rca.ac.uk/media/images/RCA_10-13_1017.width-1000.jpg

Page 5: Computational Photography OPTI 600C Jim Schwiegerling 2014

Traditional Photography

• Illumination – Sun, lamps, flash, ambient, self-luminous• Subject – spectral reflectance, polarization, 3D• Optics – focal length, aperture setting• Sensor – glass plates, film, CMOS, CCD, 2D• Post-Processing – dodge and burn, Photoshop, image

processing.

Page 6: Computational Photography OPTI 600C Jim Schwiegerling 2014

Dodge & Burn

http://petapixel.com/assets/uploads/2013/09/dean.jpg

Page 7: Computational Photography OPTI 600C Jim Schwiegerling 2014

PhotoShop

http://img.xcitefun.net/users/2010/01/143983,xcitefun-photoshopped-images-1.jpg

Page 8: Computational Photography OPTI 600C Jim Schwiegerling 2014

Computational Photography

http://web.media.mit.edu/~raskar/photo/Slides/00IntroRamesh.gif

Page 9: Computational Photography OPTI 600C Jim Schwiegerling 2014

High Dynamic Range

Page 10: Computational Photography OPTI 600C Jim Schwiegerling 2014

Extended Depth of Focus

Page 12: Computational Photography OPTI 600C Jim Schwiegerling 2014

Prokudin-Gorskii Collection

• Prokudin-Gorskii made a photographic survey of the Russian empire between 1905 and 1915.

• Used special camera to expose three images through red, green and blue filters on long strip of glass.

• Images can be combined to create single color image.• Available at www.loc.gov/pictures/collection/prok/

Page 13: Computational Photography OPTI 600C Jim Schwiegerling 2014

Prokudin-Gorskii Collection

Page 14: Computational Photography OPTI 600C Jim Schwiegerling 2014

Multispectral Imaging

Page 15: Computational Photography OPTI 600C Jim Schwiegerling 2014

Multispectral Images

Page 16: Computational Photography OPTI 600C Jim Schwiegerling 2014

Computed Tomographic Imaging Spectrometer (CTIS)

Page 17: Computational Photography OPTI 600C Jim Schwiegerling 2014

Color and Color VisionThe perceived color of an object depends onfour factors:1. Spectrum of the illumination source2. Spectral Reflectance of the object3. Spectral response of the photoreceptors (including bleaching)4. Interactions between photoreceptors

Page 18: Computational Photography OPTI 600C Jim Schwiegerling 2014

Light Sources

400 700nm

SpectralOutput

400 700nm

SpectralOutput

400 700nm

SpectralOutput

White Monochromatic Colored

Page 19: Computational Photography OPTI 600C Jim Schwiegerling 2014

Pigments

Red Green Blue

400 700nm

SpectralReflectance

400 700nm

SpectralReflectance

400 700nm

SpectralReflectance

Page 20: Computational Photography OPTI 600C Jim Schwiegerling 2014

Subtractive ColorsPigments (e.g. paints and inks) absorb different portions of the spectrum

400 700nm

SpectralOutput

400 700nm

SpectralOutput

400 700nm

SpectralOutput

400 700nm

400 700nm

400 700nm

Multiply the spectrum of the light source by the spectral reflectivityof the object to find the distribution entering the eye.

Page 21: Computational Photography OPTI 600C Jim Schwiegerling 2014

Blackbody RadiationFor lower temperatures,blackbodies appear red.As they heat up, theshift through the spectrumtowards blue.

Our sun looks like a6500K blackbody.

Incandescent lights are poorefficiency blackbodiesradiators.

Page 22: Computational Photography OPTI 600C Jim Schwiegerling 2014

Gas-Discharge & Fluorescent Lamps

A low pressure gas or vapor isencased in a glass tube. Electricalconnections are made at the ends ofthe tube. Electrical discharge excitesthe atoms and they emit in a seriesof spectral lines. We can use individual lines for illumination (e.g. sodium vapor) or ultraviolet lines to stimulate phosphors.

Page 23: Computational Photography OPTI 600C Jim Schwiegerling 2014

CIE Standard IlluminantsIlluminant A - Tungsten lamp looking like a blackbody of 2856 KIlluminant B - (discontinued) Noon sunlight.Illuminant C - (discontinued) Noon sunlight.

Illuminant D55 - (Occasionally used) 5500 K blackbodyIlluminant D65 - 6500 K blackbody, looks like average sunlight and

replaces Illuminants B and C.Illuminant D75 - (Occasionally used) 7500 K blackbody

Page 24: Computational Photography OPTI 600C Jim Schwiegerling 2014

Illuminants A and D65

0

50

100

150

200

250

300

300 500 700

Wavelength (nm)

Sp

ectr

al R

adia

nce

Illuminant A

Illuminant D65

Page 25: Computational Photography OPTI 600C Jim Schwiegerling 2014

Additive ColorsSelf-luminous Sources (e.g. lamps and CRT phosphors) emit differentspectrums which combine to give a single apparent source.

400 700nm

SpectralOutput

400 700nm

SpectralOutput

400 700nm

SpectralOutput

Add the spectrums of the different light sources to get the spectrum of the apparent sourceentering the eye.

Page 26: Computational Photography OPTI 600C Jim Schwiegerling 2014

Color Models

• Attempt to put all visible colors in a ordered system.• Mathematics based, art based and perceptually based

systems.

Page 27: Computational Photography OPTI 600C Jim Schwiegerling 2014

RGB Color ModelA red, green and blue primary are mixed in different proportionsto give a color

(1,0,0) is red(0,1,0) is green(0,0,1) is blue

24 bit color on computer monitors devote 8 bits (256 values) to eachprimary color (i.e. red can take on values (0…255) / 255)

(1,1,1) is white(0,0,0) is black

Page 28: Computational Photography OPTI 600C Jim Schwiegerling 2014

HSB (HSV) Color Model

Hue – color is represented by angleSaturation – amount of white representedby radial positionBrightness (Value) – intensity is representedby the vertical dimension

Page 29: Computational Photography OPTI 600C Jim Schwiegerling 2014

HLS Color ModelHue – color is represented by angleLightness – intensity is representedby the vertical dimensionSaturation – amount of white representedby radial position

Page 30: Computational Photography OPTI 600C Jim Schwiegerling 2014

Commission internationale de l'Eclairage, CIE

• 90 year old commission on color• Recognized as the standards body for illumination & color. • Has defined standard illuminants and human response

curves.

Page 31: Computational Photography OPTI 600C Jim Schwiegerling 2014

Luminosity FunctionsThe spectral responses of the eye are called the luminosity functions. TheThe V(l) curve (photpic response) is for cone vision and the V’(l) curve (scoptopic response) is for rod vision. V(l) was adopted as a standard by the CIE in 1924. There are some errors for l<500 nm, that remain. TheV’(l) curve was adopted in 1951 and assumes an observer younger than30 years old.

00.10.20.30.40.50.60.70.80.9

1

380 480 580 680 780

Wvaelength (nm)

Rel

ativ

e S

pec

tral

Sen

siti

vity

Photopic

Scotopic

V(l)V’(l)

Page 32: Computational Photography OPTI 600C Jim Schwiegerling 2014

Human Color Models

RG

B

Cl

Observer adjusts the Luminances of R, G and Blights until they match Cl

Cl = r’( )l R + g’( )l G + b’( )l B

R = 700.0 nmG = 546.1 nmB = 435.8 nm

Page 33: Computational Photography OPTI 600C Jim Schwiegerling 2014

1931 CIE Color Matching Functions

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

380 430 480 530 580 630 680 730 780

Wavelength (nm)

Ch

rom

ati

city

Co

ord

ina

tes

r'()g'()

b'()

435.8 546.1 700.0

Page 34: Computational Photography OPTI 600C Jim Schwiegerling 2014

Color Matching Functions

R

G

B

Cl

What does a negative value of the ColorMatching Function mean? Bring one light to the other side of the field. Observer now adjusts, for example, the Luminances of G and B lights until they match C l + R

Cl + r’( )l R = g’( )l G + b’( )l B

R = 700.0 nmG = 546.1 nmB = 435.8 nm

Page 35: Computational Photography OPTI 600C Jim Schwiegerling 2014

1931 CIE Color Matching Functions

The CIE defined threetheoretical primaries x’, y’ and z’ such that the color matching functions are everywhere positive and the “green” matching function is the same as the photopic response of the eye. -0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

380 430 480 530 580 630 680 730 780

Wavelength (nm)

Ch

rom

ati

city

Co

ord

ina

tes

x'()y'()

z'()

Page 36: Computational Photography OPTI 600C Jim Schwiegerling 2014

Conversion x’y’z’ to r’g’b’

'z

'y

'x

17860.000255.000092.0

01571.025243.009117.0

08283.015860.041846.0

'b

'g

'r

The x’, y’, z’ are more convenient from a book-keeping standpointand the relative luminance is easy to determine since it is related to y’.

The consequence of this conversion is that the spectral distribution ofthe corresponding primaries now have negative values. This meansthey are a purely theoretical source and can not be made.

Page 37: Computational Photography OPTI 600C Jim Schwiegerling 2014

Tristimulus Values X, Y, Z

d)('z)(PZ

d)('y)(PY

d)('x)(PX

0

0

0

The tristimulus values are coordinatesin a three dimensional color space. Theyare obtained by projecting the spectraldistribution of the object of interest P(l)onto the color matching functions.

Page 38: Computational Photography OPTI 600C Jim Schwiegerling 2014

Chromaticity Coordinates x, y, z

yx1ZYX

Zz

ZYX

Yy

ZYX

Xx

The chromaticity coordinatesare used to normalize out thebrightness of the object. This way,the color and the brightness can bespearated. The coordinate z is notindependent of x and y, so this isa two dimensional space.

Page 39: Computational Photography OPTI 600C Jim Schwiegerling 2014

Chromaticity Coordinates

Light SourceSpectralDistribution

Object SpectralReflectance orTransmittance

Spectral Distributionof Light entering theeye.

x =

Calculate TristimulusValues X, Y, Z

Project ontox’, y’, z’

CalculateChromaticityCoordinates

Normalize

Plot (x, y)

Page 40: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - Spectrally Pure ColorsSuppose P(l) = d(l-lo)

)('zd)('z)(Z

)('yd)('y)(Y

)('xd)('x)(X

o

0

o

o

0

o

o

0

o

yx1z

)('z)('y)('x

)('yy

)('z)('y)('x

)('xx

ooo

o

ooo

o

Plotting x vs. y for spectrally pure colors gives the boundary of color vision

Page 41: Computational Photography OPTI 600C Jim Schwiegerling 2014

CIE Chromaticity Chart

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

x chromaticty coordinate

y ch

rom

atic

ity

coo

rdin

ate

Page 42: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - White LightSuppose P(l) = 1

1d)('zZ

1d)('yY

1d)('xX

0

0

0

33.0yx1z

33.0111

1y

33.0111

1x

Page 43: Computational Photography OPTI 600C Jim Schwiegerling 2014

CIE Chromaticity Chart

Page 44: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - MacBeth Color Checker

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

380 430 480 530 580 630 680 730 780

Wavelength (nm)

Spec

tral R

efle

ctan

ce

Blue Sky

Page 45: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - Blue Sky Patch

0

20

40

60

80

100

120

140

380 480 580 680 780

Wavelength (nm)

Rel

ativ

e R

adia

nce

Illuminant C

After Reflection

The Macbeth colorchecker assumes thatIlluminant C is usedfor illumination.

Multiply the spectraldistribution of IlluminantC by the spectral reflectance of the color patch to get the light entering the eye.

Page 46: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example Blue Sky Patch

253.)9.3796.1921.188/(6.192y

247.)9.3796.1921.188/(1.188x

9.379)('z)(PZ

6.192)('y)(PY

1.188)('x)(PX

Page 47: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - MacBeth Color Checker

Dark Skin

00.050.1

0.150.2

0.250.3

0.350.4

0.450.5

380 430 480 530 580 630 680 730 780

Wavelength (nm)

Sp

ectr

al R

efle

ctan

ce

Page 48: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example - Dark Skin Patch

Dark skin is much darker than the blue sky. The chromaticity coordinates,however, remove the luminance factor and onlylook at color.

0

20

40

60

80

100

120

140

380 480 580 680 780

Wavelength (nm)

Rel

ativ

e R

adia

nce

Illuminant C

Dark Skin

Blue Sky

Page 49: Computational Photography OPTI 600C Jim Schwiegerling 2014

Example Dark Skin

35.0)9.3796.1921.188/(6.192y

41.0)9.3796.1921.188/(1.188x

2.66)('z)(PZ

8.98)('y)(PY

6.113)('x)(PX

Page 50: Computational Photography OPTI 600C Jim Schwiegerling 2014

CIE Chromaticity DiagramThink of this as a distortedversion of the HSB color model.

W = White Point (0.33,0.33)D = Dominant Wavelength (hue)

W

D

AWD

WApurity excitationp C

BC = Complimentary Color

WC

WBpurity excitationp

Page 51: Computational Photography OPTI 600C Jim Schwiegerling 2014

MacAdam EllipsesJust noticeable differences for two similar colors is nonlinear onthe CIE diagram. Would like acolor space that these ellipses become circles. (ellipses are 3xlarger than actuality)

Page 52: Computational Photography OPTI 600C Jim Schwiegerling 2014

1976 CIELABPlastic, Textile & Paint L* is related to the lightness and

is nonlinear to account for thenonlinear response of the visualsystem to luminance. The a’s andb’s distort the CIE diagram to make the MacAdam ellipses moreround.

Xn, Yn and Zn are for white

0.008856sfor 16/1167.787sf(s)

0.008856sfor sf(s) where

Z

Zf

Y

Yf200*b

Y

Yf

X

Xf500*a

16Y

Yf116*L

3/1

nn

nn

n

Page 53: Computational Photography OPTI 600C Jim Schwiegerling 2014

1976 CIELAB

*

*

ab

***ab

a

btanh

baC

1

22

180

Polar Coordinates

222 ****ab baLE

Color Difference

Page 54: Computational Photography OPTI 600C Jim Schwiegerling 2014

CIELAB to XYZPolar to Cartesian

CIELAB to XYZ

Page 56: Computational Photography OPTI 600C Jim Schwiegerling 2014

Gamma Correction

Page 57: Computational Photography OPTI 600C Jim Schwiegerling 2014

sRGB Color Space

Page 58: Computational Photography OPTI 600C Jim Schwiegerling 2014

sRGB to XYZ Conversion

Page 59: Computational Photography OPTI 600C Jim Schwiegerling 2014

Original Image

Page 60: Computational Photography OPTI 600C Jim Schwiegerling 2014

White Balanced

Page 61: Computational Photography OPTI 600C Jim Schwiegerling 2014

Calibrated to sRGB