playing detective with digital images - welcome to the …lazebnik/research/fall08/lec11...a history...

Playing detective with digital imagesPlaying detective with digital images

Two issuesTwo issues

• Can we fool casual observers?– How realistic is photorealistic?

• What about experts with lots of time and resources?– Digital image forensics

A history of photo manipulationA history of photo manipulation

• 1860: the portrait of Lincoln is a composite of Lincoln's head and John Calhoun's body

http://www.cs.dartmouth.edu/farid/research/digitaltampering/


• 1917: “Cottingley fairies”: five pictures taken by Elsie Wright (16) and Frances Griffiths (10) in England



• 1930s: Stalin had disgraced comrades airbrushed out of his pictures


http://www.newseum.org/berlinwall/commissar_vanishes/index.htm


• 1936: same story with Mao


A history of photo manipulationA history of photo manipulation• May 1970: This Pulitzer Prize winning photo by John Filo shows Mary

Ann Vecchio screaming as she kneels over the body of student Jeffrey Miller at Kent State University, where National Guardsmen had fired into a crowd of demonstrators, killing four and wounding nine. When this photo was published in LIFE Magazine, the fence post directly behind Vecchiowas removed.



• 1989: Oprah’s head on somebody else’s body



• 1994: O.J. Simpson’s mug shot modified to appear more menacing



• 2000: black student’s face inserted into UW brochure



• 2003: Beatles album cover modified to remove cigarette from McCartney’s hand


A history of photo manipulationA history of photo manipulation• 2003: This digital composite of a British soldier in Basra, gesturing to Iraqi

civilians urging them to seek cover, appeared on the front page of the Los Angeles Times shortly after the U.S. led invasion of Iraq. Brian Walski, a staff photographer for the Los Angeles Times and a 30-year veteran of the news business, was fired after his editors discovered that he had combined two of his photographs to "improve" the composition.



• 2004: Composite of John Kerry and Jane Fonda


A history of photo manipulationA history of photo manipulation• 2006: This photograph by Adnan Hajj, a Lebanese photographer, showed

thick black smoke rising above buildings in the Lebanese capital after an Israeli air raid. The Reuters news agency initially published this photograph on their web site and then withdrew it when it became evident that the original image had been manipulated.



• 2007: French Magazine Paris Match altered this photograph of French President Nicolas Sarkozy


A history of photo manipulationA history of photo manipulation• 2008: This photograph, by Liu Weiqiang of the Daqing Evening News,

won an award for "one of the ten most impressive news photos of 2006". This photograph was recently revealed to be a composite of two separate photographs: the antelopes and the train.


Detecting digital tampering: cloningDetecting digital tampering: cloning

• Exposing Digital Forgeries by Detecting Duplicated Image Regions– A.C. Popescu and H. Farid– Technical Report, TR2004-515, Dartmouth College, Computer Science

Detecting digital tampering: retouchingDetecting digital tampering: retouching

• Exposing Digital Forgeries in Color Filter Array Interpolated Images – A.C. Popescu and H. Farid– IEEE Transactions on Signal Processing, 53(10):3948-3959, 2005

Detecting digital tampering: LightingDetecting digital tampering: Lighting

• Exposing Digital Forgeries by Detecting Inconsistencies in Lighting – M.K. Johnson and H. Farid– ACM Multimedia and Security Workshop, New York, NY, 2005

Detecting inconsistencies in lightingDetecting inconsistencies in lightingI(x,y) = N(x,y)·L + A

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

),(

),(),(

1),(),(),(

1),(),(),(1),(),(),(

22

11

222222

111111

nn

z

y

x

nnznnynnx

zyx

zyx

yxI

yxIyxI

ALLL

yxNyxNyxN

yxNyxNyxNyxNyxNyxN

MMMMM

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

),(

),(),(

1),(),(

1),(),(1),(),(

22

11

2222

1111

nn

y

x

nnynnx

yx

yx

yxI

yxIyxI

A

LL

yxNyxN

yxNyxNyxNyxN

MMMM

I(x,y) = N(x,y) ·L(x,y) + A

Full 3D case:

For points on the occluding contour:

P. Nillius and J.-O. Eklundh, “Automatic estimation of the projected light source direction,” CVPR 2001

Detecting inconsistencies in lightingDetecting inconsistencies in lighting

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

),(

),(),(

1),(),(

1),(),(1),(),(

22

11

2222

1111

nn

y

x

nnynnx

yx

yx

yxI

yxIyxI

A

LL

yxNyxN

yxNyxNyxNyxN

MMMM

Detecting inconsistencies in lightingDetecting inconsistencies in lighting

Fake photo

Real photo

Lighting: Lighting: SpecularSpecular highlights in the eyehighlights in the eye

M.K. Johnson and H. Farid, “Exposing Digital Forgeries Through SpecularHighlights on the Eye,” 9th International Workshop on Information Hiding,

2007

Analyzing illumination in paintingsAnalyzing illumination in paintings• Inferring Illumination Direction Estimated from Disparate Sources in

Paintings: An Investigation into Jan Vermeer's Girl With a Pearl Earring– M.K. Johnson, D. Stork, S. Biswas and Y. Furuichi– SPIE Symposium on Electronic Imaging, San Jose, CA, 2008

Analyzing illumination in paintingsAnalyzing illumination in paintings

• Computer Vision, Image Analysis, and Master Art: Part 2 – D. Stork and M.K. Johnson– IEEE Multimedia, October-December 2006

Georges de la Tour

Christ in the Carpenter’s Studio (1645)

A historical detective story: Fenton CannonballsA historical detective story: Fenton Cannonballs

• Errol Morris, NY Times, 2007

Roger Fenton, “Valley of the Shadow of Death”, April 23, 1855

Fenton cannonballs: Which came first?Fenton cannonballs: Which came first?

“Off” “On”

HighHigh--resolution photosresolution photos

Errol Morris on locationErrol Morris on location

Some observationsSome observations

• 19th century photographic emulsions are blue-sensitive and hence cannot record the sky – overcast, partially cloudy and sunny skies are all overexposed

• If the photos were taken about an hour apart, the change in illumination angle would be about 10 degrees – within the margin of error

• The interpretation of the change in illumination direction would depend on which way the photographer was facing

• The illumination in one of the images (“On”) appears to be more diffuse than in the other (“Off”)

The solution (hopefully)The solution (hopefully)

• Compare positions of distinctive rocks

“Off” “On”

The solution (hopefully)The solution (hopefully)

• Compare positions of distinctive rocks

playing detective with digital images - welcome to the …lazebnik/research/fall08/lec11...a history...

Documents