power point presentation notes

8/1/2019 Power Point Presentation Notes

1/14

Hi, I'm Adam Steinberger, a Computer Science and

Music student here at Skidmore. This summer, Prof.

Mike Eckmann and I have been researching Digital

Image Source Identification.

Slide 1 of 14


2/14

Cameras often insert data into digital photos that

state make, model, time and place, among other

things. This extra data is easily spoofed using

modern software, so we can't trust it. When given an

image, we want to determine which camera took it.

Slide 2 of 14


3/14

Since we can't trust the text data cameras already

provide, we need something else to trust. We want to

create a fingerprint per camera based on the pixels

of digital photos. Certain data in the pixels will

hopefully allow us to uniquely determine the source

cameras of a given photo. Some examples of image

data include sensor dust markings, noise and white

balancing. These have been used to create almost

unique fingerprints and are a lot harder to spoof than

text data.

Slide 3 of 14


4/14

Given a digital image, we want to determine what camera took

it. We created a classifier built up from image attributes. The

way we created our classifier was by training it using the

attributes from about half of our images. By training the

classifier we mean that it creates a fingerprint for each camera.

The attributes from the other half of our images are used to test

the classifier to determine its accuracy. We took the pictures

ourselves, along with our colleagues, from 25 different

cameras. So obviously we know which camera each image

belongs to. The attributes the classifier uses relates to the pixel

colors in images. These attributes also relate to the

demosaicing process in cameras, which I will describe shortly.

Slide 4 of 14


5/14

Digital images are composed of pixels, each containing a red, green

andblue component. When a digital photo is taken, light from the

outside world hits sensors inside the camera that make up a Color

Filter Array. Each of these sensors couldcapture red, green orblue.

To keep costs down, cameras capture only one color per pixel. The

other two colors are computed using neighborhoods of pixels.

For example, a pixel corresponding to a blue sensor. (show blue

sensor in middle)

needs some way to obtain red and green values. Cameras will

estimate a red and green component by some computation of the

neighboring red and green sensors. (show different neighborhoods)

Differing weights of sensor outputs and different sized

neighborhoods are used in different camera makes and models.

These differing processes can help us create fingerprints for cameras.

Slide 5 of 14


6/14

The process I just described, where cameras

compute the two missing colors for each pixel from

some neighborhood around it, is called demosaicing.

Typically cameras use unique proprietary

demosaicing processes. Also, a pixel in a smooth

area is usually processed differently than pixels on

edges. (show smooth and edge pixels)

Different makes and models of cameras use different

proprietary demosaicing processes.

Slide 6 of 14


7/14

Just a head's up: We're approaching some details

worth mentioning about our attributes. I've

developed original software to generate attributes

from photos related to the demosaicing process

inside cameras. I've written code that can tell the

difference between pixels in smooth areas versus

those on an edge. The next couple slides are details

to give you a sense of how some of these attributes

were computed. The implementation of this next

process is a small sample of the stuff I worked on

this summer.

Slide 7 of 14


8/14

Here we have one color for a single pixel. This is the red channel of

a pixel, which by the way also has a green and blue channel. Since I

developed code that worked on red separately from green and blue,

I'm showing you an example here where we are just focusing on the

red channel. But this works for green and blue channels, too. (click)

The neighborhood of a smooth pixel might look like this, where the

values here are within a small range. This is a defining characteristic

of smooth areas. (click)

To get an error, we start with the origin pixel. (click)

Then we sort the pixel's neighbors. (click)

Removing the outer pixels, we find the median neighbor. (click)

Subtracting the origin pixel with its median neighbor gives us an

error value. (click)

Slide 8 of 14


9/14

Now let's consider an edge pixel. (click)

This is what the neighborhood of an edge might look

like. Notice the pixels on the left have significantly

smaller values than those on the right. (click)

Start with the origin pixel. (click)

We remove the edge that doesn't belong. (click)

Then sort the remaining neighbors. (click)

Next, we get the median of these neighbors. (click)

Finally, subtract the origin pixel with its median

neighbor to get an error value. (click)

Slide 9 of 14


10/14

We compute errors for 3 neighborhood sizes, and all

3 color channels all separately. That's 9 different

groups of data that are all handled separately. For

each group of data, we compute the mean, standard

deviation, skewness, kurtosis, energy and entropy of

the pixel errors.

(Show what each statistics moment is)

Slide 10 of 14


11/14

So from what I just described, the attributes come

from 4 statistics moments, energy and entropy for 3

colors and 3 neighborhood sizes for both edge and

smooth pixel types. That plus 18 unrelated

attributes, that due to time constraints and since

they're unrelated to the demosaicing process I won't

include in this talk, adds up to 126 total attributes.

The classifier uses these attributes to create a

fingerprint for each camera. The attributes will also

fit into our collaborator's larger system, which

includes data about noise in images.

Slide 11 of 14


12/14

For all 25 cameras, our original classifier used smooth

errors attributes to trace images to their cameras with

34.96% accuracy. When we added attributes for edge

errors, our accuracy rate rose to 36.72%. So, processing

edge pixels differently and adding the 18 other attributes

produced only marginally better results. We created an

additional classifier for a subset of 3 iPhones to see if

the attributes can distinguish between different instances

of the same make and model of cameras. This

classifier's accuracy was 79.1%, so it seems that these

can differentiate between the same camera models.

Slide 12 of 14


13/14

The attributes we designed for the classifier clearly are

capturing some good information, as our classifier beats

random choice easily. But they don't do well enough on

their own. We plan to incorporate our attributes along

with our colleagues attribute sets to improve overall

accuracy rates. We also plan to extract the top N camera

choices for each test image in our classifier. Hopefully,

we'll be able to predict with 99% accuracy the correct

top N camera choices for an image. And we plan to

implement a few common demosaicing processes that

cameras use and compute attributes from these to try to

increase our classifier's accuracy.

Slide 13 of 14


14/14

Please feel free to contact me

by visiting my website at

amsteinberger.com. Now, I'd

like to open the floor for

questions.

Slide 14 of 14

power point presentation notes

Documents