1

Resampling Detection for Digital Image ForensicsJohn Ho, Derek Ma, and Justin Meyer

Abstract—A virtually unavoidable consequence of ma-nipulations on digital images are statistical correlationsintroduced between the pixels. These correlations may notbe visible to a human, but can be detected by statisticaltechniques. This paper presents a machine learning basedapproach to image resampling detection based on thedetector by Popescu and Farid. We investigate waysto improve robustness and detection accuracy by usingsupervised learning techniques.

I. INTRODUCTION

With the proliferation of digital images and pow-erful image editing tools such as Photoshop, it hasbecome increasingly easy to manipulate images toalter content and meaning. Digital image forensicsseeks to authenticate image based on statisticalpatterns left on an image by tampering. In [1],Popescu and Farid introduced a forgery detector bybased on the correlations introduced between pixelsby resampling; that is, the operation of stretching,shrinking, or rotating an image. Since objects inimages are often on different scales, resampling isnecessary to create a visually convincing forgery.However, resampling imposes periodic correlationsbetween pixels that otherwise do not exist. Thesecorrelations can estimated by a learning algorithm.

In order to determine if the image has beenresampled, the found correlations must be passedto a classifier. The classifier by Popescu and Faridcharacterized resampling by constructing a databaseof synthetic maps for different resampling ratios.This has the advantage of being conceptually sim-ple, but requires an exhaustive database and does nottake into account the fact that natural variations canbe learned. In this paper, we show that a supervisedclassifier can improve the detector’s accuracy andallow it to work in a much more general setting.

The paper is organized as follows. Section IIdescribes the basis of correlations introduced byresampling and the EM procedure used to determinethe correlation between pixels. Section III presents

Submitted November 17, 2011 for CS229 Machine Learning, Prof.Andrew Ng, Stanford University.

a supervised learning approach to classification. Fi-nally, Section IV presents the results of our classifierand compares it with the performance by Popescuand Farid’s classifer.

II. DETECTING RESAMPLING CORRELATIONS

A. Resampling

Resampling causes certain pixels be a linear com-bination of its neighbors. These pixels are correlatedwith its neighbors and will appear periodically inthe resampled image. For each pixel, we define itsneighbors to be all pixels within a window of length2N + 1.

Resampling by a factor p/q can be representedby the linear equation ~y = Ap/q~x, where p and qare integers, ~y the resampled signal, ~x the originalsignal, and Ap/q ∈ Rn×m the resampling matrix.

For example, consider the case of p/q = 2. Theresampling matrix is given by

A2/1 =

1 0 00.5 0.5 00 1 0 . . .0 0.5 0.50 0 1

... . . .

.

It can be seen that the odd samples do not changewhile the even samples are linearly dependent ontheir neighbors. We have

y2i−1 = xi

y2i = 0.5xi + 0.5xi+1

for i = 1, . . . ,m. This implies that

y2i−1 = 0.5y2i−1 + 0.5y2i+1, (1)

so, for this case, every even sample is exactly thesame linear combination of its neighbors.

2

0%

5%

10%

15%

0%

5%

10%

15%

0o

5o

10o

15o

(a) (b) (c)

Fig. 1. Image, probability map, and periodicity map for (a) upsampling, (b) downsampling, and (c) rotation. The classifier must be able todistinguish between natural peaks in the image and those introduced by resampling.

To generalize this to a signal resampled by anarbitrary p/q it is necessary to determine when

yi =N∑

k=−N

αkyi+k (2)

where αk is the set of weights across the neighbor-hood.

Let ~ai be the ith row of the resampling matrixAp/q. It can be shown that (2) holds when

~ai =N∑

k=−N

αk~ai+k. (3)

That is, a resampled pixel yi is correlated withits neighbors whenever its corresponding row ofthe resampling matrix ai can be written as a linearcombination of its neighboring rows [1][2]. Thiswill lead to periodic correlations between resampledpixels since the resampling matrix is periodic.

B. Expectation-MaximizationIf the resampling matrix Ap/q is known, the

problem is simply that of finding a set of weights ~αthat satisfies (3) for a subset of rows. However, inpractice, the sampling factor p/q and interpolationmethod are not known. Popescu and Farid de-scribe an expectation-maximization (EM) algorithmto learn the weights ~α.

Let M1 be the set of samples that are corre-lated to their neighbors and M2 the set of samplesnot correlated to their neighbors. EM is used tosimultaneously estimate ~α and a soft assignmentPr{yi ∈ M1} for each pixel yi since neither isknown. The output of the EM algorithm is a setof coefficients ~α and a probability map P . Thecomplete algorithm is given in Appendix A.

For faster computation of the probability map,the EM algorithm can be applied to only a randomsubset of the blocks in the image to train the weights~α. Once the weights are weights are found, P canbe quickly computed everywhere. We found that thisdoes not significantly change the probability maps.Random block sampling was used throughout ourexperiments.

C. PeriodicityNeighboring pixels can be naturally correlated

based on the statistics of the natural image. How-ever, it is unlikely that these correlations are peri-odic. To detect periodicity, Fourier transform of theprobability map is taken. We refer to the magni-tude of the transform as the periodicity map. Theperiodicity map is typically high-pass filtered inorder to suppress the natural DC peak. In [1], apeak detection algorithm is also described in orderto enhance the peaks and suppress energy due tonatural correlations in the image.

3

III. SUPERVISED LEARNING

Given a periodicity map, we want to determine ifthe image has been resampled or non-resampled. In[1], an exhaustive database of synthetic maps wasgenerated for various resampling ratios. A periodic-ity map was then classified by k-nearest neighbors(kNN) with k = 1 for a similarity score. While thisyields reasonable accuracy, the kNN classifier doesnot work in a general setting with arbitrary blocksizes and resampling ratios.

We train a support vector machine (SVM) toclassify a periodicity map as resampled or non-resampled. Figure 1 shows the probability and pe-riodicity map for different images and resamplingratios. There are distinct peaks that appear when animage has been resampled. The classifier must beable to distinguish between natural peaks in the pe-riodicity map from those introduced by resampling.

We extract the following 5 features from theperiodicity map:

1) n largest coefficients. Higher peaks tend toindicate resampling. The coefficients ci aresorted by magnitude and aggregated to a sin-gle feature

f1 =n∑

i=1

m√|ci|. (4)

2) n largest coefficients after peak detection.The peak detection algorithm used in [1] isapplied to the periodicity map. The sortedcoefficients c′i are aggregated by

f2 =n∑

i=1

m√|c′i|. (5)

3) n largest coefficient distance from center.Peaks in natural images tend to be con-centrated in low frequencies. The positions(ui, vi) are used to compute

f3 =n∑

i=1

√u2i + v2i . (6)

4) n largest coefficient to local energy ratio.The ratios ri are computed for a local rect-angular window of width W and aggregatedby

f4 =

(n∑

i=1

100 ri

)m

/100. (7)

TABLE ITRAINING RESAMPLING RATES.

Start Step Size End

Upsampling ratio 1.05 0.03 1.5Downsampling ratio 0.5 0.03 0.95Rotation (◦) 1 1 45

5) Peak to total energy ratio. The ratio R wasscaled by

f5 = (104R)m/102. (8)

Since all the features are non-negative, they arescaled to [0, 1]. We selected n = 4 coefficients anda scaling factor of m = 2. LIBSVM was used withthe radial basis kernel [3].

A. Experiment SetupFor the test images, we use the uncompressed

Kodak PhotoCD set of 24 images. These imagesare not compressed and are true color; that is, freeof demosaicking, which can also introduce linearcorrelations [4]. For simplicity, we extract the greencolor channel from the image to obtain a grayscaletest image. We used a 128×128 window size, whichis significantly smaller than 512× 512 used in [1].Smaller block sizes can better localize tampering,but result in peaks that are less distinct.

For each resampling operation, we first randomlyselect an image on which to apply resamplingand then randomly select a window within theresampled image. The resampling ratios used intraining are shown in Table I. In order to obtain non-resampled samples, the images are divided into non-overlapping blocks. We used 80% of the images fortraining and 20% for testing. Since false positivesare highly undesirable in a forensic setting, theparameters of the classifier are set to have no falsepositives over the training set.

We also implemented the kNN classifier in [1]for comparison. The database consists of syntheticmaps using the same resampling ratios in Table I.

IV. RESULTS

A. Classification ResultsThe overall recall and precision rates for our clas-

sifier are shown in Table II. There is only one falsepositive (< 1%) over the test set. The performanceof the SVM classifier is compared to the kNN

4

0 10 20 30 40 50 60 70 800

20

40

60

80

100

Upsampl ing [%]

%

0 5 10 15 20 25 30 35 40 450

20

40

60

80

100

Downsampl ing [%]

%

0 5 10 15 20 25 30 35 40 450

20

40

60

80

100

Rotation [◦ ]

%

Fig. 2. Recall rate for the SVM classifier (black solid) and thekNN (dashed gray) classifier. The false positive rate for both casesare < 1%.

−0.5 0 0.5 1

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

γ1

γ2

Non−resampled

Upsampled

Downsampled

Rotated

Fig. 3. Training set features projected onto the first two principalcomponents γ1 and γ2. The upsampled and rotated data is clearlyseparable from the non-resampled data even in low dimension, butdownsampled data is difficult to separate.

TABLE IIRECALL AND PRECISION RATE FOR SVM CLASSIFIER.

Non-resampled Resampled

Recall 119/120 369/394(%) 99.16% 93.66%

Precision 119/144 369/370(%) 82.639% 99.730%

classifier over a wide range of resampling ratios inFigure 2. The threshold of the synthetic classifierwas set to have a similar < 1% false positiverate. Our classifier clearly outperforms kNN at lowresampling rates, which compared to results in [1]suffers from the smaller block. It has comparableaccuracy to kNN at large resampling ratios and candetect some highly downsampled images where thekNN classifier has zero recall.

Figure 3 shows the training set features projectedonto the first two principal components. The up-sampled and rotated data are clearly separable fromthe non-resampled data, but the down-sampling dataappears to be clustered. This is due to fact thatdownsampling peaks in the periodicity map areindistinct and can be buried in the natural imagecorrelations. Interestingly, it also appears that up-sampled and rotated images can be distinguishedfrom one another.

B. Detection Example

Figure 4 shows how the SVM classifier could beused to perform tamper detection in a more generalsetting. A beach scene contains an image of Prof.Andrew Ng of Stanford University that has beenupsampled and inserted into the foreground. Severalregions in the image are selected over which theperiodicity maps are computed. The features areextracted from the periodicity maps and passed toour classifier, which determines if the region hasbeen resampled. The classifier successfully distin-guishes the resampled and non-resampled regions.If not computationally prohibitive, a sliding windowcould be used to perform detection at every locationin the image to enable automatic detection withoutuser input.

V. CONCLUSION

In this project, the Popescu and Farid’s EMalgorithm for learning correlations between pixels

5

(a) (b)

Fig. 4. (a) Tampered image and (b) probability map and periodicity map for four blocks. The classifier correctly determines that the redblock has been resampled and the remaining blocks are not resampled.

was implemented and tested for a smaller blocksize. An SVM classifier was trained to determine ifthe correlations found by the EM algorithm resultfrom resampling. This classifier was shown to havebetter performance than the kNN classifier at lowresampling rates and does not require and exhaustivedatabase of synthetic maps.

APPENDIX AThe expectation step consists of assigning

Pr{yi ∈M1|yi} =Pr{yi|yi ∈M1}Pr{yi ∈M1}∑2

k=1 Pr{yi|yi ∈Mk}(9)

where Pr{yi ∈M1} = Pr{yi ∈M2} = 1/2.The model M1 is assumed be normally distributed

Pr{yi|yi ∈M1} =

1

σ√2π

exp

(−(yi −

∑Nk=−N ~αkyi+k)

2

2σ2

)(10)

and M2 uniformly distributed Pr{yi|yi ∈ M2} =1/N where N is the range of the pixel values.

The maximization step reduces to finding theparameters ~α by weighted least-squares

~α = (Y TWY )−1Y TW~y (11)

where the rows of Y are the blocks in the imagewith the center value removed and W is a diagonal

matrix with diagonal elements equal to Pr{yi|yi ∈M1}.

The EM steps are iterated until ||~α(i)−~α(i−1)|| < εwhere ~α(i) is the set of weights obtained from theith iteration.

REFERENCES

[1] A. Popescu and H. Farid, “Exposing digital forgeries by detectingtraces of resampling,” Signal Processing, IEEE Transactions on,vol. 53, no. 2, pp. 758 – 767, feb. 2005.

[2] M. Kirchner, “Fast and reliable resampling detection by spectralanalysis of fixed linear predictor residue,” in Proceedings of the10th ACM Workshop on Multimedia and Security, ser. Sec ’08.New York, NY, USA: ACM, 2008, pp. 11–20.

[3] C.-C. Chang and C.-J. Lin, “LIBSVM: A library for supportvector machines,” ACM Transactions on Intelligent Systems andTechnology, vol. 2, pp. 27:1–27:27, 2011, software available athttp://www.csie.ntu.edu.tw/ cjlin/libsvm.

[4] A. Popescu and H. Farid, “Exposing digital forgeries in colorfilter array interpolated images,” Signal Processing, IEEE Trans-actions on, vol. 53, no. 10, pp. 3948 – 3959, oct. 2005.