blind estimation of tone mapping transform

10
1 Blind Estimation of Tone Mapping Transform Honggang Qi, Xing Zhang, Longyin Wen, and Siwei Lyu Computer Science Department University at Albany, State University of New York Albany, NY, 12222 USA Abstract—Tone mappings (TM), examples of which include gamma correction, sigmoid stretching and histogram equal- ization, are common operations on digital images, and it is practically useful to estimate such transforms directly from an image. In this work, we describe an effective and efficient method to estimate TM from images, which takes advantage of the nature of TM as a mapping between integral pixel values and the distinct characteristics it introduces to the pixel value (PV) histograms of the transformed image. Our method recovers the original PV histogram and the TM simultaneously with an efficient iterative algorithm, and can effectively handle perturbations due to noise and compression. We perform experimental evaluation to demonstrate the efficacy and efficiency of the proposed method, as well as its robustness in the presence of additive noise and under JPEG compressions. By examining local areas in the image, we also show that using this method, we can detect spliced image regions that have undergone different TMs. Index Terms—tone mapping, pixel histograms, blind estima- tion. I. I NTRODUCTION The integral pixel values in a digital image do not directly reflect the amount of light projected on the camera sensor at the time of capture. Such is mainly due to the two mappings in the imaging process. First, inside a camera, the continuous scene radiance x is mapped to raw integral pixel value j = ψ(x) with the camera response function (CRF) [4]. Subsequently the raw pixel values may further undergo a tone mapping (TM), which can occur either inside the camera or after the image is captured and processed in a computer, to become the final integral pixel value i = φ(j ). Examples of TM include gamma correction, sigmoid stretching and histogram equalization [7]. Note that CRF and TM have very different natures, in particular, CRF is a real-to-integer mapping, while TM is an integer-to-integer mapping. While estimating CRF (a problem known as radiometric calibration)[4], [11], [12] is a well-studied subject in image processing and computer vision and has led to a vast literature, the problem of recovering TM from a single image has not gain much attention so far. From the practical point of view, estimation of TM from a single image is useful for two main reasons. First, it can help to correct nonlinearity applied to in- tegral pixel values, and thus facilitates radiometric calibration and other image processing and computer vision applications, such as image registration, matching and retrieval, that predi- cate on raw pixel values. Second, in forensic analysis of digital images [15], recovering TM can be used to reconstruct the processing history of an image in establishing its authenticity, or to differentiate regions from different source images in detecting tampering. As CRF and TM have very different nature and functions applying existing radiometric calibration methods on images may only recover the concatenation of CRF and TM. To recover TM itself from a single image, we need different algorithms. In this work, we describe an effective and efficient method for the blind estimation of TM from a single image. This method takes advantage of the special nature of TM as a mapping between integral pixel values that introduces distinct characteristics to the pixel value (PV) histograms of the transformed image. It recovers the original PV histogram and the TM simultaneously with an efficient iterative algorithm. Our method recovers the original PV histogram and the TM simultaneously with an efficient iterative algorithm, and can effectively handle perturbations due to noise and compression. We perform experimental evaluation to demonstrate the effi- cacy and efficiency of the proposed method, as well as its robustness in the presence of additive noise and under JPEG compressions. By examining local areas in the image, we also show that using this method, we can detect spliced image regions that have undergone different TMs. The rest of this paper is organized as following. In Section II, we review relevant previous works, and then in Section III, the basic definition of TM and PV are reviewed. Section IV describes our algorithm for blind TM estimation in detail. Section V reports experimental results of our algorithm, and Section VI describes an application of our algorithm in digital image forensics to detect spliced regions that undergo different TMs. Section VII concludes the article with discussion and future works. Preliminary version of this work is presented in [21] II. RELATED WORKS There have been only a few previous works addressing this problem. As the most common type of TM, several previous works have focused on the blind estimation of gamma correction. For instance, the method in [5] uses higher order statistical dependencies introduced by gamma. The method of [2] uses the the features developed in [17] to recover the actual gamma value by applying different gamma values to a uniform histogram and identifying the optimal value that best matches the observed PV histogram features. Using a set of features characterizing the shapes of PV histogram after a TM, the work in [17] develops method to determine if an image has undergone gamma correction or histogram equalization, but this method cannot recover the actual transform. The

Upload: others

Post on 07-Apr-2022

4 views

Category:

Documents


0 download

TRANSCRIPT

1

Blind Estimation of Tone Mapping TransformHonggang Qi, Xing Zhang, Longyin Wen, and Siwei Lyu

Computer Science DepartmentUniversity at Albany, State University of New York

Albany, NY, 12222 USA

Abstract—Tone mappings (TM), examples of which includegamma correction, sigmoid stretching and histogram equal-ization, are common operations on digital images, and it ispractically useful to estimate such transforms directly from animage. In this work, we describe an effective and efficient methodto estimate TM from images, which takes advantage of the natureof TM as a mapping between integral pixel values and the distinctcharacteristics it introduces to the pixel value (PV) histogramsof the transformed image. Our method recovers the originalPV histogram and the TM simultaneously with an efficientiterative algorithm, and can effectively handle perturbations dueto noise and compression. We perform experimental evaluation todemonstrate the efficacy and efficiency of the proposed method,as well as its robustness in the presence of additive noise andunder JPEG compressions. By examining local areas in the image,we also show that using this method, we can detect spliced imageregions that have undergone different TMs.

Index Terms—tone mapping, pixel histograms, blind estima-tion.

I. INTRODUCTION

The integral pixel values in a digital image do not directlyreflect the amount of light projected on the camera sensorat the time of capture. Such is mainly due to the twomappings in the imaging process. First, inside a camera, thecontinuous scene radiance x is mapped to raw integral pixelvalue j = ψ(x) with the camera response function (CRF)[4]. Subsequently the raw pixel values may further undergo atone mapping (TM), which can occur either inside the cameraor after the image is captured and processed in a computer,to become the final integral pixel value i = φ(j). Examplesof TM include gamma correction, sigmoid stretching andhistogram equalization [7]. Note that CRF and TM havevery different natures, in particular, CRF is a real-to-integermapping, while TM is an integer-to-integer mapping.

While estimating CRF (a problem known as radiometriccalibration) [4], [11], [12] is a well-studied subject in imageprocessing and computer vision and has led to a vast literature,the problem of recovering TM from a single image has notgain much attention so far. From the practical point of view,estimation of TM from a single image is useful for two mainreasons. First, it can help to correct nonlinearity applied to in-tegral pixel values, and thus facilitates radiometric calibrationand other image processing and computer vision applications,such as image registration, matching and retrieval, that predi-cate on raw pixel values. Second, in forensic analysis of digitalimages [15], recovering TM can be used to reconstruct theprocessing history of an image in establishing its authenticity,or to differentiate regions from different source images in

detecting tampering. As CRF and TM have very differentnature and functions applying existing radiometric calibrationmethods on images may only recover the concatenation ofCRF and TM. To recover TM itself from a single image, weneed different algorithms.

In this work, we describe an effective and efficient methodfor the blind estimation of TM from a single image. Thismethod takes advantage of the special nature of TM as amapping between integral pixel values that introduces distinctcharacteristics to the pixel value (PV) histograms of thetransformed image. It recovers the original PV histogram andthe TM simultaneously with an efficient iterative algorithm.Our method recovers the original PV histogram and the TMsimultaneously with an efficient iterative algorithm, and caneffectively handle perturbations due to noise and compression.We perform experimental evaluation to demonstrate the effi-cacy and efficiency of the proposed method, as well as itsrobustness in the presence of additive noise and under JPEGcompressions. By examining local areas in the image, we alsoshow that using this method, we can detect spliced imageregions that have undergone different TMs.

The rest of this paper is organized as following. In SectionII, we review relevant previous works, and then in SectionIII, the basic definition of TM and PV are reviewed. SectionIV describes our algorithm for blind TM estimation in detail.Section V reports experimental results of our algorithm, andSection VI describes an application of our algorithm in digitalimage forensics to detect spliced regions that undergo differentTMs. Section VII concludes the article with discussion andfuture works. Preliminary version of this work is presented in[21]

II. RELATED WORKS

There have been only a few previous works addressingthis problem. As the most common type of TM, severalprevious works have focused on the blind estimation of gammacorrection. For instance, the method in [5] uses higher orderstatistical dependencies introduced by gamma. The methodof [2] uses the the features developed in [17] to recover theactual gamma value by applying different gamma values to auniform histogram and identifying the optimal value that bestmatches the observed PV histogram features. Using a set offeatures characterizing the shapes of PV histogram after a TM,the work in [17] develops method to determine if an imagehas undergone gamma correction or histogram equalization,but this method cannot recover the actual transform. The

2

work [16] describes an iterative algorithm to jointly estimatea nonparametric TM and the PV histogram of the originalimage, based on a probabilistic model of PV histogram and anexhaustive matching procedure to determine which histogramentries are most likely to correspond to artifacts caused by theTM.

Compared with the previous works, the main contributionsof this work can be summarized as follows:

1) our method estimates both TMs with parametric forms(e.g., gamma correction or sigmoid stretching) and de-fined non-parametrically (e.g., histogram equalization orfree-hand transform);

2) in the noise-free case, our method achieves near per-fect estimation for several types of parametric or non-parametric TMs;

3) in contrast to previous works, our method can effectivelyhandle perturbations such as additive noise and JPEGcompression.

We show experimental results demonstrating the efficacy andefficiency of our method and its robustness in the presence ofadditive noise and under JPEG compressions.

III. TM AND ITS EFFECT ON PV HISTOGRAM

For simplicity, we focus our discussion on gray-scale imageswhose pixel values are encoded with b-bits that correspondto N = 2b different values1. Most TMs can be modeled asa continuous mapping followed by a rounding operation thatmaps real value back to integers2. Specifically, each input pixelvalue, j ∈ {0, · · · , N − 1}, is mapped to its correspondingoutput pixel value, i ∈ {0, · · · , N − 1}, as

i = φ(j) := round[m(j)],

where m(·) : [0, N−1] 7→ [0, N−1] is a continuous transform.Furthermore, most forms of m(·) of practical interest ismonotonic, which we will assume subsequently. Examples ofm(·) include• gamma correction,

mγ(j) = N

(j

N

)γ, (1)

• sigmoid stretching

mα,β(j) = N

(1 + exp

(β − j

N

α

))−1, (2)

• histogram equalization,

m(j) = ρ−1(j

N

), (3)

where ρ−1(·) is the interpolated inverse cumulative den-sity function of the pixels of the input image,

• free-form intensity transform, such as the curves toolin Adobe Photoshop.

1For RGB color images, we can change apply our algorithm to each colorchannel independently.

2Other operations recovering integer values from continuous values suchas floor or ceiling can also be used.

The first two are examples of parametric TM and the lattertwo are nonparametric TMs. It should also be pointed out thateven though m(·) is invertible, the overall TM in general maynot be invertible, which is mainly due to the rounding stepthat is not a one-to-one mapping.

Because of its point-wise nature, we can understand theeffect of TM on an image by examining the pixel value (PV)histogram. In this work, we normalize the PV histogram tohave a unit total sum, so it can be treated as a probabilitydistribution over pixel values, and the ith bin of the PV isdenoted as P(i). With these notations, we can rewrite theeffect of TM transform on an image as:

Poutput(i) =

N−1∑j=0

δ (i = round[m(j)]) · Pinput(j), (4)

where the TM transform is regarded as a conditional proba-bility distribution.

Furthermore, we can represent the normalized PV his-tograms of the input and output image as vectors on theN -dimensional probability simplex SN 3, such that xj =Pinput(j− 1) and yi = Poutput(i− 1), and introduce an N ×Nmatrix H that is defined as

Hij = δ (i = round[m(j − 1)] + 1) ,

for i, j ∈ {1, · · · , N}, then Eq.(4) can be rewritten as yi =∑Nj=1Hijxj , or in the more compact matrix and vector form,

as:q = Hp. (5)

Because of the monotonicity of the TM transform, for TMthat is not identity, there will be multiple input pixel valuesmap to a single output pixel value, and output pixel valuesto which no input pixel value maps based on the “pigeonholeprinciple” [9]. This redistribution of pixel values creates peaksand gaps in the shape of the output PV histogram, whichis illustrated in two cases (gamma correction and histogramequalization) in the second and third rows of Fig.1. In contrast,the PV histogram of typical photographic image not undergoneTM is more smooth, see the first row Fig.1. The difference insmoothness of these PV histograms can be better revealed withthe first order difference of the values of neighboring bins, asshown in the last column of Fig.1. Note that when measuredwith the `1 norm, the first order difference of neighboringcomponents for the PV histogram of the original image isconsiderably smaller than those of images undergone TMs4.This leads to the following assumption on which subsequentwork is based:

Assumption 1: Photographic images that have not undergoneTM transforms have smooth PV histograms and if TM trans-form is applied to them, their `1 total variation will increase.

We should point out that this assumption applies to photo-graphic images with wider dynamic ranges, and may not hold

3An N -dimensional probability simplex SN is a set of RN , such thatp ∈ SN means that xi ≥ 0 and 1Tp = 1.

4Similar regularities of PV histograms have also been observed on a widerange of other photographic images.

3

0 50 100 150 200 250input

0

50

100

150

200

250

outp

ut

0 50 100 150 200 2500.000

0.005

0.010

0.015

0.020

0 50 100 150 200 2500.015

0.010

0.005

0.000

0.005

0.010

0.015

`1 norm =0.057919963307

0 50 100 150 200 250input

0

50

100

150

200

250outp

ut

0 50 100 150 200 2500.000

0.005

0.010

0.015

0.020

0 50 100 150 200 2500.015

0.010

0.005

0.000

0.005

0.010

0.015

`1 norm =0.639385391584

0 50 100 150 200 250input

0

50

100

150

200

250

outp

ut

0 50 100 150 200 2500.000

0.005

0.010

0.015

0.020

0 50 100 150 200 2500.015

0.010

0.005

0.000

0.005

0.010

0.015

`1 norm =1.13748734203

image Tone Mapping PV histogram first order difference of bins

Fig. 1. Effects of tone mapping (TM) on a photographic image and its pixel value (PV) histogram.

for specific classes of images that have discrete values (e.g.,text images or pattern images). Subsequently, our discussionis focused on image classes that satisfy Assumption 1.

IV. ESTIMATING TM FROM IMAGE

The central problem in TM estimation is to recover thetransform and the vectorized PV histogram p of the inputimage using only the vectorized PV histogram of an imageundergone TM, q. More specifically, this is to solve for H andp simultaneously in Eq.(5) with known q. In Section IV-A, wefirst describe a solution to the simpler problem of estimatingp from q when H is known. This is then used as the basis fora solution to the more general problem when H is not knownin Section IV-C. In Section IV-B,

A. Recovering Original PV Histogram

The problem of recovering p from q and H is oftentimesill-posed, this is because unless matrix H is identity, it containsrows of all zeros corresponding to output pixel values towhich no input pixel value maps. As such, H is always rankdeficient, which can be severe as shown in Fig.2 for the gammacorrection (Eq.(1)) with different γ values.

To obtain a stable solution, we need to enforce moreconstraints on the values of p. As results in Fig.1 suggest,the PV histogram of the original image tends to be smooth

0.1 0.4 0.7 1.0 1.3 1.6 1.9 2.2 2.5 2.8γ value

80

102

124

146

168

190

212

234

256

rank

of

matr

ix H

Fig. 2. The rank of matrix H for N = 256 corresponding to differentγ value in gamma correction.

compared to those of the transformed images, which arereflected with a much smaller `1 norm of the differences ofneighboring components. Thus, we formulate the problem ofestimating p as an `1 constrained least squares problem – weseek the optimal p that leads to “similar” PV histogram to qmeasured by the `2 difference, and has small `1 norm for thefirst order differences of neighboring components. Formally,this is expressed with the following constrained optimizationproblem:

minp∈SN

1

2‖q−Hp‖22 + λ‖Dp‖1 (6)

where λ ≥ 0 is a system parameter that controls the tradeoffbetween the `2 and `1 losses. Matrix D corresponds to the

4

first order discrete difference operator that is defined as:

Dij =

−1 i = j+1 i = j + 10 otherwise

.

We will refer to the optimal value of the objective function of(6) as L?(H,λ). The optimization problem in (6) is convex[1] and can thus be solved with general-purpose convexprogramming package such as CVX [3], [8]. However, thespecial structure of the constraint can be exploited to achievesimpler and more efficient solutions.

Specifically, we use a modified version of the half-quadraticminimization method originally designed for total variationimage restoration [20]. To this end, we first introduce twoauxiliary variables z and v and modify (6) to the followingoptimization problem,

minp∈SN ,v>0,z

1

2‖q−Hp‖22 + λρ(Dp, z,v) (7)

where

ρ(Dp, z,v) = ‖z‖1+1

21Tv+

1

2(Dp−z)TD(v)−1(Dp−z).

Here D(v) denotes the the diagonal matrix with the diagonalelements given by v. We can prove (in Section A) that

‖Dp‖1 = minv>0,z

ρ(Dp, z,v). (8)

Minimizing (7) w.r.t. p will thus yield the optimal solution tothe original problem of (6).

Subsequently, we optimize (7) by block coordinate descent,with iteratively minimizing each of the unknown variables,i.e., p, z and v until convergence.• Solving for p: we first simplify Eq.(7) by dropping

irrelevant terms to obtain an objective function of p,which becomes

minp∈SN12p

T(HTH +DTD(v)−1D

)p

−(HTq+ D(v)−1Dz

)Tp

This is a quadratic programming problem with positivedefinite matrix and simplex constraint, and we can solveit efficiently with the Frank-Wolfe algorithm [10], whichcan take advantage of the sparse structure of matrixHTH +DTD(v)−1D. Specifically, the Frank-Wolfe al-gorithm for this problem is an iterative algorithm. Startingwith an initial value p0, at each step t, the algorithm firstsolve the following linear programming problem:

mins∈SN

sTb

where b =(HTH +DTD(v)−1D

)pt −(

HTq+ D(v)−1Dz). This problem can be solved

efficiently using LP solvers, further taking advantagethat the structure of matrices H and D makes matrixvector multiplication. The Frank-Wolfe algorithm thenproceeds with the following updating rule [10]:

pt+1 = (1− γ)pt + γs, for γ =2

t+ 2.

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

Fig. 3. Recovery of the PV histogram of an 8-bit grayscale image thathas been gamma corrected with γ = 0.4 (the corresponding matrixH has a rank of 176 out of a full rank of 256). (left) original PVhistogram. (middle) observed PV histogram after gamma correction.(right) recovered PV histogram.

The Frank-Wolfe algorithm usually converges in a fewsteps [10] and has better numerical characteristics thansimple projected gradient descent method and easier toimplement than the interior point method.

• Solving for z: Similarly, we drop terms irrelevant to zfrom (7), and obtain the following unconstrained opti-mization problem for z,

minz

N∑i=1

(((Dp)i − zi)2

2vi+ |zi|

),

which is separated into N scalar optimization problemsfor each zi. These scalar optimization problems can besolved individually, the optimal solution of which is zi =φvi((Dp)i), where

φλ(x) = max(x− λ, 0) + min(x+ λ, 0)

is the soft-thresholding function [20].• Updating v: With irrelevant terms from (7) dropped,

optimizing v reduces to the following problem,

minv>0

∑Ni=1

(((Dp)i−zi)2

2vi+ vi

2

). (9)

The solution is given by vi = |(Dp)i − zi| [13].Furthermore, To avoid division of zero, following thesuggestion in [13], a modified update step is used, as:

vi = |(Dp)i − zi|+ ε, (10)

where ε > 0 is stabilizing constant.Initializing p = z = q, we iteratively solve for v, z andp by alternating the steps given previously, in using “warmstart”, i.e., initializing the new optimization steps with thesolution from the last round of optimization. We found that thisalgorithm usually converges within 4-5 iterations of the outerloop and less than 5 iterations of the inner loop in practice.Shown in Fig.3 is an example of the recovery of the PVhistogram of an 8-bit grayscale image undergone a gammacorrection with γ = 0.4 (the corresponding matrix H has arank of 176 out of a full rank of 256). As the result shows,the recovered PV histogram is very close to the original PVhistogram, and the presence of small flat regions are the resultsof the smoothness penalty.

B. Handling Noise

TM transformed image may also be subject to perturbationsthat can be modeled as additive random noises. The effect of

5

noise on the PV histograms of images with TM is shown inFig 4 – the smoothing effect of noise conceals the peaks andgaps signatures. This is the main reason that previous worksrecovering TM [5], [2], [16] usually do not work well withthe presence of noise.

As the perturbed pixel values are rounded back to integers,we model the overall process by treating pixel values asrandom samples from their corresponding PV histograms, as:

i = round(k + z) = round(φ(j) + z),

where j, k and i correspond to the original, TM transformedand final pixel values, with z being the noise added to eachpixel. This relation can be more clearly described using PVhistograms as probabilistic models for pixel values, as:

Poutput(i) =

N−1∑k=0

∫z

p(z)δ (i = round(z + k)) dz︸ ︷︷ ︸additive noise process

×N−1∑j=0

δ (k = round[m(j)])Pinput(j)︸ ︷︷ ︸Tone Mapping

, (11)

where p(z) is the probability density function of randomvariable z corresponding to the noise process. Similar to thecase of Eq.(4), we can rewrite Eq.(11) using vectorized PVhistograms and matrices as

q = RHp, (12)

where R corresponds to the matrix induced from the noisemodel and is defined as:

Rik =

∫z

δ (i = round(z + k)) p(z)dz =

∫ i−k+ 12

i−k− 12

p(z)dz

= Pz(i− k + 1

2

)− Pz

(i− k − 1

2

),

denoting Pz(z) as the cumulative density function of z. Whenthe noise process can be modeled with a white Gaussiannoise model with zero mean and standard deviation σ5, eachcomponent of R can be computed as:

Rik = erf

(i− k + 1

2

σ

)− erf

(i− k − 1

2

σ

)(13)

where

erf(u) =

∫ u

−∞

1√2πe−

τ2

2 dτ

is the cumulative density function of the normal distribution.Matrix R is symmetric, Toeplitz – as all parallel diagonalelements (Rik with constant i−k) are the same – and positivedefinite, as it is diagonal dominant with the off-diagonal valuesdecay rapidly with small σ. As such, very efficient numericalprocedures exist to compute the inversion R such as theLevinson-Durbin algorithm [6].

5In practice, σ might be estimated from the noisy image with several recentblind noise estimation methods [22].

0 50 100 150 200 2500.000

0.005

0.010

0.015

0.020

0 50 100 150 200 2500.000

0.005

0.010

0.015

0.020

Fig. 4. Effect of additive noise. (left) PV histogram of a photographicimage transformed with gamma correction, which exhibits the distinctgaps and peaks. (right) PV histogram of the same gamma correctedimage with rounded additive white Gaussian noise with zero meanand σ = 1, of which the gaps and peaks in the left PV histogramlargely disappear.

When there is image noise, the estimation of p has toincorporate the noise matrix R. We have two options: the firstis to form a new optimization problem based on Eq.(12), as:

minp12‖q−RHp‖22 + λ‖Dp‖1

s.t. 1Tp = 1,p ≥ 0,(14)

which can be solved similarly using the method described inSection IV. The second option is to directly recover q̃, which isthe PV histogram before the noise is inserted. This is achievedby using a special version of the Levinson-Durbin algorithm[6] to directly solve the linear Toeplitz equation system Rq̃ =q. We then use q̃ in the optimization of (6) in place of q.Subsequently, we take the first approach when there is no needto recover q̃ explicitly, as in the case of recovering a parametricTM (Section V-A), and the second when it is required as inthe case of recovering nonparametric TM (Section V-B).

C. Recovering tone mapping

We now turn to the more difficult problem of estimatingthe original PV histogram p and transformation matrix Hsimultaneously. In the following, we discuss two cases:

1) parametric TM that can be specified with a small numberof parameters and,

2) nonparametric TM as a mapping for every possible inputin {0, · · · , N − 1}.

1) Parametric Tone Mapping: In the case of parametric TMtransforms, e.g., gamma correction and sigmoid stretching asgiven in Eq.(1) and Eq.(2), our observations in Fig. 1 suggeststhat the true set of parameters will lead to a minimum toboth terms of Eq.(6): the transformed PV histogram will beexactly the same as the observed histogram, thus the firstterm will reach minimum (zero), while the original histogramshould have the maximum degree of smoothness reflected bya minimal `1 total variation. This intuition leads to followingassumption.

Assumption 2: The true transform parameter corresponds toa global minimum of the objective function of (6) over theadmissible range of parameter values.

We have consistently observed that Assumption 2 holds for awide range of photographic images satisfying Assumption 1and several different types of parametric TMs. For instance,Fig.5 shows the values of L?(H(γ), λ) corresponding to

6

0.1 0.4 1 1.4 1.90

0.02

0.04

0.06

0.08

0.1

γ value

obje

ctiv

e fu

nctio

n

γ0 = 0.4

γ0 = 1.4

Fig. 5. The relation between the value of L?(H(γ), λ) and γ values,in the case of TM in the form of simple gamma correction. The trueγ value in the two cases are 0.4 and 1.4, which correspond to theglobal minima of the two curves.

gamma correction, Eq.(1), with γ ∈ [0.1, 1.8], where weuse H(γ) to make explicit the dependence of matrix H onparameter γ. (λ = 0.1 is fixed in all cases). The two curvesshown correspond to γ = 0.4 and 1.4, respectively, where thetrue parameter values lead to global minimums of L?(H(γ), λ)as shown by the red circles6.

Based on Assumption 2, our method of finding the optimalTM parameter is then to solve for

γ? = argminγ

L?(H(γ), λ), (15)

This objective function is not differentiable with regards to γdue to the rounding operation in the TM transform. Yet, whenthe TM can be specified by a small number of parameters(e.g., one or two), we can use a simpler approach, which firstperforms a grid search in a given range of parameter valuesfollowed by a binary search between the reduced ranges whereoverall values of L?(H(γ), λ) is minimal, is usually effectiveand leads to satisfactory solutions to (15). The effectivenessof this approach is validated experimentally in Section V-A.

Nonparametric Tone Mapping. When the TM cannot bespecified with a parametric form, we estimate the TM directlyand recover the original PV histogram simultaneously. This isachieved with an iterative method: starting with initial valuesfor H and p, we iterate between

1) solving for p with fixed H;2) solving H with fixed p.

The former is solved with the steps described in Section IV-A,and we turn to the solution of the latter here.

Specifically, with the vectorized input and output PV his-tograms p and q, a unique deterministic monotonic transformexists that satisfies:

1) it converts a variable with distribution given by p toanother that has distribution given by q,

2) it minimizes the overall work that is measured by thetotal displacement of bins.

6Similar observations have also been made on other types of parametricTMs, such as sigmoid stretching and cubic spline curves.

0 100 2000

50

100

150

200

250

true PWBTest. PWBT

0 100 2000

50

100

150

200

250

true PWBTest. PWBT

0 100 2000

50

100

150

200

250

true PWBTest. PWBT

iter #1 iter #3 iter #5Fig. 6. Convergence of the estimated nonparametric free-form TMusing our algorithm.

We can then formulate this problem as following: assumingtwo discrete random variables x, y ∈ {1, · · · , N}, eachwith a corresponding distribution given by p and q (i.e.,P(x = i) = pi and P(y = i) = qi), respectively. Thus,the above requirements can be represented with the followingvariational optimization problem with regards to a mappingφ : {1, · · · , N} 7→ {1, · · · , N}, as

minφ

N∑x=i

(x− φ(x))2pi, s.t. x ∼ p, φ(x) ∼ q.

The solution to this variational optimization problem is a directcorollary of the optimal transportation plan on the real line inthe Monge-Kantorovich transportation theory framework [19].This transform is obtained by a procedure usually known ashistogram matching. Formally, denote χp(·) : {0, N − 1} 7→[0, 1] and χq(·) : {0, N−1} 7→ [0, 1] as the cumulative densityfunctions induced from p and q, respectively, i.e.,

χp(i) =∑ik=1 pi and χp(i) =

∑ik=1 qi.

and χ−1q (·) : [0, 1] 7→ {0, N − 1} as the pseudo inversecumulative distribution induced from q, as

χ−1q (p) = i if χq(i− 1) < p ≤ χq(i).

Then the mapping satisfying the two aforementioned condi-tions is given by [19]

f(i) = χ−1q (χp(i)), (16)

and the corresponding transform matrix is obtained as Hij =δ(j = f(i)).

In summary, our algorithm iterates between optimizing pwith (6) and computing mapping from histogram matchingwith Eq.(16). While we have not obtain a theoretical proof ofconvergence, in practice we observe that the algorithm usuallyconverges within less than 10 iterations. Fig.6 demonstratethe convergence of one estimated TM, with the original TMobtained from cubic spline interpolation of key points thatare chosen manually. As it shows, after 5 iterations of thealgorithm, the estimated TM is already very close to theoriginal TM.

V. EXPERIMENTS

We perform experiments to evaluate the described method.The data used in our experiments are 100 grayscale imagesfrom the van Hateren database [18]. The original images arein 16-bit uncompressed TIFF format, and have been calibratedto have linearized pixel intensities (i.e., removed radiometric

7gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

gamma gammaimage 0.42 0.80 1.10 1.63 2.11 image 0.42 0.80 1.10 1.63 2.11

0.53 0.70 0.96 1.59 2.17 0.47 0.81 1.12 1.59 2.18

0.46 0.73 1.06 1.48 2.13 0.42 0.72 1.00 1.56 2.16

0.43 0.77 1.02 1.57 2.17 0.54 0.86 1.10 1.58 2.18

0.46 0.80 1.07 1.57 2.18 0.46 0.79 1.08 1.49 2.15

0.50 0.89 1.09 1.51 2.16 0.48 0.79 1.11 1.49 2.19

0.44 0.71 0.98 1.36 2.08

0.49 0.82 1.14 1.55 2.18

0.46 0.73 1.02 1.54 2.16

0.49 0.86 1.19 1.60 2.22 0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5

actual gamma

estim

ated

gam

ma

Figure 3: Each initially linear image is printed, digitally scanned and then synthetically gamma correctedwith � 2 [0.4, 2.2]. Shown are the estimated gamma values determined by minimizing the bicoherence.

6

Fig. 7. Examples of images used in our experiments. Original imagecourtesy of the van Hateren database [18].

nonlinearities caused by the lens and sensor). The test imageswere created from the central 1024 × 1024 crops of the 100original images and reduced to 8-bit pixels. Fig.7 shows a fewexamples of these images.

A. Parametric Tone Mapping

We test our method on the blind estimation of severalparametric TMs, including gamma correction, Eq.(1), andsigmoid stretching, Eq.(2). The TM transformed images arecreated from the 100 test images: for gamma correction, weuse γ values taking from the range [0.2, 2.8] with a 0.2 step; forsigmoid stretching, we choose several different combination ofα and β values as shown in the left column of Table III.

1) Gamma Correction: We apply our detection algorithmand compare with two previous works on blind estimationof gamma parameters [5], [2]7. The results, shown as theaverages and standard deviations of the estimated γ value overthe 100 test images, are reported in Table I. The estimationperformance with our method is near perfect and significantlybetter than those of the two compared previous works. Themethod of [5] recovers gamma-like luminance nonlinearitiesas a continuous value transform, and becomes less effective forthe recovery of integer-valued TM. The performances of themethod of [2] deteriorate significantly for γ values close to1, where the gap and peak features become less prominent.We attribute the improved estimation performance of ourdetection algorithm to fact that it is based on the sparsity ofPV histogram. In comparison with the other methods that arebased on higher order image statistics such as [5], [2], thisproperty is more distinct to differentiate original images fromtheir TM transforms.

2) Sigmoid Stretching: Our next experiment tests the per-formance of our method on the estimation of parameters ofsigmoid stretching, and the detection results are reported inTable II. To our best knowledge, there has been no previousmethod for this purpose. As in the case of gamma correction,our method can accurately estimate the transformation param-eters (i.e., α and β).

B. Robustness With Noise and Compression

We further provide an overall qualitative evaluation of theestimation performance and the robustness with regards toperturbations. We select gamma correction as the TM and

7We use MATLAB code implementing the works of [5], [2] in our experi-ments.

actual γ estimated γ (mean/std)our method method of [5] method of [2]

0.20 0.21 (0.01) 0.26 (0.04) 0.24 (0.04)0.40 0.40 (0.01) 0.44 (0.14) 0.42 (0.08)0.60 0.60 (0.01) 0.55 (0.17) 0.58 (0.16)0.80 0.80 (0.00) 0.94 (0.33) 0.96 (0.38)1.00 1.00 (0.00) 1.15 (0.64) 1.00 (0.01)1.20 1.20 (0.01) 1.36 (0.24) 1.32 (0.24)1.40 1.40 (0.02) 1.52 (0.44) 1.44 (0.16)1.60 1.60 (0.00) 1.87 (0.06) 1.59 (0.12)1.80 1.80 (0.00) 2.12 (0.11) 1.78 (0.14)2.00 2.00 (0.02) 2.36 (0.47) 1.99 (0.11)2.20 2.21 (0.01) 2.49 (0.24) 2.13 (0.08)2.40 2.41 (0.01) 2.64 (0.52) 2.43 (0.08)2.60 2.60 (0.01) 2.92 (0.42) 2.68 (0.09)2.80 2.81 (0.01) 3.03 (0.26) 2.91 (0.11)

TABLE IPERFORMANCES OF ESTIMATING GAMMA CORRECTION ON 100

TEST IMAGES WITH DIFFERENT γ VALUES.

apply to each test image with γ values randomly sampled fromthe range [0.1, 2.8]. The estimation performance is evaluatedby accuracy rate, which is computed as the percentage ofestimated γ that fall in the range of ±0.01 of the groundtruths. Shown in the top row of Table III are the performancesevaluated in accuracy rate. Consistent with the results onindividual parameter estimations as shown in Table I, ourmethods achieves the overall best performances out of thethree methods compared.

We further evaluate the robustness of our method in thepresence of noise. Specifically, Gaussian noises with variousnoise levels are added to the gamma corrected image and thenrounded to integral pixel values. We then apply the methodsdescribed in Section IV-B which incorporates the noise effectsfor the estimation of parameter γ. The performances, as wellas comparisons with the previous works of [5] and [2], areshown in the corresponding rows of Table III. Accuracy ratesof all tested methods are affected by the additive noises, butthe performances of our method show more robustness incomparison with those of the previous works. We speculatethe reason is that our method directly incorporate potentialperturbations, while the bases of the previous works (i.e.,higher-order signal dependencies and specific shapes of the PVhistograms) are fragile in the presence of such perturbations.

In our next experiment, JPEG compression of differentquality factors are applied to the same set of gamma correctedimages. Though the quantization artifacts introduced by JPEGcompression are not Gaussian distributed, for simplicity, wetreat it as such in our estimation, and use the method describedin Section IV-B to estimate γ. To estimate the correspondingnoise level of each JPEG quality factor, we take the 100 testimages and JPEG compress them with different quality factors,then compute the standard deviations of the difference betweenthe pixel intensities of the compressed images and the originalimages. Similar to the previous case, though performances ofall methods are affected by JPEG compression, our methodexhibits better robustness in comparison to the other methods,see Table III.

8

actual α/β estimated α/β (mean/std)0.5/-1.0 0.49 (0.01) /-1.02 (0.01)0.5/ 0.0 0.50 (0.02) / 0.01 (0.01)0.5/ 1.0 0.50 (0.01) / 1.01 (0.02)1.0/-1.0 0.98 (0.02) /-0.99 (0.01)1.0/ 0.0 1.02 (0.04) / 0.02 (0.03)1.0/ 1.0 1.01 (0.02) / 1.04 (0.03)2.0/-1.0 2.02 (0.03) /-0.98 (0.02)2.0/ 0.0 2.01 (0.03) / 0.99 (0.03)2.0/ 1.0 1.97 (0.02) /0.98 (0.01)

TABLE IIPERFORMANCES OF ESTIMATING SIGMOID STRETCHING ON 100

TEST IMAGES WITH DIFFERENT α, β VALUES.

accuracyours method of [5] method of [2]

no perturbation 99/100 74/100 87/100

noise

σ = 0.1 91/100 52/100 64/100σ = 0.5 82/100 33/100 44/100σ = 1.0 64/100 12/100 21/100σ = 2.0 24/100 3/100 9/100σ = 5.0 14/100 0/100 2/100

JPEG

Q = 90 97/100 85/100 82/100Q = 80 88/100 63/100 74/100Q = 70 84/100 54/100 69/100Q = 60 76/100 42/100 61/100Q = 50 63/100 35/100 50/100

TABLE IIIOVERALL QUALITATIVE ESTIMATION PERFORMANCE

COMPARISON UNDER DIFFERENT PERTURBATIONS INCLUDINGADDITIVE GAUSSIAN NOISE AND JPEG COMPRESSIONS FOR

ESTIMATION OF GAMMA CORRECTION.

C. Nonparametric Tone Mapping

We further test our methods to recover two different types ofnonparametric TMs – histogram equalization (Eq.(3)) and free-form nonparametric TM created by cubic spline interpolationof manually selecting key points. The latter resembles tonemapping tools in photo-editing software (e.g., the Curve toolin Photoshop).

We applied histogram equalization and free-form TMs toeach of the test images, and then use the algorithm describedin Section IV to recover the integral mapping in the TMdirectly. To further test the robustness of our algorithm,additive Gaussian noise or JPEG compression are applied tothe TM transformed images as in Section V-B. We measure theperformance by the root mean squared error (RMSE) betweenthe actual TMs and the corresponding estimated TMs, treatingboth as functions over [0, N − 1] 7→ [0, N − 1].

We compare with the only known previous work for thesame task is [16], which is based on an iterative and exhaustivesearch of PV histograms that can lead to the observed PVhistogram of an image with TM8. As the results in TableIV and V show, both algorithms achieve good estimationperformances when there is no perturbation. However, ourmethod on average is 5 − 10 times faster in running time,because the method [16] relies on exhaustive search. Moreimportantly, the performances of the method in [16] seem tobe strongly affected by noise and compression, this is in direct

8The code for this method is not made public, and the results are based onour own implementation in MATLAB.

RMSE (mean/std)ours method of [5]

no perturbation 0.12 (0.06) 0.17 (0.08)

noise

σ = 0.1 0.87 (0.32) 1.92 (0.42)σ = 0.5 0.94 (0.43) 1.99 (0.38)σ = 1.0 1.22 (0.97) 3.45 (1.11)σ = 2.0 2.85 (4.21) 7.98 (5.20)σ = 5.0 6.46 (5.62) 10.29 (9.34)

JPEG

Q = 90 0.45 (0.43) 0.94 (0.72)Q = 80 0.75 (1.10) 1.33 (0.98)Q = 70 1.48 (1.27) 2.04 (1.99)Q = 60 4.23 (2.56) 4.45 (3.77)Q = 50 7.35 (7.03) 9.95 (10.57)

TABLE IVPERFORMANCE FOR ESTIMATION OF HISTOGRAM EQUALIZATION

UNDER DIFFERENT PERTURBATIONS INCLUDING ADDITIVEGAUSSIAN NOISE AND JPEG COMPRESSIONS. SMALLER RMSE

VALUES CORRESPOND TO BETTER PERFORMANCES.

RMSE (mean/std)ours method of [5]

no perturbation 0.09 (0.02) 0.14 (0.06)

noise

σ = 0.1 0.64 (0.27) 1.56 (0.33)σ = 0.5 0.86 (0.35) 1.82 (0.29)σ = 1.0 1.43 (0.68) 3.12 (1.41)σ = 2.0 2.92 (1.21) 6.57 (3.62)σ = 5.0 5.94 (4.71) 8.32 (7.18)

JPEG

Q = 90 0.34 (0.27) 1.05 (0.62)Q = 80 0.85 (0.63) 1.29 (1.06)Q = 70 1.11 (0.93) 1.94 (1.48)Q = 60 3.54 (1.87) 4.63 (2.97)Q = 50 5.18 (3.23) 8.32 (5.54)

TABLE VPERFORMANCE FOR THE ESTIMATION OF FREE-FORM TM UNDER

DIFFERENT PERTURBATIONS INCLUDING ADDITIVE GAUSSIANNOISE AND JPEG COMPRESSIONS. SMALLER RMSE VALUES

CORRESPOND TO BETTER PERFORMANCES.

contrast to our method, which can take such perturbation intoconsideration to become more robust.

VI. APPLICATION: REGION SPLICING DETECTION

In this section, we describe how locally applied blind imageTM estimation can be used to identify spliced regions in animage for digital image forensics [15]. Region splicing is asimple and common digital image tampering operation, wherea composite image is created by replacing a contiguous setof pixels in one image with a set of pixels correspondingto an object from a separate image with the aim to modifythe original image’s content. If the two images used to createthe composite image were captured under different lightingenvironments, an image forger may need to perform tone map-ping so that lighting conditions match across the compositeimage, and failure to do this may result in a composite imagewhich does not appear realistic. Image forgeries created inthis manner can then be identified by using localized blindTM estimation to reveal inconsistent TM transforms acrossdifferent regions.

To this end, we created several tampered images with the aidof Adobe Photoshop using region splicing, in which thespliced regions have undergone subtle tone mapping (gammacorrection). Two examples are shown in Fig.8, where the

9

(a) (b) (c) (d) (e)

Fig. 8. (a,b) Two pairs of original JPEG images from Flickr.com and (c) forged images generated with region splicing using the originalimages with Adobe Photoshop. (d,e) Region splicing detection using our method corresponding to block size of 100× 100 and 50× 50,respectively.

first three columns show the original images contributingthe background, the original images contributing the splicedregions, and the tampered images, respectively. To detect thespliced regions, we apply the blind TM estimation to non-overlapping pixel blocks of the forged image, from each ofwhich the tone mapping transform is estimated with the afore-mentioned algorithm. It is important to ensure that the sizeof the testing blocks to balance the accuracy in determiningthe regions and sufficient number of pixels in estimating thehistogram. Fig.8(d) shows the detection results correspondingto block size of 100×100 pixels, showing as binary maps afterwe segment estimated gamma values into two clusters, andFig.8(e) shows detection results when the block size is reducedto 50 × 50 pixels. As these result shows, our method canidentify spliced regions, but several false alarms occurred dueto areas where the pixel values are nearly constant, leading toa pixel value histogram that contains an impulsive componentoutside of the pinch off region.

Acknowledgement This material is based upon work sup-ported by the National Science Foundation under Grant No..

VII. CONCLUSION

In this work, we have described an effective method that canblindly recover tone mappings as mappings between integralpixel intensities. Our method takes advantage of the observedsmoothness of PV histograms of the untransformed digitalimages. We demonstrate the effectiveness of our method on aset of images and showing recovery results close to perfect.We further show the robustness of our method in the presenceof image noise and compression. By examining local areas inthe image, we also show that using this method, we can detectspliced image regions that have undergone different TMs.

There are a few directions we would like to further improvethe current work. First, we would like to analyze the con-vergence of the iterative algorithm for the nonparametric TMestimation algorithm. Second, in applying to image forensics,local TM estimation benefits from using overlapping blocks,

which increases the accuracy in identifying the boundary ofspliced regions. However, using overlapping regions increasescomputation cost, and we would like to investigate the useof data structures such as integrated histograms [14] for moreefficient algorithms.

APPENDIX

PROOF OF EQ.(8)

First, we prove a simpler result in the 1D case, as:

|x| = minv>0,z

(1

2

[(x− z)2

v+ v

]+ |z|

). (17)

To prove (17), we denote ρ(x, v, z) = 12

[(x−z)2v + v

]+ |z|,

and we find its minimum by differentiating with regards to vand z. First, differentiating v and setting the result to zero, weobtain

∂ρ(x, v, z)

∂v= −1

2

[(x− z)2

v2+ 1

]= 0,

i.e., v? = |x−z|. Replacing this result back into (17), we haveρ(x, v?, z) = |x − z| + |z|. Using the triangle inequality, wehave |x − z| + |z| ≥ |x − z + z| = |x|, or ρ(x, v?, z?) = |x|with z? = x or z? = 0. Thus proves the result.

Next, note that Eq.(9) can be decomposed into a series ofminimization problem of (17) as the variables are separable.Therefore this also proves Eq.(9).

REFERENCES

[1] S. Boyd and L. Vandenberghe. Convex Optimization. CambridgeUniversity Press, 2005. 4

[2] G. Cao, Y. Zhao, and R. Ni. Forensic estimation of gamma correctionin digital images. In Image Processing (ICIP), 2010 17th IEEEInternational Conference on, pages 2097 –2100, sept. 2010. 1, 5, 7,8

[3] CVX Research Inc. CVX: Matlab software for disciplined convexprogramming, version 2.0 beta. http://cvxr.com/cvx, Sept. 2012. 4

[4] P. Debevec and J. Malik. Recovering high dynamic range radiance mapsfrom photographs. In ACM SIGGRAPH, 1997. 1

[5] H. Farid. Blind inverse gamma correction. IEEE Transactions On ImageProcessing, 10(10), 2001. 1, 5, 7, 8

[6] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns HopkinsUniversity Press, Baltimore, second edition, 1989. 5

10

[7] M. Gonzalez and F. Woods. Digital image processing. Printice Hall,2nd edition, 2002. 1

[8] M. Grant and S. Boyd. Graph implementations for nonsmooth convexprograms. In V. Blondel, S. Boyd, and H. Kimura, editors, RecentAdvances in Learning and Control, Lecture Notes in Control andInformation Sciences, pages 95–110. Springer-Verlag Limited, 2008.http://stanford.edu/∼boyd/graph dcp.html. 4

[9] R. P. Grimaldi. Discrete and Combinatorial Mathematics: An AppliedIntroduction. 4th edition, 1998. 2

[10] M. Jaggi. Revisiting Frank-Wolfe: Projection-free sparse convex opti-mization. In International Conference on Machine Learning, Atlanta,Georgia, 2013. 4

[11] S. Lin, J. Gu, S. Yamazaki, and H. Shum. Radiometric calibration froma single image. In IEEE Conference on Computer Vision and PatternRecognition, 2004. 1

[12] S. Lin and L. Zhang. Determining the radiometric response functionfrom a single grayscale image. In IEEE Conference on Computer Visionand Pattern Recognition, 2005. 1

[13] C. A. Micchelli and M. Pontil. A family of penalty functions forstructured sparsity. In NIPS, Grenada, Spain, 2011. 4

[14] F. Porikli. Integral histogram: a fast way to extract histograms incartesian spaces. In Computer Vision and Pattern Recognition, 2005.CVPR 2005. IEEE Computer Society Conference on, volume 1, pages829–836 vol. 1, June 2005. 9

[15] H. T. Sencar and N. Memon, editors. Digital Image Forensics: There isMore to a Picture than Meets the Eye. Springer, 2012. 1, 8

[16] M. Stamm and K. Liu. Forensic detection of image manipulation usingstatistical intrinsic fingerprints. Information Forensics and Security,IEEE Transactions on, 5(3):492–506, 2010. 2, 5, 8

[17] M. Stamm and K. Liu. Forensic estimation and reconstruction ofa contrast enhancement mapping. In Acoustics Speech and SignalProcessing (ICASSP), 2010 IEEE International Conference on, pages1698 –1701, march 2010. 1

[18] J. van Hateren and A. van de Schaaf. Independent component filtersof natural images compared with simple cells in primary visual cortex.Proceedings: Biological Sciences, 265(1394):359–366, 1998. 6, 7

[19] C. Villani. Optimal Transport: Old and New. Springer, Berlin, 2008. 6[20] Y. Wang, J. Yang, W. Yin, and Y. Zhang. A new alternating minimization

algorithm for total variation image reconstruction. SIAM J. ImagingScience, 1(3):248–272, 2008. 4

[21] X. Zhang and S. Lyu. Estimating covariance matrix of correlated imagenoise. In IEEE Conference on Image Processing (ICIP), Paris, France,2014. 1

[22] D. Zoran and Y. Weiss. Scale invariance and noise in nature image.In IEEE International Conference on Computer Vision,, Kyoto, Japan,2009. 5