J Math Imaging VisDOI 10.1007/s10851-009-0177-7

Infinite Photography: New Mathematical Modelfor High-Resolution Images

Tapio Helin · Matti Lassas · Samuli Siltanen

© Springer Science+Business Media, LLC 2009

Abstract A new mathematical image model is introducedusing the photographic process as the starting point. Im-ages are represented as infinite sequences of photons al-lowing analysis at arbitrarily high resolution and leading tonovel computational methods for processing, representation,transmission and storage of images. The resulting space ofinfinite photographs is proved to have a metric structure andto be intimately connected with bounded Borel measures.Theorems are proved indicating that the imaging power ofthe photographic process exceeds function spaces in the highresolution limit; this implies in particular that natural photo-graphic images need to be modelled by generalized func-tions. Furthermore, computational results are presented toillustrate the novel algorithms based on photon sequences.The algorithms include stochastic halftoning, representationof cartoon images with outlines, anti-aliasing, blurring andsingularity extraction.

Keywords Image model · Photons · Metric space · Borelmeasure · Fractal · Local dimension · Halftoning · MonteCarlo

1 Introduction

We introduce a new mathematical image model based on thephysical process of photography. Images are represented as

T. Helin (!)Department of Mathematics and System Analysis,Helsinki University of Technology, P.O. Box 1100(Otakaari 1 M), 02015 TKK, Helsinki, Finlande-mail: tapio.helin@iki.fi

M. Lassas · S. SiltanenDepartment of Mathematics and Statistics, University of Helsinki,P.O. Box 68 (Gustaf Hallstromin katu 2b), 00014, Helsinki,Finland

infinite sequences of photons, allowing analysis at arbitrar-ily high resolution and leading to novel computational meth-ods for the analysis, processing, representation, transmissionand storage of images. Moreover, the model provides newunderstanding of the structure of photographs: they are fi-nite approximations of Borel measures.

The starting point of our approach is a simplified modelof a monochromatic digital camera comprising a light-sensitive surface called sensor, an optical arrangement, anda shutter for preventing light from passing through the op-tics. Photographs are taken by opening the shutter for a suit-able period of time, allowing a large number of photonsto arrive at the sensor which is divided into square-shapeddisjoint subsets called pixels. Each pixel then measures thewavelength-dependent energy delivered by the photons, andthe gray level of the pixel is determined by the ratio betweenenergy detected at that pixel and the total energy received bythe whole sensor.

We model the sensor surface as a unit square. An ex-posure is an infinite ordered sequence e = (e1, e2, e3, . . . ),where each ej = (zj ,!j ) models a photon with wavelength!j > 0 arriving at the sensor at the location zj . When a digi-tal photograph is taken, the first J photons in an exposuree reach the sensor. The resulting gray level of a pixel D

is

Eg(e, J,D)

J |D| , (1.1)

where |D| denotes the area of the pixel and

Eg(e, J,D) :=!

zj!D1"j"J

g(!j ) (1.2)

is the energy detected at the pixel D. Here g : R # Ris the spectral sensitivity function of the sensor satisfying

J Math Imaging Vis

0 " g(!) "M with some sensor-dependent constant M <

$.The above is a mathematical model of digital black-and-

white photography based on finite exposures. Let us maketwo observations clarifying why we need to consider infinitesequences of photons.

First, images formed by practical sensors always containsome error. Typical sensor noise can be modeled as a com-bination of quantum mechanical effects in the photon count-ing process and normally distributed additive random errorarising from sensor electronics. Assume for the sake of theargument that the latter is negligible. Then the photon countat each pixel is accurately modeled by a Poisson distributedrandom variable with expectation " and standard deviation%" , and the signal-to-noise ratio "/

%" =%" tends to in-

finity as " grows. Hence, the ideal limit image formed byinfinitely many photons is free of noise and therefore an in-teresting object to study.

Second, consider taking a correctly exposed photographof a natural scene with a digital camera. Suppose we wanta higher-resolution picture of the same scene, and to thatend we use another camera with identical optics and same-size sensor divided into more pixels. Then, according to thefirst observation above, we need to extend the exposure timeto get an image with the same signal-to-noise ratio as thefirst one. Therefore, allowing infinite sequences of photonsis necessary for the analysis of photographs at arbitrarilyhigh resolutions.

The gray level resulting from an infinite number of pho-tons is

limJ#$

Eg(e, J,D)

J |D| . (1.3)

We construct a sequence of successively finer pixelizationsof the sensor and define digital images formed by e at anyresolution using formula (1.3) in each pixel. A concept ofdistance between two exposures is defined by comparingtheir pixel images at all resolutions simultaneously, lead-ing to the metric space of infinite photographs consistingof equivalence classes of exposures: two exposures belongto the same class if and only if their pixel images coincide atall resolutions.

Our infinite photography model is inspired by the physicsof counting photons with digital detectors. However, whilethe mathematical model applies to arbitrarily high resolu-tions, it does not properly describe quantum mechanical ef-fects at sub-nanometer scale. The value of our model isin (i) deeper understanding of the structure of digital pho-tographs provided by asymptotic analysis, and in (ii) newimage processing methods that can be applied consistentlyat any given finite resolution.

Concerning (i), we prove that a set of uniformly boundedBorel measures can be identified with a subset of infi-

nite photographs. In particular, our result implies that pho-tographs need to be modelled by generalized functions inthe high resolution limit. Also, our mathematical construc-tion using infinite sequences of photons comes with a price:it turns out that there exist irregular photon sequences thatproduce unexpected limit images. However, excluding suchanomalies leads to a set of regular exposures that can be usedto define an embedding of finitely exposed photographs as asubset of infinite photographs.

Concerning (ii), we describe in Sect. 5 practical algo-rithms based on infinite photography. Pixel images andmore general Borel measures can be mapped to expo-sures using a Markov chain Monte Carlo algorithm, lead-ing to a scalable image representation. Further, applyingthis method to a negative image and interpreting photonsas spots of ink constitutes a novel and flexible stochastichalftoning method applicable in graphical printing tech-nology. Moreover, the familiar techniques of anti-aliasingand blurring allow photon-based implementations, and theRadon-Nikodym decomposition of Borel measures can like-wise be turned into a new method for extracting singulari-ties.

Although not discussed further in this paper, our MonteCarlo algorithm can be used for creating useful distributionsof photons for virtual illumination in computer graphics,see [14]. Also, infinite photography may prove useful forthe development of active detectors based on controlling themovement of photons, see for instance [4].

Our image model provides a new contribution tothe tradition of modeling photographs with more generalobjects than functions. Mumford and Gidas give statisti-cal analysis of natural images viewed as distributions in[25]. See also the statistical modeling done in [5, 32,33]. Extensive analysis and some numerical algorithms areavailable for Meyer’s image model based on dual spaces[3, 10, 23]. Also, our work can be seen via the Radon-Nikodym decomposition as a novel u + v model divid-ing images into two parts, but in a different way than in[2, 9, 19, 26, 31, 34, 37].

2 The Metric Space of Infinite Photographs

We present a mathematically rigorous definition of the met-ric space of infinite photographs. The construction involvesadding some technical details to the streamlined explanationin the introduction.

2.1 Resolutions and Pixels

We take D to be the two-dimensional torus constructed bygluing together sides of the square [0,1]2 appropriately.More specifically, let us periodize the plane R2 by identi-fying all points of the form (x + m,y + n) with arbitrary

J Math Imaging Vis

Fig. 1 Canonical pixelizationsfor resolutions n = 0,1,2

m ! Z and n ! Z. We will identify points z ! D with pla-nar points (x, y) ! R2 satisfying 0 " x < 1 and 0 " y < 1.Considering the torus instead of the square simplifies thecomputational treatment of sensor boundaries.

Let us introduce a series of pixelizations of D at variousresolutions. For each resolution parameter n = 0,1,2, . . .

define pixels

Dn(k,#) :="

(x, y)

#####(#& 1)2&n " x < #2&n,

1& k 2&n " y < 1& (k & 1)2&n

$

for 1" k " 2n and 1" #" 2n. For fixed n' 0 the pixels aredisjoint:

Dn(k,#)(Dn(k),#)) = * if k += k) or # += #),

and they cover the image domain completely:

D =2n%

k=1

2n%

#=1

Dn(k,#).

In particular we have D0(1,1) = D. Moreover, any pixelDn(k,#) is a union of exactly four pixels of the formDn+1(k

),#)). See Fig. 1.

2.2 Pixel Images from Exposures

Define the set of exposures as

E := (D,R)N,

where N = {1,2,3, . . . }. More explicitly, exposures e ! E

are of the form e = (ej )$j=1 = (e1, e2, e3, . . . ), where the

photons ej can be written as ej = (zj ,!j ) = ((xj , yj ),!j )

with 0" xj < 1 and 0" yj < 1.Throughout the paper we consider spectral sensitivity

functions with the following properties.

Definition 1 We call a continuous function g : R # Ra spectral sensitivity function if 0 " g(!) " M for some0 < M <$ and there exist !(0) ! R and !(M) ! R so thatg(!(0)) = 0 and g(!(M)) = M .

Given a resolution parameter n ' 0, the pixel imageIn(e, J ) produced by the J first photons of e is the follow-ing 2n , 2n matrix:

In(e, J ) :=

&

'(In(e, J,1,1) · · · In(e, J,1,2n)

.... . .

...

In(e, J,2n,1) · · · In(e, J,2n,2n)

)

*+ ,

(2.1)

where the matrix elements, or pixel values, are defined by

In(e, J, k,#) := Eg(e, J,Dn(k,#))

2&2nJ

= 12&2nJ

!

zj!Dn(k,#)1"j"J

g(!j ), (2.2)

where 1 " k " 2n is row index and 1 " # " 2n is columnindex and Eg is defined in (1.2). Note that (2.2) is con-sistent with (1.1) since |Dn(k,#)| = 2&2n. The assumption0" g(!)"M implies the estimates

0" In(e;J, k,#)" 22nM (2.3)

and

0 "!

1"k,#"2n

In(e;J, k,#)

= 12&2nJ

!

zj!D1"j"J

g(!j )" 22nM. (2.4)

Definition 2 The pixel image In(e, J ) defined in (2.1) iscalled the J -snapshot of e !E at resolution n.

Next we construct ideal images, called snapshots, createdby exposures with infinitely many photons. For this we needthe concept of Banach limit B-lim, which is a linear func-tional on the space of sequences (cj )

$j=1 of real numbers.

A Banach limit has the property

lim infj#$

cj " B- limj#$

cj " lim supj#$

cj . (2.5)

J Math Imaging Vis

Moreover, if the sequence (cj )$j=1 is convergent, then

B- limj#$

cj = limj#$

cj ,

but a Banach limit is well-defined even for divergent se-quences [29].

Definition 3 The snapshot of e !E at resolution n is

In(e) :=

&

'(In(e,1,1) · · · In(e,1,2n)

.... . .

...

In(e,2n,1) · · · In(e,2n,2n)

)

*+ , (2.6)

where the pixel values are defined by

In(e, k,#) := B- limJ#$

In(e, J, k,#). (2.7)

Note that a combination of (2.3), (2.5) and (2.7) yieldsthe estimate

0" In(e; k,#)" 22nM. (2.8)

The reason for using B-lim in formula (2.7) instead of moreexplicitly defined operations, such as lim inf or lim sup, isthe linearity of B-lim with respect to the sequence. We willuse the linearity property in several proofs below. However,we will later restrict our analysis to so-called regular expo-sures, for which there is no need to use Banach limit in for-mula (2.7).

The next example shows that in general we cannotreplace B-lim in (2.7) by lim since the sequence(In(e, J, k,#))$J=1 may diverge.

Example 1 Let g be as in Definition 1 with M > 0. Fixn ' 0 and let k,# be the unique indices satisfying ( 1

2 , 12 ) !

Dn(k,#). Set J1 = 1 and e1 = (( 12 , 1

2 ),!(M)); then by (1.2)we have Eg(e, J1,Dn(k,#)) = g(!(M)) = M. We proceed byinduction. Assume that Jm first photons of e have been con-structed with m ' 1. Then repeat these two steps ad infini-tum:

• If !Jm = !(M), then set ej := (( 12 , 1

2 ),!(0)) for j = Jm +1, . . . , Jm+1 with so large Jm+1 that

Eg(e, Jm+1,Dn(k,#)) = 1Jm+1

Jm+1!

j=1

g(!j ) <M

4.

• If !Jm = !(0), then set ej := (( 12 , 1

2 ),!(M)) for j = Jm +1, . . . , Jm+1 with so large Jm+1 that

Eg(e, Jm+1,Dn(k,#)) = 1Jm+1

Jm+1!

j=1

g(!j ) >M

2.

Now the sequence (In(e, J, k,#))$J=1 diverges since

In(e;Jm, k,#) > M22n&1 for odd m,

In(e;Jm, k,#) < M22n&2 for even m.

2.3 The Snapshot Metric

For each n' 0 we introduce a pseudometric dn in the set E

by the formula

dn(e, e)) := cn-In(e)& In(e

))-2, (2.9)

where cn = (%

2 · 22n)&1 and - ·- 2 denotes the Euclid-ean distance between matrices interpreted as elements inR2n,2n

. Let us check that dn really is a pseudometric. Firstthe triangle inequality:

dn(e, e))) = cn-In(e)& In(e

)))-2

" cn-In(e)& In(e))-2 + cn-In(e

))& In(e)))-2

= dn(e, e)) + dn(e

), e))).

Then symmetry: dn(e, e)) = cn-In(e) & In(e

))-2 =cn-In(e

)) & In(e)-2 = dn(e, e)), and finally, dn(e, e) =

cn-In(e)& In(e)-2 = 0.The estimates (2.8) and (2.4) imply -In(e)-2 " 22nM

and thus we get a bound on dn(e, e)). Namely,

dn(e, e)) = cn

, 2n!

k=1

2n!

#=1

|In(e; k,#)& In(e); k,#)|2

-1/2

" cn

.-In(e)-2

2 + -In(e))-2

2/1/2

" cn((22nM)2 + (22nM)2)1/2

= M (2.10)

where we have used the fact that intensities are alwayspositive. We use the sequence d0, d1, d2, . . . to construct apseudometric involving all resolutions simultaneously. De-fine

d(e, e)) := supn'0

dn(e, e))

n + 1. (2.11)

Note that (2.10) implies that d(e, e))"M .The following result defines examples with d(e, e)) = 0

but e += e).

Lemma 1 Let e = (e1, e2, e3 . . . ) ! E and K > 0. De-fine another exposure by e) := (eK+1, eK+2, eK+3, . . . ) !E.Then for any n' 0 we have In(e) = In(e

)).

Proof Fix n' 0 and 1" k " 2n and 1" #" 2n. We need toshow

B- limJ#$

In(e);J, k,#) = B- lim

J#$In(e;J, k,#). (2.12)

J Math Imaging Vis

Denote Dn = Dn(k,#) and compute

Eg(e), J,Dn) =

!

zj+K!Dn

1"j"J

g(!j )

=!

zj!Dn

1"j"J+K

g(!j )&!

zj!Dn

1"j"K

g(!j )

= Eg(e, J + K,Dn)&GK, (2.13)

where |GK | <$. By (2.13) and the linearity of a Banachlimit we have

B- limJ#$

Eg(e), J,Dn)

J

= B- limJ#$

Eg(e, J + K,Dn)

J&GK

0B- lim

J#$J&1

1

= B- limJ#$

Eg(e, J + K,Dn)

(J + K)&K

=0

B- limJ#$

Eg(e, J + K,Dn)

J + K

1,

limJ#$

1

1& KJ+K

-

= B- limJ#$

Eg(e, J + K,Dn)

J + K,

and (2.12) follows from (2.2). !

2.4 The Space P of Infinite Photographs

Define an equivalence relation . between exposures e ! E

and e) !E by

e. e) if and only if d(e, e)) = 0,

where d is the pseudometric defined in (2.11). Denote theequivalence class of e by [e] = {e) !E : e) . e}.

Recall that the difference between a pseudometric and ametric is that a metric needs to additionally satisfy that if thedistance between e and e) is zero then e = e); see [16]. It iseasy to see that the pseudometric d of E extends to a metricon the quotient space E/.= {[e] : e !E}. Namely, assumethat d(e, e)) = 0 so that [e] = [e)]. Then by (2.9) and (2.11)we can estimate for any fixed n' 0

-In(e)& In(e))-2 = 1

cndn(e, e

))" n + 1cn

d(e, e)) = 0,

so In(e) = In(e)). Thus we can set

dn([e], [e)]) := dn(e, e)) (2.14)

with arbitrary choice of representatives of equivalenceclasses e ! [e] and e) ! [e)], and

d([e], [e)]) := maxn'0

dn([e], [e)])n + 1

. (2.15)

Definition 4 The set P := E/. is called the space of infi-nite photographs. The metric d of P is called the snapshotmetric.

Note that the snapshot map In : E # R2n,2ndefined in

(2.6) extends naturally to a map In : P #R2n,2n.

3 Infinite Photographs and Borel Measures

3.1 Mapping Exposures to Measures

In this section we show how an exposure e ! E, togetherwith a spectral sensitivity function g as in Definition 1, in-duces a Borel measure µe on D. Define

µ/(D) = inf

" $!

$=1

2&2n$ In$ (e, k$,#$) :

D 0$%

$=1

Dn$ (k$,#$)

$

(3.1)

for any subset D 0 D. Note that 0 " µ/(D) "M . Due to[24, Prop. 2.4] the mapping µ/ is an outer measure.

Now µ/ satisfies the Caratheodory condition: if D1 0 Dand D2 0D are such that

0 < inf {|x1 & x2| : x1 !D1, x2 !D2} ,

then

µ/(D1 1D2) = µ/(D1) + µ/(D2).

This implies that Borel sets are µ/-measurable [8,Thm. 4.12], and we can define a Borel measure µe as therestriction of µ/ to Borel sets.

Since the definition (3.1) only depends on the snapshotmap In, we see that µe = µe) whenever e . e). Thus thefollowing mapping is well-defined.

Definition 5 Denote by BM(D) the set of all positive Borelmeasures µ on D satisfying µ(D)"M . The limit image mapM : P # BM(D) is defined by

M([e]) := µe,

where µe is the restriction of the outer measure µ/ to Borelsets.

J Math Imaging Vis

The snapshot metric space for exposures was constructedso that converging sequence must converge at every resolu-tion. Similarly we can ask whether the snapshots approxi-mate the induced measure somehow. Let e ! E be an ex-posure and µ = M([e]) with a given spectral sensitivityfunction g. Now we can define a sequence µ0,µ1,µ2, . . .

of simple measures by

µn =2n!

k=1

2n!

#=1

In(e, k,#)%Dn(k,#)(z)dz, (3.2)

where dz is Lebesgue measure on D. Then we have the fol-lowing approximation result.

Theorem 1 The sequence µ1,µ2, . . . converges to µ in theweak topology of Borel measures.

Proof We need to show that

limn#$

2

D& dµn =

2

D& dµ

for an arbitrary continuous function & ! C(D). The mea-sures µn and µ are regular Borel measures with finite varia-tion:

|µn| (D)"M and |µ| (D)"M.

Let now & ! C(D) and note that due to compactness of thetorus D the function & is uniformly continuous. Let ' > 0.Let m = m(&) be such that we can find a piecewise constantfunction

&m(x) =2m!

k=1

2m!

#=1

a(k,#)%Dm(k,#)(x)

for which -& & &m-$ < '/(2M). When j ' m we haveµj (Dm(k,#)) = µ(Dm(k,#)) for all 1 " k " 2m and 1 "#" 2m and thus####

2

D&dµ&

2

D&dµj

####

"####

2

D(& & &m)dµj

#### +####

2

D(& & &m)dµ

####

+####

2

D&m(dµ& dµj )

####

< '/2 + '/2 + 0 = ',

which concludes the proof. !

Example 2 Assume that g(!) = 1 for some ! ! R and de-fine an exposure ( = ((j )

$j=1 ! E by (j = (z0,!), where

z0 = ( 12 , 1

2 ). Because all photons arrive at the same point,

this exposure models photographing a bright and distant starin otherwise black night sky.

Let us determine the J -snapshots of (. For a fixed n' 0let k and # be the unique indices for which z0 ! Dn(k,#).Then

In((, J, k),#)) = 22n

J

!

zj!Dn(k),#))1"j"J

g(!)

="

22n if k) = k and #) = #,

0 otherwise,(3.3)

so all pixel values are constant with respect to J . See Figs. 2and 3 for illustration of J -snapshots of ( with varying valuesof J .

Consider now the measure µ( induced by (. For everyBorel set D containing z0 it is easy to see that µ((D) =1 and µ((D \ D) = 0. This follows since any cover D 03$

$=1 Dn$ (k$,#$) includes a pixel that contains z0. Thus weconclude that µ( = (z0 , a point mass at z0.

3.2 Mapping Measures to Exposures

Our starting point here is a positive Borel measure µ !BM(D) and a spectral sensitivity function g as in Defini-tion 1 with M ' µ(D)' 0.

Let (),*,P) be a complete probability space and letZ(j) : )# D with j = 0,1,2, . . . be a sequence of inde-

Fig. 2 J -snapshots of the exposures ( and (+ and (& at resolutionn = 4 (size 16 , 16 pixels). See Example 2 for the definition of (,and Example 4 for the definitions of (+ and (&. Note that the imagesrelated to ( in the top row are actually independent of J , as shownby formula (3.3). According to the theory, the images in the middlerow converge to the images in the top row as the number of photonincreases. The scale is the same in all the images: black is zero andwhite is 22n = 28 = 256

J Math Imaging Vis

Fig. 3 J -snapshots of the exposures ( and (+ and (& at resolutionn = 6 (size 64 , 64 pixels). See Example 2 for the definition of (,and Example 4 for the definitions of (+ and (&. Note that the pixelimages related to ( in the top row are actually independent of J , asshown by formula (3.3). According to the theory, the images in themiddle row converge to the images in the top row as the number ofphoton increases. Comparison to Fig. 2 shows that the convergencerate is slower in the higher-resolution case. The scale is the same in allthe images: black is zero and white is 22n = 212 = 4096

pendent random variables. If µ(D) > 0 assume that eachZ(j) has probability distribution µ(D)&1µ, and in the caseµ(D) = 0 assume that each Z(j) is uniformly distributedon D.

For each pixel Dn(k,#) with n ' 0 and 1 " k " 2n and1 " # " 2n consider the sequence (%Dn(k,#)(Z

(j)))$j=1 ofrandom variables. The strong law of large numbers impliesalmost surely

limJ#$

1J

J!

j=1

%Dn(k,#)(Z(j)(+))

="

µ(Dn(k,#))µ(D) , if µ(D) > 0,

|Dn(k,#)|, if µ(D) = 0.(3.4)

Let )n,k,# denote the subset of ) in which (3.4) fails. Thecountable union over all sets )n,k,# has zero measure:

P,

%

n'01"k,#"2n

)n,k,#

-

= 0.

Thus for every µ ! BM(D) one can always choose a (notnecessarily unique) +µ ! ) for which (3.4) holds for anychoice of n, k and #.

Definition 6 The illumination map P : BM(D)# P is de-fined by

P (µ) := [((Z(j)(+µ),!))$j=1],

where ! satisfies g(!) = µ(D).

It is straightforward to show that P (µ) is well-defined.We note that due to (3.4) any exposure e ! P (µ) satisfies

In(e, k,#) = 22nµ(Dn(k,#)) (3.5)

for any n !N and 1" k " 2n and 1" #" 2n.We remark that the map P is deterministic although its

construction includes a random procedure.In Example 2 we constructed a point measure using in-

finite sequences of photons. Measures supported on one-dimensional curves can be represented by photon sequencesas well, leading to a scalable anti-aliasing technique:

Example 3 Let , be a continuous curve , : [0,1]# D andlet us consider the measure µ satisfying

µ(E) =2 1

0%E(, (t))dt

for any measurable E 0 D. Assume that g(!) = 1 for some! ! R. We construct an exposure c ! E by choosing a uni-formly distributed random sequence of points s1, s2, . . . onthe interval [0,1] and defining photons cj by

cj := (, (sj ),!), j = 1,2,3, . . .

It is now easy to see that P (µ) = [c]. See Fig. 4 for a sim-ple sine curve example where, up to scaling the coordinates,, (s) = (s, sin(2-s)).

3.3 The Set of Regular Exposures

In this subsection we discuss some important properties ofthe limit image map M and the illumination map P . Let usfirst study two examples illustrating some counter-intuitivebehaviour.

Example 4 Define exposures (+ = ((+j )$j=1 and (& =

((&j )$j=1 by

(±j =

0z0 ±

03

10j,

310j

1,!

1,

where z0 = ( 12 , 1

2 ). See Figs. 2 and 3 for J -snapshots of (+

and (&.

J Math Imaging Vis

Fig. 4 Pixel images produces by the curve exposure c !E constructedin Example 3. The images in the top row are at resolution n = 4 andhave size 16,16. The images in the bottom row are at resolution n = 6and have size 64, 64. Note the adaptive anti-aliasing (smoothing ofcorners by gray pixel values instead of having only black and whitepixels) provided by the photon-based approach. Note also how morephotons are needed at the higher resolution to achieve good image qual-ity. The scales are different in all the eight plots; black is always zerobut each image has been scaled so that maximal pixel value is white

Let ( be as in Example 2. It is easy to see thatlimJ#$ In((

+, J, k,#) = In((, k,#) for all n, k,#, show-ing that d((, (+) = 0. Furthermore,

M(() = (z0 = M((+),

where (z0 denotes a point measure at z0.However, for any n ' 1 and j ! N we have z0 &

( 310j , 3

10j ) /!Dn(k,#) if z0 !Dn(k,#). This means that

In((&, k,#) = lim

J#$In((

&, J, k,#) = 0

for all pixels Dn(k,#) with n' 1 and containing z0. For thedistance of ( and (& this implies that d((, (&) += 0. More-over, we can show that

M((&)2 0.

Namely, denote by t$ = 4$j=1

12j+1 , $ ' 1, a sequence of

points converging to 12 as $ increases. Let Dn$ (k$,#$) be

the pixel with lower left corner at (t$, t$) and n$ = $+2. Wenotice that In$ ((

&, k$,#$) = 0. Consider now the definitionof µ(&(D) in (3.1). By writing D as a union of disjoint pixelsincluding Dn$ (k$,#$) one sees that µ(&(D) must be zero.

Example 5 Take 0 " x < 1 and 0 " y < 1 to be irrationalnumbers and set z0 = (x, y). Consider the exposures (+ and(& defined in Example 4. Then due to our dyadic pixel con-struction we have M((+) = (z0 = M((&).

Example 4 shows that there exists no such positive Borelmeasure µ on D that P (µ) = [(&] since due (3.5) we haveµ(Dn(k,#)) = 2&2nIn((

&, k,#) and this leads to a contra-diction with the assumption that µ is countably additive.

It turns out that we can characterize the subset of P thatis natural for mappings M and P . Let us define regular ex-posures and regular photographs as follows.

Definition 7 An element e ! E is a regular exposure if theequality µe(Dn(k,#)) = 2&2nIn(e, k,#) holds and the limitlimJ#$ In(e;J, k,#) exists for any n ' 0 and for all in-dices 1 " k " 2n and 1 " # " 2n. The subset of regular ex-posures is denoted by E0 0E.

Further, let P0 be the subset of P such that every equiv-alence class [e] ! P0 has a regular exposure representative.This P0 is called the set of regular infinite photographs.

Note that for regular exposures regular limit can be usedin formula (2.7) instead of a Banach limit. Let us study in thefollowing how regular exposures are related to mappings Mand P .

Theorem 2 The maps M : P # BM(D) and P : BM(D)#P have the following properties:

(i) M 3P is the identity map on BM(D),(ii) P 3 M : P # P is a projection onto P0,

(iii) P 3 M is the identity map on P0.

Proof Let us prove (i)–(iii) via five separate claims: weshow that

(a) M : P0 # BM(D) is injective,(b) P : BM(D)# P is injective,(c) P (M([e])) = [e] for any [e] ! P0 and(d) M(P (µ)) = µ for any µ ! BM(D).

Note that two positive Borel measures µ and µ) coincide onD if for any pixel Dn(k,#) with n ! N and 1 " k " 2n and1" #" 2n we have

µ(Dn(k,#)) = µ)(Dn(k,#)).

This is due to the fact that pixels Dn(k,#) span the Borel" -algebra on D, see [30].

To prove (a) let [e], [e)] ! P0 and assume [e] += [e)]. Thenthere exists n, k and # such that In(e, k,#) += In(e

), k,#).Since e and e) are regular we have µe(Dn(k,#)) +=µe)(Dn(k,#)). This implies that M([e]) += M([e)]).

Now for problem (b) let µ,µ) ! BM(D) and µ += µ).Then there exists n, k and # such that µ(Dn(k,#)) +=µ)(Dn(k,#)). Let e ! P (µ) and e) ! P (µ)). Accordingto (3.5) we have In(e, k,#) += In(e

), k,#) and furthermoreP (µ) += P (µ)).

(c) Let [e] ! P0, f ! P (M([e])) and denote µ =M([e]). We see that by definition Ran(P ) 0 P0 and thus[f ] ! P0. From this notion and (3.5) we have that

In(f, k,#) = 22nµ(Dn(k,#)) = In(e, k,#) (3.6)

J Math Imaging Vis

for any n ' 0 and 1 " k " 2n and 1 " # " 2n. This provesthat [e] = [f ] = P (M([e])).

(d) Let µ ! BM(D) and denote [e] = P (µ) and µ) =M(P (µ)). For any pixel Dn(k,#) we have

µ)(Dn(k,#)) = 2&2nIn(e, k,#) = µ(Dn(k,#))

since e !E0 and second equality due to (3.5). !

3.4 Embedding Finite Photographs to P

In practice, every photograph is formed by a finite number ofphotons. For our infinite photography model to be useful, weneed to find an appropriate way to embed finite exposuresinto the space P . There are two main properties we wishfrom the embedding.

Photographic experiments show that detecting more pho-tons leads to an increase in image quality. This can be eas-ily observed using any digital camera: take a picture of thesame scene with two different sensitivity settings, for ex-ample ISO 800 and ISO 100, and the latter photograph willbe less noisy. This suggests that finite photographs of nat-ural scenes are created by truncating an infinite exposure forwhich ordinary limit exists in (2.7).

Our way of embedding practical photographs into the in-finite photography model is to represent them as repeatedfinite sequences.

Definition 8 Define an infinite sequencing operation SN :E# E for N > 0 as follows. For any e = (e1, e2, . . . ) ! E

set

SNe = (e1, e2, . . . , eN , e1, e2, . . . , eN , e1, e2, . . . , eN , . . . ).

The following result shows the convergence of pho-tographs at any resolution as the exposure time increases.

Theorem 3 Let g be as in Definition 1 and let e ! E0 be aregular exposure. Then

limN#$

d(SNe, e) = 0. (3.7)

Proof Let n ' 0 be arbitrary and calculate using the #1-norm

-In(SNe)& In(e)-1

=2n!

k=1

2n!

#=1

#### limJ#$

Eg(SNe, J,Dn(k,#))

2&2nJ

& limJ#$

Eg(e, J,Dn(k,#))

2&2nJ

####

= 22n2n!

k=1

2n!

#=1

#####1N

!

zj!Dn(k,#)1"j"N

g(!j )

& limJ#$

1J

!

zj!Dn(k,#)1"j"J

g(!j )

#####. (3.8)

We see that (3.8) implies

limN#$

dn(SNe, e) = limN#$

-In(SNe)& In(e)-2

" limN#$

-In(SNe)& In(e)-1 = 0,

where the last inequality follows from the 2-norm being al-ways smaller than 1-norm on R22n

. So we have proved for-mula (3.7) with d replaced by dn.

It remains to prove convergence in (3.7) with the actualpseudometric d . Let ' > 0. Choose an integer q so largethat M/(q + 1) < '. Further, choose Q = Q(q) so thatdn(SNe, e) < ' with every N > Q and 0 " n " q . WhenN > Q we have by (2.10)

maxn'0

dn(SNe, e)

n + 1

= max5

max0"n"q

dn(SNe, e)

n + 1,max

n>q

dn(SNe, e)

n + 1

6< ',

and the proof of (3.7) is complete. !

4 Decomposition and Fractal Analysis of Images

The well-known decomposition theorem by Lebesgue statesthat a Borel measure µ on D can be uniquely divided intotwo parts

µ = & + !,

where & is absolutely continuous with respect to Lebesguemeasure m and ! is singular in the sense that it vanishesoutside a set of Lebesgue measure zero. In the following wedenote a(µ) = & and s(µ) = !. Furthermore, the Radon-Nikodym derivative fµ = d(a(µ))

dm belongs to L1(D) and wecan write

µ = fµdm + s(µ). (4.1)

In our image model, an exposure e ! E induces a Borelmeasure µe = M([e]), which we can write in the form

µe = fµedm + s(µe). (4.2)

Now the question is how to construct exposures ea and es

such that

M([ea]) = fµedm and M([es]) = s(µe). (4.3)

In Sect. 4.1 we develop theoretical tools that can be used forlocating the support of the singular part of a measure and for

J Math Imaging Vis

analysing the pointwise degree of singularity using local di-mension. This approach is closely connected to multifractalformalism [6, 7, 13, 18, 20, 28, 35, 36]. Further, in Sect. 4.2we outline a photon-based edge detection method.

Our photon-based image model together with formula(4.2) provides a new contribution to the research of u + v

image models [19, 23, 31, 37], where typically u representsa regular part of the image, while v is a singular part (tex-ture or noise). In some models, including ours, the singularpart v cannot be represented as a function, leading to seri-ous difficulties in computational approaches. Some promis-ing methods have been introduced with the problem of rapidoscillations [3, 10, 23], and in Sect. 5.5 below we present anovel photon-based algorithm.

4.1 Singularity Indication and Local Dimension

In this subsection we construct tools for studying the sup-port and singularity of s(µ) in (4.1). First let us showhow to estimate the location of the support. Notice thatthe proof regarding Lebesgue’s decomposition of measures(given e.g. in [30]) is constructive in nature. Namely, the sin-gular part s(µ) with respect to Lebesgue measure in (4.1) isobtained by restricting µ to a specific measurable set A0Dwith m(A) = 0 so that

(s(µ))(E) = µ(A(E) (4.4)

for any measurable E 0 D. Naturally instead of A any setA = A 1 B with m(B) = 0 can be used in (4.4). Next weexplain how A can be recovered uniquely up to a zero m-measurable set.

Assume µ is a positive and bounded Borel measure onD and define using the Lebesgue measure m an auxiliarymeasure . = µ + m. We see that the mapping

f 4#2

Df dµ

is bounded linear functional in L2(D,.), i.e., the L2 inte-grable functions on D with respect to the measure .. Bythe Riesz representation theorem there exists / ! L2(D,.)

such that2

Df dµ =

2

Df/d. (4.5)

for all f ! L2(D,.). It can be shown that 0 " /(z) " 1 in.-a.e. Now by setting

A = {z !D |/(z) is defined and /(z) = 1} (4.6)

one obtains s(µ) by defining (s(µ))(E) = µ(A ( E) (forproof see [30, Thm. 6.10]). Let us denote

/r (z) = µ(B(z, r))

µ(B(z, r)) + m(B(z, r))

for any z ! D and r > 0. The function /r is a fundamen-tal tool for us since computation of µ(B(z, r)) can be doneefficiently using exposure data.

Lemma 2 For .-almost all z !D it holds that

/(z) = limr#0

/r (z). (4.7)

Proof By [21, Cor. 2.14] .-almost every point z ! D is aLebesgue point of / and at these points

/(z) = limr#0

1.(B(z, r))

2

B(z,r)/(x)d.(x).

Therefore by substituting f = %B(z,r) into (4.5) we get

/(z) = limr#0

µ(B(z, r))

.(B(z, r))= lim

r#0

µ(B(z, r))

µ(B(z, r)) + m(B(z, r))

(4.8)

for .-almost all z !D. !

Hence by estimating the limit limr#0 /r (z) we havemeans to evaluate whether z ! A. We call / the singular-ity indication function.

Measure theory also contains ways to classify the lo-cal behaviour of a measure. One of them is by computingso-called (lower) local dimension or local Hölder exponent[12] of µ at z !D. It is defined by

dimloc µ(z) = lim infr#0

log(µ(B(z, r)))

log r(4.9)

when there exists no r > 0 such that µ(B(z, r)) = 0. Whensuch an r exists we define dimloc µ(z) =$. It is easy tosee that 0" dimloc µ(z)"$. For further information aboutlocal dimension see [12].

Later in Sect. 5.5 we show how to use exposure data forcomputing /(z) and dimloc µ(z). Before that let us now dis-cuss some connections between these concepts. As was dis-cussed earlier we want to locate the points z ! A. It is theninteresting to study what the local dimension at these pointsis. It turns out that the convergence speed of /r (z) plays arole here.

Lemma 3 Let us assume that z !D satisfies /(z) > 0. Thenwe have that

dimloc µ(z) = 2& lim supr#0

log(1&/r (z))

log r" 2.

Proof Let r > 0 be fixed. The definition of /r (z) impliesthat

µ(B(z, r)) = /r (z)

1&/r (z)m(B(z, r)).

J Math Imaging Vis

The logarithm of the right-hand side simplifies

log0

/r (z)

1&/r (z)m(B(z, r))

1

= log(/r (z)) + log(m(B(z, r)))& log(1&/r (z)).

Since limr#0log(/r (z)))

log r = 0 and limr#0log(m(B(z,r)))

log(r) = 2we have that

dimloc µ(z) = 2 + lim infr#0

& log(1&/r (z))

log(r)

which proves the equality. Here dimloc µ(z) " 2 sincelog(1&/r (z))

log r is positive for r < 1. !

Corollary 1 Assume that z ! D satisfies (4.7) anddimloc µ(z) < 2. Then we have limr#0 /r (z) = 1.

Proof There exists a sequence {rj }$j=1, rj > 0, such that

dimloc µ(z) = limj#$

log(µ(B(z, rj )))

log(rj )

and

log(µ(B(z, rj )))

log(rj )" 2& ' (4.10)

for some ' > 0. Inequality (4.10) yields that

µ(B(z, rj ))' r2&'j

and by assumption we have

/(z) = limj#$

/rj (z)' limj#$

r2&'j

r2&'j + -r2

j

= 1.

This yields the claim. !

Note that conditions limr#0 /r (z) = 1 and dimloc µ(z) <

2 are not equivalent for a fixed z ! D. It is possible to con-struct a bounded measure µ (e.g. with suitably weightedsum of point masses) in such a way that at some point z !Dboth limr#0 /r (z) = 1 and dimloc µ(z) = 2 hold. We donot discuss this discrepancy further in this paper. Instead,in Sect. 5.5 we simply approximate the set A by sets

Ad = {z !D | dimloc µ(z) < d}

for some 0 < d < 2.

4.2 Photon-Based Edge Detection

We discuss a gradient-based approach for detecting edges ininfinite photographs. Furthermore, we study how to differ-entiate images given in exposure form.

Let us assume that e is a regular exposure and µ =M([e]). Furthermore, assume that L is a linear first orderdifferential operator (e.g. L = 0

0x ) and Lµ is a boundedmeasure. Notice that this assumption is not implicated byour model. Hence we are considering only a subset of allpossible images here.

Now we would like to locate the singular part of the totalvariation of Lµ using the method proposed above. In prac-tise, to be able to fully use the exposure data e we have torelax this objective a bit.

Denote by &r ! C$(D) a sequence of positive functionssuch that limr#05&r ,&6L2(D) = &(0) for any & ! C$(D).Furthermore, in the following µ/&' denotes the convolutionon torus D = T2.

Now |&' /Lµ|-measure of a ball B(z, r) can be computedas follows:

|&' /Lµ|(B(z, r))

= 5|(L&') /µ|,%B(z,r)6= sup

h!C(D)|h|"1

5(L&') /µ,%B(z,r)h6

= suph!C(D)|h|"1

5µ, (L&') / (%B(z,r)h)6. (4.11)

Now if our data is SNe the duality pairing reduces to

5µSNe, (L&') / (%B(z,r)h)6

= 1N

!

1"j"N

g(!j )(L&') / (%B(z,r)h)(zj ).

The supremum in (4.11) can be approximated by usinga sufficiently large number of randomly chosen functionsh1, h2, . . . , hT ! C(D) and taking a maximum. Thus wehave presented a method for approximating the value of|&' /Lµ|(B(z, r)). We postpone the implementation of thismethod to a further study.

5 Computational Methods

5.1 Monte Carlo Simulation of the Illumination Map

The construction of the illumination map in Definition 6 wasbased on a random construction and the strong law of largenumbers. In the same way we are able to simulate it stochas-tically.

Assume that the image has the form µ = fµdm with apiecewise continuous and bounded function fµ satisfying7

D fµdm = 1. Further, assume that the point values fµ(x, y)

are available for any (x, y) ! D. Let us describe a simpleapproach for simulating the J first photons based on theMetropolis-Hastings algorithm [11, 22].

J Math Imaging Vis

Fig. 5 Pixel images formed by the exposure w ! E defined in Exam-ple 6. The images in the top row are at resolution n = 4 and have size16 , 16. The images in the bottom row are at resolution n = 6 andhave size 64, 64. The scale is the same in all eight images: black cor-responds to zero and white corresponds to pixel value 1.2 (or greater).Note that convergence is slower when resolution parameter n is greater

(1) Pick an initial photon location (x1, y1) arbitrarily. Set#= 1.

(2) Choose a candidate location (x, y) by drawing x andy from the uniform probability density on the interval[0,1).

(3) We need to decide whether to accept the candidate asthe next photon in the sequence or to repeat the previousphoton. The decision is based on the number

1 = min5

1,fµ(x, y)

fµ(x#, y#)

6.

Draw a random number r from the uniform probabilitydistribution on the interval [0,1]. If 1 ' r then acceptthe candidate:

(x#+1, y#+1) = (x, y),

otherwise reject the candidate and repeat the previousphoton:

(x#+1, y#+1) = (x#, y#).

(4) If # <J then set #7 #+ 1 and go to 2. Otherwise stop.

The exposure data e = ((xj , yj ),!(1))$j=1 produced by

Metropolis-Hastings algorithm satisfies M(e) = µ. Resultsconcerning properties of the chain can be found e.g. in [27].

Let us test the above algorithm in the simple cases of aconstant measure fµ 2 1, a completely white image, andmodify the result to construct a middle gray constant imageas well.

Example 6 We construct an exposure w ! E by drawingpoints zj from the uniform probability distribution on D andsetting wj := (zj ,!

(1)) where g(!(1)) = 1. See Fig. 5 for theresulting approximate pixel images In(w,J ).

Fig. 6 Pixel images formed by the exposure w) ! E defined in Ex-ample 6. The images in the top row are at resolution n = 4 and havesize 16 , 16. The images in the bottom row are at resolution n = 6and have size 64, 64. The scale is the same in all eight images: blackcorresponds to zero and white corresponds to pixel value 1 (or greater).Note that convergence is slower when resolution parameter n is greater.Compare to Fig. 5

We modify w to cover a constant gray value as well. As-sume that g(!(0)) = 0 and set !j = !(1) for every odd valueof j and !j = !(0) for every even value of j . Then we con-struct an exposure w) ! E by w)j = (zj ,!j ). Then everyother photon in w) delivers unit energy and every other de-livers zero energy. The result represents a constant imagewith gray level, as seen in Fig. 6.

Example 7 Let us turn to a nontrivial example. Figure 7(a)shows a monochrome 512 , 512 pixel digital image of astrawberry. The above algorithm is then used to constructa finite exposure with J = 1500000 photons. Approximatepixel images In(e, J ) are shown in top right of Fig. 7 forn = 6 and in top right of Fig. 8 for n = 8.

5.2 Halftoning with Black Photons

Given a grayscale pixel image, the goal of halftoning is toproduce a set of black dots on a white surface in such a waythat to the human eye the pixel image and the set of dotsappear as similar as possible. Halftoning is routinely usedin graphical printing industry, and there are plenty of quickand simple methods available for halftoning a given imageso that the local average gray levels of the pixel image andthe halftoned rendition match very closely. However, a typ-ical problem is that if the black dots are distributed too sys-tematically, then disturbing patterns are perceived by humanobservers. There are several ways around this in the litera-ture [1, 17], including stochastic halftoning methods.

Our image model based on sequences of photons can beapplied to halftoning by using the Metropolis-Hastings algo-rithm to the inverted pixel image, and interpreting photonsas black dots of ink. The result is a novel halftoning method.

J Math Imaging Vis

Fig. 7 (a) Monochrome digital photograph of a strawberry. (b) Ap-proximate pixel image I6(e,1500000) of size 64, 64 computed fromrandom exposure of (a). (c) Approximate pixel image I6(c,1500000)of the outline curve. (d) Approximate pixel image of an exposure mix-ing the exposures used to create (b) and (c). This is a low resolutionpixel representation of the underlying cartoon image

Let us describe the stochastic halftoning algorithm. As-sume given a grayscale image G)) with N , N pixels andnonnegative pixel values G))(i, j)' 0. Invert the picture bydefining

G) := max1"i,j"N

G))(i, j)&G)).

Further, introduce a normalized image by defining

G := G)

1N2

4Ni=1

4Nj=1 G)(i, j)

.

Then we can interpret G as a piecewise constant simplefunction defined on D and satisfying2

DG(z)dz = 1,

where dz denotes Lebesgue measure on D. Then we canuse the Metropolis-Hastings algorithm of Sect. 5.1 for con-structing a sequence of J photons distributed according tothe absolutely continuous measure G.

Once the J photons have been chosen using the Metropo-lis-Hastings algorithm we need to print the black dots. How-ever, there are a couple of problems. First, what should bethe area of the black dot printed in place of a photon? Sec-ond, if we print a constant size dot at the location of eachphoton, the repeated photons appear as the same as just onephoton, which is not the correct result.

Fig. 8 (a) Monochrome digital photograph of a strawberry. (b) Ap-proximate pixel image I8(e,1500000) of size 256 , 256 com-puted from random exposure of (a). (c) Approximate pixel imageI8(c,1500000) of the outline curve. (d) Approximate pixel image ofan exposure mixing the exposures used to create (b) and (c). This is ahigher resolution pixel representation of the underlying cartoon imageas compared to Fig. 7

We solve the above problems approximately as follows.Let us compute the average gray level of the original imageG)) by setting

G))) := G))

max1"i,j"N G))(i, j)

and calculating

a :=2

DG)))(z)dz = 1

N2

N!

i=1

N!

j=1

G)))(i, j).

We interpret the number a as follows: a bitmap image on Dhas the same average gray level than G)) if the area of blackcolor is a and the area of white color is 1 & a. The termbitmap image means here that all pixel values of the imageare either zero (interpreted as black) or one (interpreted aswhite).

The outcome of our halftoning algorithm is a bitmap im-age H of size 8N , 8N pixels. For the halftoning idea to makesense we must take 8N > N , for example 8N = 4N would do.

In the locations of photons appearing only once we drawa black dot of area a/J . In the location any photon repeated$ > 1 times we draw a black dot of area $a/J . The resultis a halftoned version of G)). Let us still clarify what wemean by “dot” here. In the context of the 8N , 8N bitmapimage H we reproduce dots as square areas of black pixels.

J Math Imaging Vis

Of course, the fact that a pixel in the image H has area 8N&2

implies that we cannot reproduce dots of exactly the rightsize, but the larger 8N we take the better we can approximatethe correct area.

See Fig. 10 for a halftoned gradient and Fig. 11 for ahalftoned image of a key.

Fig. 9 (a) Monochrome digital photograph of a strawberry. (b) Ap-proximate pixel image I6(e

),6000000) of size 64,64 computed fromthe blurred exposure. (c) Approximate pixel image I8(e

),6000000) ofsize 256, 256 computed from the blurred exposure

5.3 Modelling Cartoon Images Including Outlines

It is possible to combine photon-based representations ofcontinuous images and curves in the same exposure. Thisway we can introduce a novel representation of cartoon im-ages consisting of piecewise continuous grayscale imagesand outlines of shapes. The outlines will be printed as anti-aliased curves as narrow as possible in the chosen resolution.

Figures 7 and 8 show an image of a strawberry togetherwith outline at low and high resolutions, respectively.

5.4 Photon-Based Gaussian Blurring

Convolving an image with a Gaussian kernel is one of thebasic image processing operations. Here we show how it canbe implemented for an image given in exposure form.

Our probabilistic approach is based on the following ob-servation. Let X and Xj , j = 1,2,3, . . . , be identically

Fig. 10 Halftoning of agradient using the stochasticalgorithm described in Sect. 5.2.The size of the image matrix is128, 8192, and 107 photonswere used

J Math Imaging Vis

Fig. 11 Halftoning of agrayscale pixel image using thestochastic algorithm describedin Sect. 5.2. (a) The originalpixel image of size 512, 512.The halftoned binary image (notshown in full size) has size1013 , 1013. The number ofphotons used is 107.(b, c) Details of the larger pixelimage, sizes 52, 52. (d, e)Parts of the halftoned binaryimage corresponding to (b, c)

distributed independent random variables taking values inD whose probability density function is $. Moreover, lete = ((zj ,!j ))

$j=1 ! E0 be a regular exposure and define a

(D,R)N-valued random variable by

f (+) := ((zj + Xj(+),!j ))$j=1. (5.1)

We recall that above and in the lemma below addition“+” is the group operation on D = R2/Z2; in other words,(x1, x2) + (y1, y2) = (z1, z2) ! D when x# + y# ! {z# + n |n ! Z} for #= 1,2.

Lemma 4 Let e = ((zj ,!j ))$j=1 ! E0 and µ = M(e) and

define random variable f by (5.1). Then almost surely f isa regular exposure, i.e., f ! E0, and the limit image mapsatisfies M(f ) = $ /µ.

Proof Let D = Dn(k,#) be an arbitrary pixel such that n !N and 1" k " 2n and 1" #" 2n. As the spectral sensitivityfunction g is bounded, Kolmogorov’s variance criterion foraverages [15, Cor. 4.22] yields almost surely

limJ#$

1J

J!

j=1

g(!j )%D(zj + Xj(+))

= limJ#$

1J

J!

j=1

E.g(!j )%D(zj + Xj)

/

= E

9

: limJ#$

1J

J!

j=1

g(!j )%D(zj + Xj)

;

<

= E (µ(D&X))

= ($ /µ)(D), (5.2)

J Math Imaging Vis

Fig. 12 A line measuresupported on a circle with graybackgroud. (a) The originalpixel image (J -snapshot).(b) The absolutely continuouspart of Lebesguedecomposition. (c) The singularpart of Lebesguedecomposition. (d) The inversevalue of local dimensioncomputed from singular part (c)

where we have used the Lebesgue dominated convergencetheorem and the knowledge that e is regular. Furthermore,the number of all pixels Dn(k,#) is countable and hence(5.2) holds simultaneously for all pixels almost surely. Con-sequently, the limit limJ#$ In(f (+), k,#) exists and satis-fies almost surely

limJ#$

In(f (+), k,#) = ($ /µ)(Dn(k,#))

for all n !N and 1" k " 2n and 1" #" 2n. Thus the outermeasure M(f ) coincides almost surely with $ /µ, and theclaim follows. !

We demonstrate the convolution by constructing an ex-posure e) and randomly translating each photon in the aboveexposure e = ((xj , yj ),!j ) computed from the strawberryimage as follows:

e) =00

xj + 1100

randn, yj + 1100

randn

1,!j

1,

where randn stands for an independent realization of a nor-mally distributed random variable with mean zero and unit

variance. Here the addition is performed on R and the resultis identified to T by periodizing the real axis.

See Fig. 9 for snapshots of e).

5.5 Singularity Extraction

Let us now tackle the question about how to decompose ex-posure data in the sense of Lebesgue discussed in Sect. 4.Assume that e is a regular exposure. Next we formulate aheuristic method for obtaining two exposures ea

n and esn such

that their induced measures approximate with respect to n

the absolutely continuous and singular part of M([e]). Inpractise we never have the full data of e, but instead a trun-cated data SNe for some finite N . Therefore the two keyparameters of our algorithm are n and N and during the fol-lowing scheme we assume

(1) that N = N(n) is always large with respect to n in thesense of (3.7) so that

dn(e, SNe)8 0

J Math Imaging Vis

Fig. 13 Image of sunlightarriving from behind a tree.(a) The original pixel image.(b) The absolutely continuouspart of Lebesguedecomposition. (c) The singularpart of Lebesguedecomposition. (d) The inversevalue of local dimensioncomputed from singular part (c)

(2) and that we can approximate

µe(B(x, r))8 µSNe(B(x, r)) = 1N

!

zj!B(x,r)1"j"N

g(!j ).

Let us now assume that n is given. Denote by znk,# the

center point of pixel Dn(k,#) for each 1 " k " 2n and1 " # " 2n and recall !(0) was chosen so in Definition 1that g(!(0)) = 0.

Remark 1 We noted earlier before Lemma 3 that the con-vergence speed of /r (z) plays a role when computingdimloc µ(z) for a given point z ! D. In fact, it is straight-forward to formulate Lemma 3 in the following form usingan auxiliary function h(r) that satisfies h : R+ # [0,1] andlimr#0 h(r) = 0. Assume that function /r (z) satisfies

/r (z)' 1& h(r). (5.3)

Then it follows that

dimloc µ(z)" 2& lim supr#0

log(h(r))

log r. (5.4)

For example,

(i) if h(r) = r2&a , a < 2, condition (5.3) implies thatdimloc µ(z)" a and

(ii) if h(r) = &(log r)&1, then for point z which has aneighbourhood not intersecting supp(s(µ)) the condi-tion (5.3) is not valid.

Using the observations above we formulate the algorithmto decompose Sne to ea

n and esn:

(1) Compute value of

/Nrn

(znk,#) =

µSNe(B(znk,#, rn))

µSNe(B(znk,#, rn)) + m(B(zn

k,#, rn))

for all 1" k " 2n and 1" #" 2n, where rn = 2&n.(2) Define /N

rneverywhere by linearly interpolating the

value at z from three closest neighbours znk,#.

(3) For all photons ej = (zj ,!j ) ! e, j = 1, . . . ,N do thefollowing(a) Compute t = /N

rn(zj ).

(b) If t > 1 & h(rn) then set ean,j := (zj ,!

(0)) andesn,j := (zj ,!j ).

J Math Imaging Vis

(c) If t " 1&h(rn) then set ean,j := (zj ,!j ) and es

n,j :=(zj ,!

(0)).(4) Set ea

n = (ean,j )

Nj=1 and es

n = (esn,j )

Nj=1.

It is straightforward to see now that ean and es

n are re-constructed in such a way that M([SNe]) = M([ea

n]) +M([es

n]). By this construction we do not necessarily approx-imate the whole singular part of M([e]). In fact, as can beseen from Remark 1 we do not expect to see singularities forwhich the convergence speed of /r is slower as h.

We have implemented the algorithm above in Figs. 12and 13 using convergence speed h(rn) = &(log rn)

&1.Moreover, after separating the absolutely continuous andsingular part we have computed the local dimension of es . InFig. 12 the original image has been a line measure on a cir-cle and gray background. A snapshot at resolution n = 10 isgiven on the top. The decomposition is shown in the middle.Furthermore, the inverse of local dimension dimloc µe(x)

is plotted on the bottom. Here we have used N = 2 · 106

photons. In Fig. 13 the original image a photo of sunlightarriving from behind a tree. A snapshot taken at resolu-tion n = 10, and the decomposition and the inverse localdimension are plotted as in Fig. 12. For the sunlight picturewe have used N = 107 photons. We have used the inversevalue of local dimension in order to visualize the absolutelycontinuous background as black and the singular points asbright spots.

Acknowledgements The authors thank Eero Saksman for valuablecomments, especially for pointing out the benefits of using the con-cept of Banach limit. During part of the preparation of this work,S.S. worked as professor at the Department of Mathematics of Tam-pere University of Technology. T.H. was supported by Emil Aaltonenfoundation and Graduate School of Inverse problems (Academy of Fin-land). The research work of all three authors was funded in part by theFinnish Centre of Excellence in Inverse Problems Research (Academyof Finland CoE-project 213476).

References

1. Adler, R.L., Kitchens, B.P., Martens, M., Tresser, C.P., Wu, C.W.:The mathematics of halftoning. IBM J. Res. Develop. 47(1), 5–15(2003)

2. Aubert, G., Kornprobst, P.: Mathematical Problems in ImageProcessing, Partial Differential Equations and the Calculus ofVariations, 2nd edn. Springer, New York (2006)

3. Aujol, J.-F., Aubert, G., Blanc-F’eraud, L., Chambolle, A.: Im-age decomposition into a bounded variation component. J. Math.Imaging Vis. 22(1), 71–88 (2005)

4. Boiko, D.L., Gunther, N.J., Brauer, N., Sergio, M.,Niclass, C., Beretta, G.B., Charbon, E.: A quantum im-ager for intensity correlated photons. New J. Phys. (2009).doi:10.1088/1367-2630/11/1/013001

5. Choi, H., Baraniuk, R.G.: Wavelet statistical models and Besovspaces. In: Nonlinear Estimation and Classification. Lecture Notesin Statist., vol. 171, pp. 9–29. Springer, New York (2003)

6. Falconer, K.: Fractal Geometry. Wiley, Chichester (1990)

7. Fraysse, A.: Generic validity of the multifractal formalism. SIAMJ. Math. Anal. 39(2), 593–607 (2007)

8. Gariepy, R., Ziemer, W.: Modern Real Analysis. PWS Publishing,Boston (1995)

9. Garnett, J.B., Le, T.M., Meyer, Y., Vese, L.A.: Image decom-positions using bounded variation and generalized homogeneousBesov spaces. Appl. Comput. Harmon. Anal. 23(1), 25–56 (2007)

10. Haddad, A., Meyer, Y.: An improvement of Rudin-Osher-Fatemimodel. Appl. Comput. Harmon. Anal. 22(3), 319–334 (2007)

11. Hastings, W.K.: Monte Carlo sampling methods usingMarkov chains and their applications. Biometrika (1970).doi:10.1093/biomet/57.1.97

12. Pap, E. (ed.): Handbook of Measure Theory, vol II. Elsevier, Am-sterdam (2002)

13. Harte, D.: Multifractals. Theory and Applications. ChapmanHall/CRC, Boca Raton (2001)

14. Jensen, H.W.: Realistic Image Synthesis Using Photon Mapping.A K Peters, Natick (2001)

15. Kallenberg, O.: Foundations of Modern Probability, 2nd edn.Springer, New York (2002)

16. Kelley, J.L.: General Topology. Van Nostrand, Toronto (1955)17. Lau, D.L., Arce, G.R.: Modern Digital Halftoning. CRC Press,

Boca Raton (2001)18. Li, Y., Dai, C.: Multifractal dimension inequalities in a probability

space. Chaos Solitons Fractals 34(2), 213–223 (2007)19. Lieu, L.H., Vese, L.A.: Image restoration and decomposition

via bounded total variation and negative Hilbert-Sobolev spaces.Appl. Math. Optim. 58(2), 167–193 (2008)

20. Mallat, S., Zhong, S., et al.: Wavelet transform maxima and mul-tiscale edges. In: Ruskai, M.B., et al. (eds.) Wavelets and TheirApplications. Jones and Bartlett, Boston (1991)

21. Mattila, P.: Geometry of Sets and Measures in Euclidian Spaces:Fractals and Rectifiability. Cambridge University Press, Cam-bridge (1995)

22. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H.,Teller, E.: Equations of state calculations by fast computing ma-chine. J. Chem. Phys. 21, 1087–1091 (1953)

23. Meyer, Y.: Oscillating Patterns in Image Processing and in SomeNonlinear Evolution Equations, The Fifteenth Dean JacquelineB. Lewis Memorial Lectures, University Lecture Series, vol. 22.American Mathematical Society, Providence (2001)

24. Mukherjea, A., Pothoven, K.: Real and Functional Analysis. PartA: Real Analysis. Plenum, New York (1984)

25. Mumford, D., Gidas, B.: Stochastic models for generic images.Q. Appl. Math. 59 (2001)

26. Mumford, D., Shah, J.: Optimal approximations by piecewisesmooth functions and associated variational problems. Commun.Pure Appl. Math. 42(5), 577–685 (1989)

27. Nummelin, E.: MC’s for ’MCMC’ists. Int. Stat. Rev. 70(2), 215–240 (2002)

28. Olsen, L.: Multifractal geometry. In: Fractal Geometry and Sto-chastics, II (Greifswald/Koserow, 1998). Progr. Probab., vol. 46,pp. 3–37. Birkhäuser, Basel (2000)

29. Rao, K.C.: Functional Analysis. Alpha Science Intl Ltd (2002)30. Rudin, M.: Real and Complex Analysis. McGraw-Hill, New York

(1987)31. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based

noise removal algorithms. Physica D 60, 259–268 (1992)32. Simoncelli, E.P.: Statistical modeling of photographic images. In:

Bovik, A. (ed.) Handbook of Video and Image Processing. Acad-emic Press, New York (2005)

33. Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On ad-vances in statistical modeling of natural images. Special issue onimaging science (Boston, MA, 2002). J. Math. Imaging Vis. 18(1),17–33 (2003)

J Math Imaging Vis

34. Starck, J.-L., Elad, M., Donoho, D.L.: Image decomposition viathe combination of sparse representations and a variational ap-proach. IEEE Trans. Image Process. 14(10), 1570–1582 (2005)

35. Turiel, A., Parga, N.: The multifractal structure of contrast changesin natural images: from sharp edges to textures. Neural Comput.12, 763–793 (2000)

36. Turiel, A., Del Pozo, A.: Reconstructing images from their mostsingular fractal manifold. IEEE Trans. Image Process. 11(4), 345–350 (2002)

37. Vese, L., Osher, S.: Modeling textures with total variation mini-mization and oscillating patterns in image processing. J. Sci. Com-put. 19, 553–572 (2003)

Tapio Helin received the Master’sdegree in 2004 from Helsinki Uni-versity of Technology, Finland, andcontinued there as a graduate stu-dent of Prof. Matti Lassas. He sub-mitted his dissertation in 2009 andis currently employed by JohannRadon Institute for Computationaland Applied Mathematics in Linz,Austria. His research interests in-clude statistical inverse problems,image processing and mathematicalmethods in adaptive optics.

Matti Lassas is a Professor of Ap-plied Mathematics in the Depart-ment of Mathematics and Statisticsat the University of Helsinki, Fin-land. He received the PhD degree in1996 from University of Helsinki.Since then he has conducted re-search at the Rolf Nevanlinna In-stitute, Finland, the University ofWashington, US, the Mathemati-cal Sciences Research Institute, US,and Helsinki University of Tech-nology. His research interests areinverse problems and imaging, inparticular, analytic and differential

geometrical methods on these subjects. He has been awarded Calderonprize of International Inverse Problems Association in 2007, Vaisalaprize of Finnish Academy of Science and Letters in 2004, and RolfNevanlinna doctoral thesis award in 1995. He is a senior scientist ofthe Finnish Centre of Excellence in Inverse Problems Research andmember of boards of the Finnish Inverse Problems Society and theFinnish Mathematical Society, and the executive committee of InverseProblems International Association.

Samuli Siltanen is a Professor ofIndustrial Mathematics in the De-partment of Mathematics and Sta-tistics at the University of Helsinki,Finland. He received the PhD de-gree in 1999 from Helsinki Univer-sity of Technology, and since thenhe has worked as JSPS PostdoctoralFellow in Gunma University, Japan,as R&D Scientist at GE Healthcare,and as professor of mathematics atTampere University of Technology,Finland. His research interests in-clude inverse problems, electricalimpedance imaging, X-ray tomog-

raphy with sparse data and image processing. He is a senior scientistof the Finnish Centre of Excellence in Inverse Problems Research andmember of Board of Governors of Finnish Inverse Problems Society.