Physica A 391 (2012) 2215–2224

Contents lists available at SciVerse ScienceDirect

Physica A

journal homepage: www.elsevier.com/locate/physa

Physical consideration of an image in image restoration usingBayes’ formulaHirohito KiwataDivision of Natural Science, Osaka Kyoiku University, Kashiwara, Osaka 582-8582, Japan

a r t i c l e i n f o

Article history:Received 19 August 2011Received in revised form 12 November2011Available online 1 December 2011

Keywords:Image restorationBayes’ formulaMarkov random fieldPotts modelQ -Ising model

a b s t r a c t

We consider image restoration by Bayes’ formula and investigate the relationship betweenan image and a prior probability from the following two viewpoints: hyperparameterestimation and the accuracy of a restored image. The Q -Ising model is adopted asa prior probability in Bayes’ formula. Not the Q -Ising energy, but the Potts energyplays an important role in the hyperparameter estimation. From the viewpoint ofthe hyperparameter estimation, the relationship between a natural image and a priorprobability is characterized through the Potts energy and magnetization of an image. ThePotts energy and magnetization of an image are defined by a set of pixels’ state of animage. The closer to the average Potts energy and magnetization over a prior probabilitythe Potts energy and magnetization of a natural image is, the closer to the true value ofa hyperparameter the estimated value of a hyperparameter from a degraded image is.For the accuracy of a restored image, the image which has a smaller Q -Ising energy isbetter restored by Bayes’ formula composed of the Q -Ising prior. The consideration for therelationship between an image and a prior probability is expected to be valid for a morecomplicated prior probability.

© 2011 Elsevier B.V. All rights reserved.

1. Introduction

When digital data is transmitted through a channel, the data is corrupted by noise. In order to infer the original data fromthe corrupted one, redundancy is added to the original data, and the original data with the redundancy is transmitted. Theoriginal data is retrieved from the corrupted data using the redundancy. In the present study, we consider the transmissionof digital images without the redundancy. An original image has to be inferred from only a degraded image. Instead of theredundancy, we use the a priori knowledge of images. For the inference from a degraded image, a probabilistic method hasbeen widely adopted for several decades. Bayes’ formula is very useful in such a case [1–6]. Bayes’ formula is composed ofthe product of a conditional probability and a prior probability. The conditional probability depends on the properties of achannel. The prior probability is characterized by an original image. Natural images are composed of smooth monochromeareas. A digital image is composed of a set of pixels. Taking account of the above characteristic of natural images, we adopta prior probability that gives a larger probability for adjoining two pixels in the same color than that in different colors. Forthe sake of simplicity consider a binary image for a while. Each pixel shows black or white, and so a black or white pixel canbe represented by the bit value 0 or 1, respectively. We assign the bit value 0 (1) of a pixel the down spin −1 (up spin +1).As a result, an image composed of a set of pixels is represented by a set of Ising spins on a lattice. A typical prior probabilitythat shows the above property is given by a ferromagnetic Ising-model prior. The Ising prior shows that a probability foradjoining two spins in the same state is larger than that in different states. For a set of Ising spins generated by the Ising prior,

E-mail address: kiwata@cc.osaka-kyoiku.ac.jp.

0378-4371/$ – see front matter© 2011 Elsevier B.V. All rights reserved.doi:10.1016/j.physa.2011.11.025

2216 H. Kiwata / Physica A 391 (2012) 2215–2224

adjoining two spins tend to turn in the same direction. Consider a global phase of a set of spins. The Ising prior depends ontemperature, which controls the probability. When the temperature is high, each spin takes a value of +1 or −1 randomly.In the low-temperature case, large masses of spins with up spin or down spin are constituted. When the temperature drops,the phase of a set of spins changes markedly at the critical temperature.

The conditional probability as well as the prior probability is rewritten in terms of a set of spins. Thus, a posteriorprobability formulated by Bayes’ formula is expressed in terms of a set of spins. The problem of image restoration by Bayes’formula is replaced with that of a spin model. In the case of the Ising prior, the posterior probability corresponds to theGibbs’ distribution of a ferromagnetic Ising model in random external magnetic fields. The state at which the posteriorprobability achieves its maximum corresponds to a state inferred by Bayes’ formula. We have to find the maximum point ofthe posterior probability. Since a vast number of pixels constitute an image, it is an extremely difficult task to find the statemaximizing the posterior probability. The posterior probability is rewritten in terms of a set of spins, and the maximizationof the posterior probability corresponds to the minimization of the energy of the ferromagnetic Ising model with randomfields. Then approximate methods developed in statistical mechanics are helpful for treating the problem. In particular, thesimulated annealing and mean-field theory have been widely used to treat the problem [1,7–10].

The conditional probability and the temperature in the prior probability are called hyperparameters, which are importantquantities in image restoration. The accuracy of image restoration by Bayes’ formula depends on the values of thehyperparameters [11]. Although finding the proper hyperparameters is of great significance for image restoration, we haveto estimate the hyperparameters from a received degraded image. To estimate the hyperparameters, the maximization of amarginalized likelihood function is used. However, the computational requirement for the marginalization is exponentiallylarge and we have to resort to simulations or approximate methods to implement the idea [12–16]. In our previous study,we reported a method of estimating the hyperparameters [17]. For artificial images generated by a prior probability, thehyperparameters are estimated accurately without any approximation. For natural images that are not generated by a priorprobability, the hyperparameters are not estimated accurately. It is doubtful that a prior probability is appropriate for naturalimages. In the present study, we find the condition under which the prior probability is applicable to natural images. Thedemonstration of the validity of the prior probability originates from derivation of the hyperparameters. The method ofestimating the hyperparameters is as follows. We defined the energy E and the magnetizationM of an image and found therelationships between E andM of an original image and those of a degraded one. The hyperparameters are estimated usingthese relationships. Since we estimate them through E and M , the hyperparameters are obtained accurately for an originalimage, of which E andM are close to average E and M over the given prior probability.

For grayscale images, the Potts model, which is an extension of a binary valued spin system to amultivalued spin system,is useful for a prior probability. The Potts prior takes the flatness of grayscale images into consideration. In this case, thehyperparameters are also estimated through average E and averageM over the Potts prior. In the present study, we adopt aQ -Ising prior and investigate image restoration by Bayes’ formula composed of the Q -Ising prior [18,19]. For the Ising priorand Potts prior, the above E andM have physical significance. For theQ -Ising prior, the above E does not coincidewith that ofthe Q -Ising model. We investigate the applicability of our method of estimating the hyperparameters to a prior probabilityexcept the Ising prior and Potts prior. If our method of estimating the hyperparameters is applicable to the Q -Ising prior,the method is applicable to any prior probability such as a prior probability that emphasizes particular lines.

For artificial images generated by a given prior probability, the hyperparameters are estimated accurately. However, theaccuracy of the image restoration byBayes’ formula for artificial images is not always higher than that for natural images. Thisresult seems to contradict the result of the hyperparameter estimation. In the present study, we assume that the accuracyof the image restoration by Bayes’ formula depends on a prior probability that generates an original image. The suppositionshows that the higher the prior probability that generates an image is, the more accurately an original image is restoredfrom a degraded image. Since the prior probability depends on the energy of the original image, the accuracy of the imagerestoration depends on the energy of the original image. In the present study, we adopt the Q -Ising prior and verify thesupposition for the Q -Ising prior. For comparison, we investigate the dependence of the accuracy on the energy of bothPotts and Q -Ising models. It is found that the accuracy of the image restored by Bayes’ formula with the Q -Ising prior showsa strong correlation with the energy of the Q -Ising model of an original image, that is, with the Q -Ising prior probabilitygenerating an original image.

2. Model

In this section, we introduce the Q -Ising model, which is the extension of the Ising model to a multiple-valued spinsystem. A set of Q -valued spins is represented by x = {xi}. The index i denotes an i-th lattice site in the spin system andcorresponds to a pixel index of an image. The value of xi ranges from 0 to Q − 1. The Q -valued spin corresponds to a pixelwith Q gray levels. Although we consider an eight-valued spin system henceforth, the following consideration is applicableto other valued spin systems. When Q = 2, the Q -Ising model reproduces the results of the Ising model. The energy of theQ -Ising model is expressed by

EQ -Ising =

⟨i,j⟩

(xi − xj)2, (1)

H. Kiwata / Physica A 391 (2012) 2215–2224 2217

Fig. 1. Artificial images generated by Q -Ising prior (a) and Potts prior (b). The thermodynamic beta β in the Q -Ising prior is 0.6. The thermodynamic betaβ in the Potts prior is 1.341. The value of the critical point of the Potts prior is 1.34245 · · ·.

where the subscript ⟨i, j⟩ means the sum of a pair of adjoining sites. When states of all the spins are identical, the systemhas the lowest energy. For comparison, the energy of the Potts model is also shown as

EPotts = −

⟨i,j⟩

δ(xi, xj), (2)

where δ(a, b) is the Kronecker delta [20]. For the Potts model, if the values of all the spins are identical, the system has thelowest energy like the Q -Ising model. However, there is a great difference in the tendency of adjoining spins between theQ -Ising and Potts models. The Q -Ising model has a tendency for adjoining spins to have similar values of the states. ThePotts model has no such tendency. If the state of a spin is different from the state of a neighboring spin, the energy of thePotts model is independent of the state of a neighboring spin. The prior probability of the Q -Ising model is expressed by

PQ -Ising(x | β) =exp

−βEQ -Ising

x′

exp−βEQ -Ising

=

exp−β

⟨i,j⟩

(xi − xj)2

x′

exp−β

⟨i,j⟩

(x′

i − x′

j)2 , (3)

where the subscript x′ in the denominatormeans the sumover all the states of all the spins, andβ is an inverse of temperaturethat controls the prior probability. Fig. 1 shows artificial images generated by the Q -Ising prior and Potts prior. The imagesconsist of 512 × 512 spins (pixels). The values of β in the Q -Ising prior and Potts prior are 0.6 and 1.341, respectively. Thecritical β of the Potts prior is exactly estimated at 1.34245 · · ·. The image generated by the Potts prior shows a snapshot inthe vicinity of the critical point. For the image generated by the Q -Ising prior, states of the spins vary gradually in all places.Although the image generated by the Potts prior shows a sharp contrast, the image generated by the Q -Ising prior shows anobscure contrast.

Important quantities characterizing the macroscopic properties of the Q -Ising model are average energy and magneti-zation. The average energy per site is defined by

εQ -Ising =1N

⟨EQ -Ising⟩ =1N

x

⟨i,j⟩

(xi − xj)2PQ -Ising(x | β), (4)

where N is the total number of the spins. The average energy of the Potts model over the Q -Ising prior is defined by

εPotts =1N

⟨EPotts⟩ = −1N

x

⟨i,j⟩

δ(xi, xj)PQ -Ising(x | β). (5)

Although formula (5) does not describe physical characteristics of the Q -Isingmodel, it is an important quantity for estimat-ing the hyperparameters. Fig. 2 shows formulae (4) and (5) as functions of β . The summation for all the states of all the spinsis performed by aMonte Carlomethod for various lattice sizes such as 32×32, 64×64, 128×128, 256×256, 512×512 and1024 × 1024 [21,22]. The results do not depend on system size. These figures show that the average energy of the Q -Isingmodel and the average energy of the Potts model are monotonically decreasing functions of β .

The magnetization of the system is defined as follows: we define the number of spins of the state a as Na, where arepresents one of the following: 0, 1, . . . , 7. Arrange Na according to size as Na ≥ Nb ≥ Nc ≥ Nd ≥ Ne ≥ Nf ≥ Ng ≥ Nh.The magnetization of the image is defined as

M = Na + Nb + Nc + Nd − Ne − Nf − Ng − Nh. (6)

2218 H. Kiwata / Physica A 391 (2012) 2215–2224

a b

Fig. 2. Average energy of Q -Ising model (4) as a function of β (left) and of Potts model (5) (right).

Fig. 3. Average magnetization of Q -Ising model (7) as a function of β .

Formula (6) is the extension of the Isingmodel to amultiple-valued spin system. The averagemagnetization per site is givenby

m =1N

⟨M⟩ =1N

x

MPQ -Ising(x | β). (7)

Fig. 3 shows the magnetization as a function of β . The magnetization for various system sizes is calculated by a Monte Carlomethod. The results do not depend on the system size like the energy. The magnetization is a monotonically increasingfunction of β , and Fig. 3 shows that there is no critical point where the phase of the system markedly changes. For theQ -Ising prior, the system easily relaxes from an initial state to an equilibrium state.

3. Result

3.1. Hyperparameter estimation

In this subsection, we briefly review a method of estimating hyperparameters from a degraded image and apply themethod to the analysis of the Q -Ising prior [17]. Assume that a degraded image is generated from an original image bychanging the state of each spin (pixel) to one of the other states with the same probability p, which is independent of otherspins. The probability that a state of a spin is unchanged is 1 − (Q − 1)p. The probability p is one of the hyperparameters.When an original image is transmitted through a channel, the image is corrupted by the above degradation process. Therelationship between the magnetization of the original image and that of the degraded image is obtained as

M ′= (1 − Q p)M, (8)

H. Kiwata / Physica A 391 (2012) 2215–2224 2219

Fig. 4. Magnetization per site versus Potts energy per site. The dots indicate formula (7) versus formula (5) through β as the parameter. The squaresindicate themagnetization per site versus the Potts energy per site of twenty-three natural images. The numerical values in the vicinity of the square showp∗ estimated from the degraded images.

where the prime means the magnetization of the degraded image. Similarly, the relationship between the Potts energy ofthe original image and that of the degraded image is obtained as

QE ′

Potts + 2N = (1 − Q p)2(QEPotts + 2N). (9)

Dividing the square of formula (8) by formula (9), we obtain

M ′2

QE ′

Potts + 2N=

M2

QEPotts + 2N. (10)

The quantityM2/(QEPotts +2N) is invariant with respect to the transmission. The left-hand side of formula (10) is evaluatedfrom the degraded image. In order to estimate p, we carry out the following procedure. We replace the quantities EPotts andM on the right-hand side of formula (10) with the average values ⟨EPotts⟩ = NεPotts and ⟨M⟩ = Nm, respectively. If theoriginal image is generated by the Q -Ising prior, the Potts energy and magnetization of the artificial image are accuratelyexpressed byNεPotts andNm. The average εPotts andm are functions ofβ . The dependences of εPotts andm onβ are determinedfrom simulation data by curve fitting. As a result, the right-hand side of formula (10) is a function of β . The value of β isadjusted to equalize the right-hand side of formula (10) with the left-hand side of formula (10). To distinguish the truehyperparameters from the estimated hyperparameters, we put asterisks against the estimated hyperparameters such as p∗

and β∗. The numerical value of β∗ is estimated by the bisection method. If β∗ is obtained, the probability p∗ is estimatedusing

p∗=

1Q

1 −

M ′

Nm(β∗)

, (11)

which is derived from formula (8). We degrade Fig. 1(a) using noise with p = 0.05 and estimate the hyperparameters fromthe degraded image. The values of p∗ and β∗ are 0.049579 and 0.597485, respectively, and these values are very close top = 0.05 and β = 0.6, respectively.

Since the hyperparameters p and β are estimated from the degraded image through εPotts and m, the hyperparametersare accurately estimated when the original image is generated by the Q -Ising prior. On the other hand, EPotts and M ofa natural image do not always agree with ⟨EPotts⟩ and ⟨M⟩, respectively, because a natural image is not generated by theQ -Ising prior. Fig. 4 shows the magnetization per site versus the Potts energy per site. The dots indicate m versus εPottsthrough β as a parameter. The curve composed of dots is different from that of the Potts prior or Ising prior. Generally,the shape of the curve differs between prior probabilities. The squares in Fig. 4 represent the magnetization per site versusthe Potts energy per site of twenty-three natural images. The twenty-three natural images are shown in the Appendix. Eachimage consists of 512×512 pixels. These natural images are degraded by noisewith p = 0.05.We estimate p∗ fromdegradedimages by the above method. The numerical values in Fig. 4 show p∗ estimated from the degraded images generated by theselected original images. The numerical values of the estimated p∗ show that the closer the magnetization per site and Pottsenergy per site of the original image to m and εPotts, the closer the estimated p∗ to the true value p. From the viewpoint ofthe hyperparameter estimation, a prior probability, which produces the magnetization and Potts energy of a given originalimage, represents the image well.

2220 H. Kiwata / Physica A 391 (2012) 2215–2224

a

c

b

Fig. 5. For twenty-three images, energy of Potts model versus Hamming distance and energy of Q -Ising model versus Hamming distance. Thehyperparameters p and β are fixed at 0.02 and 0.65 in (a), 0.03 and 0.65 in (b) and 0.04 and 0.65 in (c), respectively.

3.2. Accuracy of image restoration

In this subsection, we discuss the relationship between the accuracy of image restoration using Bayes’ formula and anoriginal image. When the Potts energy and magnetization of an original image are close to the average Potts energy andaverage magnetization over a given prior probability, the estimated p∗ is close to the true p. Following the hyperparameterestimation, we consider image restoration by Bayes’ formula. Bayes’ formula is expressed by the product of a conditionalprobability and a prior probability. We denote a set of pixel states of an original image and a degraded image by x = {xi}and y = {yi}, respectively. The conditional probability incorporating the degradation process is expressed as

P(y | x, p) =

i

p1 − δ(xi, yi)

+

1 − (Q − 1)p

δ(xi, yi)

, (12)

where the product is taken over all the pixels (lattice sites). Using Bayes’ formula, a posterior probability is expressed as

P(x | y, p, β) =P(y | x, p)PQ -Ising(x | β)zP(y | z, p)PQ -Ising(z | β)

, (13)

where the Q -Ising prior is adopted as a prior probability. A set of pixel states of a restored image is determined by thestates that maximize the posterior probability formula (13) on the condition that y is given. We define x∗

= {x∗

i } asstates maximizing formula (13). The maximization of the posterior probability (13) with respect to x requires a largenumber of computations. Therefore, we have to resort to approximate methods such as simulated annealing or mean-field approximation. In the present study, we adopt simulated annealing to maximize formula (13). Consider twenty-threenatural grayscale images. The twenty-three natural images are shown in the Appendix. Each image is composed of 512×512

H. Kiwata / Physica A 391 (2012) 2215–2224 2221

pixels and each pixel takes eight grades. These natural images are degraded by the conditional probability formula (12) withp = 0.02, 0.03 and 0.04.We spent a sufficiently long time performing the simulated annealing. If we lower the temperaturesufficiently slowly, the qualitative results of the image restoration do not depend on the annealing schedule.

To evaluate the accuracy of the image restoration, we define the Hamming distance as

d(x, x∗) =1N

i

1 − δ(xi, x∗

i ), (14)

where the sum is taken over all the pixels (sites). A restored image with smaller Hamming distance is more accuratelyrestored. For the artificial image generated by the Q -Ising prior, the hyperparameters are estimated accurately from adegraded image, but the image restoration of a degraded image is not always operating effectively. We suppose that whenthe prior probability generating a given image is high, the image restoration by Bayes’ formula works well. The priorprobability generating an image is evaluated by substituting states of pixels into formula (3). In other words, when thedegraded image is restored by Bayes’ formula with the Q -Ising prior, the accuracy of the image restoration is determinedby the Q -Ising energy (1) of an image. Since the image with a small Q -Ising energy has a large Q -Ising prior probability,the image with a small Q -Ising energy is restored well by Bayes’ formula. Fig. 5 shows the Potts and Q -Ising energy oftwenty-three images versus the Hamming distance between the original images and the images restored by the simulatedannealing. The left and rightwindows in these figures show the cases of the Potts andQ -Ising energies of the original images,respectively. The value β of the Q -Ising prior in formula (13) is fixed at 0.65. We adjust the value of p in formula (13) to thatused in the degradation process. The qualitative results of the image restoration are the same as those for the other β values.For comparison, the horizontal distances between the maximum andminimum values of the Potts and Q -Ising energies areadjusted to be identical in appearance in these figures. The data of both the Potts and Q -Ising energies of the images showthat a larger energy of the original images leads to a larger Hamming distance, which corresponds to the inaccuracy of therestored images. However, the dispersion of the data points for the Potts energy is larger than that for the Q -Ising energy.To ensure our supposition, we investigate the Hamming distance for original images generated by the Q -Ising prior (3) withβ = 0.30 ∼ 1.10. Fig. 6 shows theQ -Ising energy of the original images versus the Hamming distance. Thewhite circles andblack squares indicate the Hamming distance versus the Q -Ising energy of the natural images in Appendix and the artificialimages generated by the Q -Ising prior, respectively. We can see there is remarkable agreement among them.

We conclude that the correlation between the Q -Ising energy of an original image and the Hamming distance is strongerthan the correlation between the Potts energy of an original image and the Hamming distance. Although the Q -Ising energyis similar in quantity to the Potts energy, as shown in Fig. 2, there is a large difference in an evaluation of the accuracy of theimage restoration. These results support our supposition that a large prior leads to an accurate image restoration.

4. Summary and discussion

The image restoration by probabilistic methods has attracted much interest for several decades. To infer an originalimage from a degraded one, Bayes’ formula has been adopted widely. Although Bayes’ formula uses the a priori knowledgeof an image, the relationship between an image and a prior probability has not been investigated sufficiently so far. In thepresent study,we investigated the relationship between an image and a prior probability from the following two viewpoints:hyperparameter estimation and the accuracy of image restoration. We found the relationships between the energy andmagnetization of an original image and those of a degraded image. The energy and magnetization are defined using a set ofpixel states of an image. The hyperparameters are estimated using the above relationships. For grayscale images, the definedenergy agrees with the energy of the Potts model. When we adopt the Potts prior, the energy has a physical significance. Inthe present study, we adopt the Q -Ising prior. For the Q -Ising prior, the method proposed in our previous study has beenshown to be applicable to the estimation of the hyperparameters. In order to estimate the hyperparameters, the maximummarginal likelihood estimation has been employed. The marginal likelihood is defined by

P(y | p, β) =

x

P(y | x, p)PQ -Ising(x | β). (15)

The marginal likelihood formula (15) is maximized at p∗ and β∗, which are estimated values of the hyperparameters. Theconditions that maximize formula (15) lead to the following simultaneous conditions [19]:

i

x

1 − δ(xi, yi)

P(x | y, p∗, β∗) = (Q − 1)p∗, (16)

⟨i,j⟩

x

(xi − xj)2P(x | y, p∗, β∗) =

⟨i,j⟩

x

(xi − xj)2PQ -Ising(x | β∗). (17)

The left-hand side of formula (16) averages the number of pixels, whose state is changed by noise, over a posteriorprobability. Formula (17) shows the relationship between the Q -Ising energy averaged over a posterior probability andthat averaged over a prior probability. In the maximum marginal likelihood estimation, the energy of a prior probabilityis an important quantity for estimating the hyperparameters. Our method of estimating the hyperparameters employs the

2222 H. Kiwata / Physica A 391 (2012) 2215–2224

a b

c

Fig. 6. For twenty-three images (white circles) and artificial images generated by Q -Ising prior with β = 0.30 ∼ 1.10 (black squares), energy of Q -Isingmodel versus Hamming distance. The hyperparameters p and β in the posterior probability are fixed at 0.02 and 0.65 in (a), 0.03 and 0.65 in (b) and 0.04and 0.65 in (c), respectively.

relationship between the Potts energy of an original image and that of a degraded image. This is the marked differencebetween our method and the maximum marginal likelihood estimation. Since it is difficult to calculate the average overa posterior probability and a prior probability in formulae (16) and (17), we have to rely on an approximate method ofestimating the hyperparameters. From a practical viewpoint, our method is useful for estimating the hyperparameters,because the estimation can be performed without any approximations. Moreover, the Potts energy is an important quantityfor estimating the hyperparameters. The closeness between natural images and artificial images generated by a priorprobability is measured by the Potts energy and magnetization of images. Although we adopt the Q -Ising prior in thepresent study, it is expected that our method is applicable to a more complicated prior probability with line field. Sincethere are other relationships besides those of the Potts energy and magnetization, hyperparameters other than p and β canbe estimated using our method.

We also investigated the relationship between an image and a prior probability from the viewpoint of the accuracy ofimage restoration by Bayes’ formula. The Q -Ising prior is adopted as a prior probability and a degraded image is restored byBayes’ formula. The accuracy of a restored image depends on the prior probability of an original image. We compared theHamming distance as a function of the Potts energy of an original image with the Hamming distance as a function of theQ -Ising energy of an original image. The smaller the Potts or Q -Ising energies are, the more accurately a degraded image isrestored by Bayes’ formula. The correlation between the Hamming distance and the Q -Ising energy of an original image isstronger than that between the Hamming distance and the Potts energy of an original image. This result demonstrates thatthe larger prior probability of an original image leads to more accurate image restoration. If we adopt a prior probability haslarger value for a given original image, the degraded image is restored more accurately by Bayes’ formula with the givenprior probability.

H. Kiwata / Physica A 391 (2012) 2215–2224 2223

Fig. 7. Twenty-three natural images of an eight-step grayscale.

Appendix

See Fig. 7.

2224 H. Kiwata / Physica A 391 (2012) 2215–2224

References

[1] S. Geman, D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(1984) 721–741.

[2] J. Besag, On the statistical analysis of dirty pictures, J. Roy. Statist. Soc. Ser. B 48 (1986) 259–302.[3] J. Marroquin, S. Mitter, T. Poggio, Probabilistic solution of Ill-posed problems in computational vision, J. Amer. Statist. Assoc. 82 (397) (1987) 76–89.[4] J.M. Pryce, A.D. Bruce, Statistical mechanics of image restoration, J. Phys. A 28 (3) (1995) 511–532.[5] H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction, Oxford University Press, 2001.[6] K. Tanaka, Statistical-mechanical approach to image processing, J. Phys. A: Math. Gen. 35 (37) (2002) R81–R150.[7] G.L. Bilbro, W.E. Snyder, S.J. Garnier, J.W. Gault, Mean field annealing: a formalism for constructing GNC-like algorithms, IEEE Trans. Neural Netw. 3

(1992) 131–138.[8] J. Zhang, The mean field theory in EM procedures for Markov random fields, IEEE Trans. Signal Process. 40 (1992) 2570–2583.[9] K. Tanaka, T. Morita, Cluster variation method and image restoration problem, Phys. Lett. A 203 (1995) 122–128.

[10] S. Stramaglia, A. Refice, L. Guerriero, Statistical mechanics approach to the phase unwrapping problem, Physica A 276 (2000) 521–534.[11] H. Nishimori, K.Y.M. Wong, Statistical mechanics of image restoration and error-correcting codes, Phys. Rev. E 60 (1999) 132–144.[12] S. Lakshmanan, H. Derin, Simultaneous parameter estimation and segmentation of Gibbs random fields using simulated annealing, IEEE Trans. Pattern

Anal. Mach. Intell. 11 (1989) 799–813.[13] J. Zhang, J.W.Modestino, D.A. Langan,Maximum-likelihood parameter estimation for unsupervised stochasticmodel-based image segmentation, IEEE

Trans. Image Process. 3 (1994) 404–420.[14] Z. Zhou, R.M. Leahy, J. Qi, Approximatemaximum likelihood hyperparameter estimation for Gibbs priors, IEEE Trans. Image Process. 6 (1997) 844–861.[15] R.A. Molina, A.K. Katsaggelos, J. Mateos, Bayesian and regularization methods for hyperparameter estimation in image restoration, IEEE Trans. Image

Process. 8 (1999) 231–246.[16] J. Inoue, K. Tanaka, Dynamics of the maximum marginal likelihood hyperparameter estimation in image restoration: gradient descent versus

expectation and maximization algorithm, Phys. Rev. E 65 (2001) 016125.[17] H. Kiwata, Hyperparameter estimation in image restoration, J. Phys. A 41 (2008) 332004.[18] J. Inoue, D.M. Carlucci, Image restoration using the Q -Ising spin glass, Phys. Rev. E 64 (2001) 036121.[19] K. Tanaka, J. Inoue, D.M. Titterington, Probabilistic image processing by means of the Bethe approximation for the Q -Ising model, J. Phys. A: Math.

Gen. 36 (2003) 11023–11035.[20] F.Y. Wu, The Potts model, Rev. Modern Phys. 54 (1982) 235–268.[21] K. Binder, D.W. Heermann, Monte Carlo Simulation in Statistical Physics: An Introduction, Springer-Verlag, 1992.[22] M.E.J. Newman, G.T. Barkema, Monte Carlo Methods in Statistical Physics, Clarendon Press, 1999.