Journal of Structural Biology 166 (2009) 144–155

Contents lists available at ScienceDirect

Journal of Structural Biology

journal homepage: www.elsevier .com/locate /y jsbi

Structure enhancement diffusion and contour extraction for electron tomographyof mitochondria

Carlos Bazán a,*, Michelle Miller b,1, Peter Blomgren b,1

a Computational Science Research Center, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1245, USAb Department of Mathematics & Statistics, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-7720, USA

a r t i c l e i n f o

Article history:Received 15 July 2008Received in revised form 7 February 2009Accepted 13 February 2009Available online 28 February 2009

PACS:68.37.-d07.05.Pj87.63.lm

Keywords:Electron tomographyImage processingAnisotropic nonlinear diffusionBilateral filter

1047-8477/$ - see front matter � 2009 Elsevier Inc. Adoi:10.1016/j.jsb.2009.02.009

* Corresponding author. Fax: +1 619 594 2459.E-mail addresses: carlos.bazan@sdsu.edu (C. Bazá

com (M. Miller), blomgren.peter@gmail.com (P. Blom1 Fax: +1 619 594 6746.

a b s t r a c t

The interpretation and measurement of the architectural organization of mitochondria depend heavilyupon the availability of good software tools for filtering, segmenting, extracting, measuring, and classify-ing the features of interest. Images of mitochondria contain many flow-like patterns and they are usuallycorrupted by large amounts of noise. Thus, it is necessary to enhance them by denoising and closing inter-rupted structures. We introduce a new approach based on anisotropic nonlinear diffusion and bilateralfiltering for electron tomography of mitochondria. It allows noise removal and structure closure at certainscales, while preserving both the orientation and magnitude of discontinuities without the need forthreshold switches. This technique facilitates image enhancement for subsequent segmentation, contourextraction, and improved visualization of the complex and intricate mitochondrial morphology. We per-form the extraction of the structure-defining contours by employing a variational level set formulation.The propagating front for this approach is an approximate signed distance function which does notrequire expensive re-initialization. The behavior of the combined approach is tested for visualizing thestructure of a HeLa cell mitochondrion and the results we obtain are very promising.

� 2009 Elsevier Inc. All rights reserved.

1. Introduction

To date, it is firmly established that mitochondrial functionplays an important role in the regulation of apoptosis (Green andReed, 1998; Obeid et al., 2007). For instance, following a varietyof cell death signals, mitochondria exhibit early alterations in func-tion and morphologic changes, such as the opening of the perme-ability transition pore or mitochondrial megachannel (Franket al., 2001; Zamzami et al., 2007). There is also strong evidencethat defects in function may be related to many of the most com-mon diseases of aging, such as Alzheimer dementia, Parkinson’sdisease, type II diabetes mellitus, stroke, atherosclerotic heart dis-ease, and cancer. This is founded on the observation that mito-chondrial function undergoes measurable disturbanceaccompanied by drastic morphologic alterations in the presenceof these multisystem diseases (Frey et al., 2006; Munnich andRustin, 2001; Tandler et al., 2002).

Concurrent with the aforementioned conceptual advances therehas been a significant increase in the types of tools available to

ll rights reserved.

n), miller.michelle.m@gmail.gren).

study the correlation between mitochondrial structure and func-tion. Along with the now classic methods for isolating mitochon-dria and assaying their biochemical properties, there are new andpowerful methods for visualizing, monitoring, and perturbingmitochondrial function while assessing their genetic consequences(Marco et al., 2004; Pon and Schon, 2007). Electron tomography(ET) has allowed important progress in the understanding of mito-chondrial structure. This imaging technique currently provides thehighest three-dimensional (3D) resolution of the internal arrange-ment of mitochondria in thick Section (Perkins and Frey, 1999;Mannella et al., 1994). Nevertheless, the interpretation and mea-surement of the structural architecture of mitochondria dependheavily on the availability of good software tools for filtering, seg-menting, extracting, measuring, and classifying the features ofinterest (Frey et al., 2002; Perkins et al., 1997).

This paper is organized as follows: Section 2 presents an over-view of anisotropic nonlinear diffusion models in image processingin general, and in electron microscopy in particular. The level setmethod is also presented briefly as it is applied to the extractionof contours in images. In Section 3 we propose a new imagesmoothing and edge detection technique for electron tomographyas an extension to the model proposed by Bazán and Blomgren(2007). This approach employs a combination of anisotropic non-linear diffusion and bilateral filtering. In Section 4 we exhibit the

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 145

performance of the combined approach for visualizing the struc-ture of a HeLa cell mitochondrion with very promising results.We end this paper with a summary and discussion in Section 5.

2. Related work

In this section we present an overview of anisotropic nonlineardiffusion models in image processing and electron microscopy. Thelevel set method is also presented briefly as it is applied to theextraction of contours in images. We only review here the worksthat serve as background to the model we propose in Section 3.For an excellent and comprehensive survey of diffusion methodsin image processing we refer the interested reader to the book byWeickert (1998) and the references therein. Two very good refer-ences for the level set method are the books by Osher and Fedkiw(2003) and by Sethian (1999).

2.1. Nonlinear diffusion in image processing

Nonlinear diffusion is a very powerful image processing tech-nique used for the reduction of noise and enhancement of struc-tural features. It was first introduced to the image processingcommunity by Perona and Malik (1990) as an attempt to overcomethe shortcomings of linear diffusion processes, namely the blurringof edges and other localization problems. Their model accom-plishes this by applying a process that reduces the diffusivity inareas of the image with higher likelihood of belonging to edges.This likelihood is measured by a function of the local gradientj ru j. The model can be written as

ut �r � g ruj j2� �

� ru� �

¼ 0; ð1Þ

for t > 0, on a closed domain X, with the observed image as initialcondition uðx;0Þ ¼ u0ðxÞ, and homogeneous Neumann boundaryconditions hg � ru;ni ¼ 0, on the boundary oX. Here, n denotesthe outward normal to the domain’s boundary oX, and hg � ru;niindicates the inner product

RoXðg � ruÞ � n ds. In this model the diffu-

sivity has to be such that gðj ruj2Þ ! 0 when j ru j! 1 andgðj ruj2Þ ! 1 when j ru j! 0.

Notwithstanding the practical success of the Perona–Malikmodel, it presents some serious theoretical problems such as (i)ill-posedness (Nitzberg and Shiota, 1992; Weickert and Schnörr,2000); (ii) non-uniqueness and instability (Catté et al., 1992;Kichenassamy, 1997); (iii) excessive dependence on numericalregularization (Benhamouda, 1994; Fröhlich and Weickert, 1994).The last observation motivated an enormous amount of research to-wards the incorporation of the regularization directly into the par-tial differential equation (PDE), to avoid too much implicit relianceon the numerical schemes. A variety of spatial, spatio-temporal, andtemporal regularization procedures have been proposed over theyears (Alvarez et al., 1992; Catté et al., 1992; Cottet and Germain,1993; Weickert, 1994b, 1996b, 2001; Whitaker et al., 1993). InSection 2.2 we describe one of the variants to the Perona–Malikmodel that has been successfully used in electron microscopy,and in Section 3 we propose a new model based on a combinationof anisotropic nonlinear diffusion and bilateral filtering.

2.2. Anisotropic nonlinear diffusion in electron tomography

One way of introducing regularization to the Perona–Malikmodel is through anisotropic diffusion. Förstner and Gülch(1987) and Bigün and Granlund (1987) concurrently introducedthe matrix field of the structure tensor for image processing, andit is the basis for today’s anisotropic diffusion models. The mainidea behind these models is to construct the orthogonal systemof eigenvectors v1, v2, of the diffusion tensor Dr in such way that

they will reveal the presence of edges, i.e., v1krur (parallel) andv2 ? rur (perpendicular). Then one chooses appropriate (corre-sponding) eigenvalues that will allow smoothing parallel to theedges and avoid doing so across them. The main advantage ofanisotropic diffusion models over their isotropic counterparts isthat they not only account for the modulus of the edge detector,but also its directional information.

Cottet and Germain (1993) and Weickert (1994a, 1996a) wereamong the first authors to propose anisotropic nonlinear diffusionmodels for image processing. They devise a diffusivity matrix ofthe form

Dr ¼ v1 v2 v3½ �k1 0 00 k2 00 0 k3

264

375

vT1

vT2

vT3

264

375; ð2Þ

where the vectors vi are the eigenvectors of the image’s structuretensor Jr ¼ rur � ruT

r or its convolved version Jq ¼ Gq � Jr, whereur ¼ Gr � u and Gr, Gq are Gaussian kernels of width r, q, respec-tively. The parameters ki are functions of the eigenvalues,l1 P l2 P l3, of the structure tensor Jr (or Jq). Together, the eigen-values li and the eigenvectors vi, characterize the local structuralfeatures of the image u, within a neighborhood of size OðqÞ. Eacheigenvalue li reflects the variance of the gray level in the directionof the corresponding eigenvector vi, while each parameter ki con-trols the diffusion flux in the direction of vi and has to be chosencarefully.

Anisotropic nonlinear diffusion in electron microscopy wasintroduced by Frangakis et al. (1999, 2001). Based on the worksof Weickert(1998, 1999a,b), they chose the parameters ki to createa hybrid model that combines both edge enhancing diffusion (EED)and coherence enhancing diffusion (CED). EED is based on thedirectional information of the eigenvectors of the structure tensorJr, and its aim is to preserve and enhance edges. CED is based onthe directional information of the eigenvectors of the convolvedstructure tensor Jq, and is intended for improving flow-like struc-tures and curvilinear continuities. To combine the advantages ofEED and CED, the Frangakis–Hegerl model uses a switch basedon comparing an ad hoc threshold parameter to the local relationbetween structure and noise ðl1 � l3Þ. The threshold parameteris based on the mean value of ðl1 � l3Þ in a subvolume of the im-age containing only noise. EED is used when the differenceðl1 � l3Þ is smaller than the threshold parameter. When it is lar-ger, the model switches to CED for the enhancement of line-likestructure patterns. In a separate publication, Frangakis et al.(2001) applied the hybrid model to two-dimensional (2D) and 3Delectron tomography data and compared it with conventionalmethods as well as with wavelet transform filtering. Theyconcluded that the model exhibits excellent performance at lowerfrequencies, achieving considerable improvement in the signal-to-noise-ratio (SNR) that greatly facilitated the posterior segmenta-tion and visualization.

Fernández and Li (2003, 2005) proposed a variant to the modelby Frangakis et al. (1999, 2001) for ET filtering by anisotropic non-linear diffusion, capable of reducing noise while preserving bothplanar and curvilinear structures. They provided their model witha background filtering mechanism that highlights the interestingbiological structural features and a new criterion for stopping theiterative process. The Frangakis–Hegerl model diffuses unidirec-tionally along the direction of minimum change, v3, and efficientlyenhances line-like structures (where l1 � l2 � l3). It was arguedby Fernández and Li (2003) that a significant number of structuralfeatures from biological specimens resemble plane-like structuresat local scale. Therefore, they defined a set of metrics to discernwhether the features are plane-like, line-like, or isotropic. The met-rics they defined are:

146 C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155

P1 ¼l1 � l2

l1; P2 ¼

l2 � l3

l1; P3 ¼

l3

l1; ð3Þ

which satisfy 0 6 Pi 6 1, 8 i and P1 þ P2 þ P3 ¼ 1. In Eq. (3) l1, l2,and l3 are the eigenvalues of the convolved structure tensor Jq.These metrics are such that when P1 > P2 and P1 > P3, we have aplane-like structure; when P2 > P1 and P2 > P3, we have a line-likestructure; and when P3 > P1 and P3 > P2, we have an isotropicstructure. To achieve edge enhancing diffusion and linear and pla-nar enhancing diffusion, the Fernández–Li model uses the parame-ters in Eqs. (4) and (5), respectively, for the diffusion tensor:

k1 ¼ k2 ¼ g rurj j2� �

; ð4Þ

k3 ¼ 1;

k1 ¼ a;

k2 ¼a if l1 ¼ l2

aþ 1� að Þ exp �C2= l1 � l2

� �2� �

else;

8<: ð5Þ

k3 ¼a if l1 ¼ l3

aþ 1� að Þ exp �C3= l1 � l3

� �2� �

else;

8<:

with user-defined free parameters a (regularization constant, typi-cally set to 10�3) and C2 > 0 and C3 > 0. For the case of isotropicstructure, the model employs what Fernández and Li (2005) call‘background diffusion’ based on Gaussian smoothing.

The clever anisotropic nonlinear diffusion for the ET approachmentioned above also presents a practical drawback. Brox andWeickert (2002) argued that the linear structure tensor Jq derivedfrom Jr by smoothing each component using a Gaussian kernelwith standard deviation q, closes structures of a certain scale verywell and removes the noise appropriately. However, it only pre-serves orientation discontinuities and does not preserve magni-tude discontinuities, causing object boundaries to dislocate. Broxet al. (2005) argued that as soon as the orientation in the localneighborhood is not homogeneous, the local neighborhood in-duced by the Gaussian filter integrates ambiguous structure infor-mation. This information might not belong together and could leadto erroneous estimations. They proposed two alternatives to over-come this problem. The first solution involves the use of robust sta-tistics for choosing one of the ambiguous orientations (van denBoomgaard and van de Weijer, 2002). The second solution is toadapt the neighborhood to the data by using the Kuwahara–Nagaooperator (Bakker et al., 1999; Kuwahara et al., 1976; Nagao andMatsuyama, 1979). van den Boomgaard (2002) showed that theclassic Kuwahara–Nagao operator can be regarded as a ‘macro-scopic’ version of a PDE image evolution that combines linear dif-fusion with morphologic sharpening.

Based on the latter observation, Brox and Weickert (2002) pro-posed to address the aforementioned problem by replacing theGaussian convolution by a discontinuity preserving diffusionmethod. This is obtained by considering the structure tensor Jras an initial matrix field that is evolved under the diffusionequation

otuij �r � D rr

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiXk;l

u2kl

4

s0@

1A � rr

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiXk;l

u2kl

4

s0@

1A

T0@

1A � ruij

0@

1A ¼ 0;

ð6Þ

8 i; j, where the evolving matrix field uijðx; tÞ uses JrðxÞ as the initialcondition for t ¼ 0. The matrix DðAÞ ¼ TðgðkiÞÞ � TT is the diffusiontensor for A ¼ TðkiÞ � TT. The latter represents a principal axis trans-formation of A with the eigenvalues ki as the elements of a diagonalmatrix, diagðkiÞ, and the normalized eigenvectors as the columns of

the orthogonal matrix, T. For the diffusivity, g ¼ 1� expð�c=ðs=kÞ8Þ,for c > 0, and k the contrast parameter. In Eq. (6),rr denotes theroperator where Gaussian derivatives with standard deviation r areemployed. This approach tends to prevent boundary dislocationswhile keeping the desirable properties of the linear structure tensor.In Section 3 we introduce a new approach based on anisotropicnonlinear diffusion and bilateral filtering for electron tomography.It allows noise removal and structure closure at certain scales, whilepreserving both the orientation and magnitude of discontinuities.

2.3. Contour extraction using the level set method

Osher and Sethian (1988) developed a framework relying on aPDE approach for modeling propagating interfaces. These methodshave been applied to recover shapes of 2D and 3D objects from vi-sual data, as shown by Malladi et al. (1996). This modeling schememakes no a priori assumptions about the object’s shape and startswith an arbitrary function, propagating it in the direction normalto the curve along its gradient field with a certain speed, to recovershapes in the image. The level set formulation allows both forwardand backward motion of the initial front through the creation of ahigher dimensional function /ðx; tÞ, where the initial position ofthe front is embedded as the zero level set. The evolution of thefunction /ðx; tÞ is then linked to the propagation of the front itselfthrough a time-dependent initial value problem. Many implemen-tations (Osher and Fedkiw, 2003) of the level set method utilize azero (or initial) level set such that /ðx; t ¼ 0Þ ¼ �dðxÞ, where�dðxÞ is the signed distance to the position of the front. Through-out the evolution of the front, in order to avoid the formation ofshocks, very flat shapes, and/or very sharp shapes, a re-initializa-tion process is often used periodically to restore a signed distancefunction. The process of re-initialization can be complicated andexpensive. There is no simple way to determine how and whenthe level set function should be re-initialized to a signed distancefunction.

Li et al. (2005) presented a variational formulation whose prop-agating front is an approximate signed distance function yet doesnot require re-initialization. The variational energy functional con-sists of both an internal energy term that forces the level set func-tion to be kept as an approximate signed distance function, and anexternal energy term that drives the zero level set toward thesought object contours in the image. The total energy functionalis given by

E /ð Þ ¼ lP /ð Þ þ Eg;k;m /ð Þ: ð7Þ

The first term in the sum is the internal energy. It helps prohibitthe deviation of / from a signed distance function, where l > 0 isthe parameter controlling the effect of the penalizing the deviation.Pð/Þ is a metric that characterizes how close / is to a signed dis-tance function, e.g., /t ¼ signð/0Þð1� j r/ jÞ:

P /ð Þ ¼ 12

ZXr/j j � 1ð Þ2dx: ð8Þ

The second term in the sum of Eq. (7) is the external energyterm that moves the zero level curve toward the object boundaries.Given an image u we can define the following edge indicator func-tion where Gr is the Gaussian kernel with standard deviation r,g ¼ ð1þ j rGr � uj2Þ�1.

With this we can further specify our external energy term:

Eg;k;m /ð Þ ¼ kLg /ð Þ þ mAg /ð Þ ð9Þ

for constants: k > 0, m, and terms: Lgð/Þ ¼R

X gdð/Þjr/jdx, andAgð/Þ ¼

RX gHð�/Þdx where dð�Þ is the univariate Dirac function

and Hð�Þ is the Heaviside function. The energy term Lgð/Þ computesthe length of the zero level curve of / while Agð/Þ is the weightedarea on the interior of the zero level set and speeds up the curve

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 147

evolution. The coefficient m serves to control both the speed anddirection of the curve propagation and should be chosen appropri-ately depending on the relative location of the initial contour to theobject of interest. For an initial contour outside the object, m shouldbe a negative value so that the contours may shrink to the objectboundary; whereas, a positive value should be chosen for m if theinitial contour is inside the object so that the contours might ex-pand to the boundary.

The use of this energy functional completely eliminates theneed for the expensive re-initialization as the evolution of the levelset function is the gradient flow that minimizes the overall energyfunctional. The internal energy term maintains the level set func-tion as an approximate signed distance function while the externalenergy term drives the propagation. The evolution equation isdetermined using calculus of variations to differentiate E and set-ting its Gâteaux derivative equal to zero, yielding the steepest des-cent process for minimization of the functional E:

o/ot¼ l r2/�r � r/

r/j j

� �� �þ kd /ð Þr � g

r/r/j j

� �þ mgd /ð Þ: ð10Þ

3. Anisotropic nonlinear diffusion and bilateral filter inelectron tomography

In Section 2.2 we discussed the application of anisotropic non-linear diffusion in electron tomography. The approach used byFrangakis et al. (1999, 2001) and Fernández and Li (2003, 2005)is based on a hybrid EED/CED denoising mechanism that performsvery well on data containing low- to mid-frequency signal compo-nents. The technique greatly facilitates image enhancement forsubsequent segmentation and improved visualization of complexbiological specimens. In this section we propose a new imagesmoothing and edge detection technique for electron tomographyas an extension to the model proposed by Bazán and Blomgren(2007). This approach employs a combination of anisotropic non-linear diffusion and bilateral filtering. Jiang et al. (2003) introducedbilateral filtering for the removal of noise from biological electronmicroscopy data. They showed that bilateral filtering is a veryeffective mechanism for suppressing the noise in tomograms whilepreserving high resolution secondary structure features. To thebest of our knowledge, Bajaj et al. (2003), Bajaj and Yu (2005) werethe first ones to experiment with bilateral filtering coupled to anevolution driven anisotropic geometric diffusion PDE. They calledtheir method ‘volumetric anisotropic diffusion model’ and it isbased on a level set formulation that uses bilateral filtering as apre-filtering step in order to obtain more precise curvature infor-mation. For the enhancement of 2D features, the model requiresthe selection of two free parameters that control the diffusion rate,and a threshold switch associated with the image. Our proposedmodel aims at incorporating the best of both approaches, aniso-tropic nonlinear diffusion and bilateral filtering, in a single compu-tationally robust implementation that requires neither thechoosing of parameters nor the setting of threshold switches. Fur-thermore, this approach permits preservation of both orientationdiscontinuities and magnitude discontinuities, and avoids struc-ture dislocations. The model is equipped with the diffusion stop-ping criterion proposed by Bazán and Blomgren (2007), based onthe second derivative of the correlation between the noisy imageand the filtered image. (See Appendix A for details on this diffusionstopping criterion.)

3.1. Local structure analysis

The structure tensor of a 3D tomogram u is a symmetric positivesemidefinite matrix, Jr. This structure tensor is the most stable andreliable descriptor of the local structure of an image (Weickert,

1995). Similar to the approach in (Bazán and Blomgren, 2007),we propose using a refined estimate of the gradient of u at voxelx ¼ ðx; y; zÞ obtained by applying a bilateral filter in place of theGaussian kernel. Bilateral filtering is a technique for smoothingimages while preserving edges. The first application of this methodis attributed to Aurich and Weule (1995) and it was subsequentlyrediscovered by Smithm and Brady (1997) and Tomasi and Mandu-chi (1998). The basic idea underlying bilateral filtering is to com-bine domain and range filtering, thereby enforcing bothgeometric and photometric locality. The model can be expressed as

Gbf � u xð Þ ¼ 1W xð Þ

ZX

Grdn;xð ÞGrr u nð Þ;u xð Þð Þu nð Þdn; ð11Þ

with the normalization constant

W xð Þ ¼Z

XGrd

n;xð ÞGrr u nð Þ; u xð Þð Þdn: ð12Þ

Typically, Grdwill be a spatial Gaussian that decreases the influ-

ence of distant pixels, while Grr will be a range Gaussian that de-creases the influence of pixels uðnÞ with intensity values that arevery different from those of uðxÞ, e.g.,

Grd¼ exp � n� xj j2

2r2d

!; Grr ¼ exp � u nð Þ � u xð Þj j2

2r2r

!: ð13Þ

The parameters rd and rr dictate the amount of filtering applied inthe domain and the range of the image, respectively.

The new structure tensor is therefore Jbf ¼ rubf � ruTbf . The

image’s local structural features can be determined by performingthe eigen-analysis of the structure tensor Jbf where, as before, theeigenvalues provide the average contrast along the eigen-direc-tions, and the corresponding eigenvectors give the preferred localorientations. We will take advantage of the information providedby the structure tensor at each voxel x to devise a robust aniso-tropic structure enhancement model for 3D electron tomogramsof mitochondria. Images of mitochondria contain many flow-likepatterns and they are often perturbed by large amounts of noise(and may also include the artifacts related to the limited tiltrange). Thus, it is necessary to denoise and enhance them by clos-ing interrupted structures. To exploit the coherence and curvilin-ear continuity while connecting possible interrupted lines andsurfaces, we average the structure tensor Jbf over a region byapplying bilateral filtering in the form Jbf ¼ Gbf � Jbf . The direc-tional information is thereby averaged, although the structure ofthe region is still maintained by the discontinuity preservingbilateral filtering.

3.2. Diffusion tensor construction

The diffusion tensor, Dbf 2 R3�3, controls the smoothing acrossthe 3D tomogram. We define the diffusion tensor as a function ofthe structure tensor Jbf ,

Dbf ¼ v1 v2 v3½ �k1 0 00 k2 00 0 k3

264

375

vT1

vT2

vT3

264

375; ð14Þ

where vi are the structure tensor’s eigenvectors. The eigenvalues ofthe diffusion tensor, ki, define the strength of the smoothing alongthe eigen-directions, vi, and allow the application of different diffu-sion processes: (i) Linear diffusion or Gaussian smoothing is appliedwhen ki ¼ 1;8 i; (ii) Nonlinear diffusion is applied ifki ¼ gðj ruj2Þ;8 i; (iii) Anisotropic nonlinear diffusion can be appliedby setting the values ki so they reflect the image’s underlying localstructure.

As mentioned in Section 2.2, it is now common to use a hybridapproach that switches the diffusion process from EED to CED and

x

y

xu

yu

Fig. 1. Fragment of an image where an edge pixel’s gradient has components uy ¼ 0and 0 <j ux j6 1, depending on the gray values in the two regions.

148 C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155

vice versa, based on selected ad hoc thresholds. Switching to athird diffusion mode, Gaussian diffusion (GD), in areas where theimage becomes predominantly isotropic (based on another adhoc threshold) has also been suggested (Fernández and Li, 2003,2005). We propose to use the anisotropic diffusion process wherethe model switches among the three modes, EED/CED/GD, auto-matically based on information extracted locally from the signal.The model can be regarded as ‘structure enhancing diffusion’(SED), where the eigenvalues are defined as

k1¼g rubf

2� �;

k2¼g rubf

2� �if l1¼l2

g rubf

2� �þ 1�g rubf

2� �� �exp �C2= l1�l2

� �2� �

else;

8><>: ð15Þ

k3¼g rubf

2� �if l1¼l3

g rubf

2� �þ 1�g rubf

2� �� �exp �C3= l1�l3

� �2� �

else:

8><>:

The coherence measures ðl1 � l2Þ2 and ðl1 � l3Þ

2 are com-puted based on the eigenvalues of the averaged structure tensorJbf , and the parameters C2 and C3 act as thresholds such that struc-tures where ðl1 � l2Þ

2> C2 and ðl1 � l3Þ

2> C3 are regarded as

planar patterns, while structures where ðl1 � l2Þ2< C2 and

ðl1 � l3Þ2> C3 are regarded as linear patterns. In practice, the log-

ical ‘if l1 ¼ l2 then’ and ‘if l1 ¼ l3 then’ are unnecessary if we useexpð�Ci=ððl1 � liÞ

2 þ eÞÞ with i ¼ 2;3, for small e.In Eq. (15), gðj rubf j2Þ is a monotonically decreasing function

such as Perona–Malik’s

g rubf

�j2

� �¼ 1

1þ rubf

2=k2; ð16Þ

with k > 0 the typical contrast threshold parameter. There are sev-eral ways to set this parameter. Perona and Malik (1990) suggestedusing the idea presented by Canny (1986) and set k as a percentile,p, of the image gradient magnitudes at each iteration (they recom-mended the value p ¼ 90%.) A by-product of this approach is adecreasing k, which has an stabilizing effect on the diffusion process(Mrázek, 2001).

The advantages of the proposed definitions for the diffusiontensor’s eigenvalues become evident when we perform a 2D anal-ysis of the behavior of k1 and k2. Assume we are standing on anedge pixel of an image u 2 ½0;1�, as shown in Fig. 1. In this case,uy ¼ 0 and 0 <j ux j6 1, depending on the gray values in the two re-gions. The typical hybrid approach will use a chosen threshold toswitch between EED and CED (see Eqs. (4) and (5)), and set the dif-fusion tensor’s eigenvalues to either k1 ¼ gðj ruj2Þ or k1 ¼ a; andeither k2 ¼ 1 or k2 ¼ aþ ð1� aÞ expð�C=ðl1 � l2Þ

2Þ. Assuming forexample the values, a ¼ 10�3 and C ¼ 4:5� 10�4, and consideringthat ðl1 � l2Þ

2 ¼ ðu2x � u2

yÞ2 þ 4ðuxuyÞ2, we can plot k1 and k2 along

uy ¼ 0 and interpret the following (see Fig. 2):

(i) For low gradient j ux j! 0, we want the model to assume GDmode (recall that j uy j¼ 0). In this case, the hybrid EED/CEDmodel will assume EED mode where k1 � 1 and k2 ¼ 1. Sim-ilarly, the SED model will assume k1 � 1 and k2 � 1.

(ii) For mid-range gradients 0 <j ux j< 1, the values for k1 mustdecrease to prevent diffusing across the edge. In the hybridEED/CED model, k1 will decrease monotonically until it sud-denly switches from EED to CED where k1 ¼ a. In the SEDmodel, k1 will decrease monotonically to zero. Also, formid-range gradients, 0 <j ux j< 1, the values for k2 mustdecrease to prevent creating artificial edges. In the hybridEED/CED model, k2 ¼ 1 until the model switches sharplyfrom EED mode to CED mode where a 6 k2 6 1. In the SEDmodel, k2 decreases gradually.

(iii) For high gradients j ux j! 1, we want the model to assumelow values for k1 (no diffusion across the edge), and largevalues for k2 (full diffusion along the edge). In this case, aftera sudden jump, the hybrid EED/CED will be in CED mode andwill assume the correct values for k1 and k2. The SED modelwill continuously assume the correct values for k1 and k2.

(iv) The above rationale for the 2D case extends naturally to 3D.Also, if a voxel belongs to a linear structure, l1 � l2 � l3,then ðl1 � l2Þ ! 0 and consequently the termexpð�C2=ðl1 � l2Þ

2Þ ! 0, making the hard switch betweenplanar and linear structures unnecessary.

Before applying the proposed SED model to a 3D electron tomo-gram of a mitochondrion, we apply the model to a more familiar2D image and to a 2D slice of the electron tomogram of a HeLa cellmitochondrion below. The experiment demonstrates the excellentperformance of the proposed model, where the structures of theimages have been enhanced while most the noise has been re-moved. For the Clown image, we ran the four models BF, EED,CED, and SED to the points of maximum similarity between thenoise-free image and each of its filtered images. The similaritywas measured by the correlation coefficient between the noise-free image and each of the filtered images, corrðf ;uÞ. In Fig. 3 wecan observe that the SED model not only enhances the structuresof the images, but also removes sufficient noise for the filtered im-age to move closer to the noise-free images than the other filteredimages. For the 2D slice of the electron tomogram of a HeLa cellmoitochondrion, we ran the three models EED, CED, and SED totheir stopping-points (see Appendix A). We can see in Fig. 4 thatthe SED model allows for good preservation of the edges and thesmoothing of the background, without creating artificially en-hanced details.

4. Image acquisition and processing

4.1. Image acquisition

The electron tomogram employed in our experiments corre-sponds to a HeLa cell and was obtained from a 250 nm semi-thicksection across a mitochondrion expressing cytochrome c-GFP. Inthe interest of research not discussed here, apoptosis was inducedin the mitochondria with 100 lM etoposide for 15 h. The imagingoccurred before the release of cytochrome c or loss of membranepotential allowing the maintenance of normal mitochondrion pro-files; however, the treatment caused elongation of the crista junc-tions. The use of a semi-thick section is advantageous because itallows accurate depiction of the inner membrane topology of themitochondrion.

The microscope used was the FEI Tecnai 12 Transmission Elec-tron Microscope (TEM) with magnification set at 11,000. The EMtomography single-tilt series 3D reconstruction was obtained fromthe semi-thick sample by progressively tilting the specimen and

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1λ 1

CEDEED

xu0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

xu

λ 2

CEDEED

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

λ 1

SED

xu0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1λ 2

SED

xu

Fig. 2. Eigenvalues k1 and k2 for the EED, CED, and SED models. The hybrid EED/CED model switches sharply between the EED and CED curves based on an ad hoc threshold.

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 149

recording images using a Teitz 214 digital camera. The tilting wasconducted in increments of 2� over an angular range of ±60�; theangular range is limited by the geometry of the apparatus thatholds the sample. Once the tilt-series was collected on the digitalcamera, the IMOD Software Suite (Kremer et al., 1996) was usedto process the images and obtain the 3D reconstruction of the elec-tron tomogram.

3D models are constructed using the electron tomogram. The3D tomogram can be represented as a series of parallel sections,one pixel thick, of constant z. (Here, the image pixel size is1.27 nm.) Constructing the models then requires the tracing ofthe membrane profiles of the outer membrane, inner membrane,and cristae structures in each of many parallel sections of thetomogram. These tracings form a stack of membrane contours that,when input to a computer display program, create a 3D model thatcan be rotated and viewed at any angle Frey et al., 2002. The trac-ing of the mitochondrial structures in the tomogram is currentlydone manually. However, by applying the proposed variational le-vel set algorithm, the process is made less tracer-dependent.

4.2. Image smoothing and structure enhancement

After the 3D tomogram of the HeLa cell mitochondrion has beenreconstructed, we apply the algorithm described in Section 3 for theremoval of noise and the enhancement of the structural features.This step is critical for the posterior segmentation and extractionof the structure-defining contours. The problem to solve is

ut �r � Dbf � ru� �

¼ 0; on X� 0;1½ Þ;u x; 0ð Þ ¼ u0 xð Þ; on X;

Dbf � ru;n �

¼ 0 on oX� 0;1ð Þ;ð17Þ

where the diffusion tensor Dbf is given by Eq. (14), the eigenvectorsvi for i ¼ 1;2;3, are the eigenvectors of the structure tensor Jbf , andthe eigenvalues ki for i ¼ 1;2;3, are defined in Eq. (15). We applythe standard explicit finite difference scheme using central differ-ence to approximate the spatial derivatives, and forward differenceto approximate the time derivative. The condition for stability,assuming dx ¼ 1, is given by dt ¼ 1=6 (Weickert et al., 1998).

Fig. 3. Noise reduction using different techniques. (a) Test image: the Clown (grayscale (0,1), 256� 256 pixels. (b) Test image corrupted by additive Gaussian noise of zeromean and variance 0.01. (c) Bilateral filter, iterative implementation with rd ¼ 1 and rr ¼ 0:1, best corrðf ;uÞ ¼ 0:9787. (d) Edge enhancing diffusion, best corrðf ; uÞ ¼ 0:9782.(e) Coherence enhancing diffusion, best corrðf ; uÞ ¼ 0:9613. (f) Structure enhancing diffusion, best corrðf ;uÞ ¼ 0:9817. Note how the structures of the image have beenenhanced while most the noise has been removed by using the SED model.

Fig. 4. Noise reduction applied to an electron tomographic reconstruction of a HeLa cell mitochondrion. (a) x–y slice of the original 3D reconstruction. (b–d) Filtered versionsof the x–y slice using (b) edge enhancing diffusion, (c) coherence enhancing diffusion, and (d) structure enhancing diffusion. The SED model allows good preservation of theedges and smoothness of the background without creating artificially enhanced details.

150 C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 151

Figs. 5 and 6 show some results of the proposed approach ap-plied to the 3D electron tomogram of the HeLa cell mitochondrion.The proposed approach achieves excellent noise reduction whilepreserving the salient edge features. In order to facilitate theextraction of the structures, we synthetically enhance the contrastby applying the confidence connected segmentation algorithm

Fig. 5. Results of the proposed approach as a slice taken from the 3D electron tomogramafter applying the structure enhancing diffusion method. We observe the good denoisingedges. The filtered image facilitates the segmentation and posterior extraction of the str

Fig. 6. Results of the proposed approach as a slice taken from the 3D electron tomog‘confidence connected’ segmentation algorithm, and a fragment of the image’s contours

Fig. 7. (a) The ‘confidence-connected’ segmented image contains a large number of smapart of the mitochondrion of interest). (b) Labeling the connected components and filterinnumber of slices (as shown here) potentially unwanted structures are connected to thunwanted structures can easily be ‘clipped,’ the segmentation re-labeled and filtered an

(Meier et al., 1997). In this context, this simple region-growing seg-mentation method produces sufficiently good results for theextraction stage, but more flexible methods such as the watershedtechnique (Volkmann, 2002) or the Chan and Vese (2001) algo-rithm can easily be substituted. In order to effectively remove un-wanted small-scale features, including noise and extraneous

of the HeLa cell mitochondrion. (left–right) The noisy image and the filtered imagecapability of the proposed approach along with its excellent ability of preserving theucture’s contours.

ram of the HeLa cell mitochondrion. (left–right) The image segmented with theover which the structure tensor’s term k2v2 � vT

2 was superimposed.

ll-scale components which are not of interest (either noise, or cellular material notg out the small scale ‘noise’ yields a much cleaner segmentation; however, in a smalle outer membrane due to noise or limitations in the imaging process. (c) These

ew.

Fig. 8. (left) Results of extraction of the interior structures and outer membrane of a mitochondrion in an ET image. Algorithm was applied twice to the segmented image(Fig. 5) using parameters: k ¼ 5:0, l ¼ :04, s ¼ 5:0, m ¼ �25 for interior initial contour and m ¼ 10 for outer initial contour. Final contours were plotted on the original ETimage slice. (right) Fragment of the 3D rendering of the structural contours extracted from the mitochondrion image.

152 C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155

cellular material, we label the components in the segmentedFig. 7a, and keep the large-scale features corresponding to themitochondrion of interest. A typical slice in the series of imagesprocessed for this paper contained over 300 distinct components.In all but one slice were the components of interest contained inthe three largest labeled structures; in the ‘outlier’ case a smallpiece of the inner membrane was disconnected in the particularslice and was the 7th largest component. The result of filteringout the small scale features is shown in Fig. 7b and c; in the latterwe have ‘clipped’ external features which have become connectedto the outer membrane due to noise and limitations in the imagingprocess. After segmentation, the features are extracted using thelevel set approach described in Section 2.3.

4.3. Contour extraction

We adopt the formulation by Li et al. (2005) presented in Sec-tion 2.3 to extract the contours in the mitochondrion images.One significant advantage of this formulation is the liberty allowedin selecting the initial level set function. Traditionally, using levelset methods requires the initial level set to be a signed distancefunction /0 so that re-initialization can be applied. However, withthis need eliminated, a much simpler initial function may bedefined:

/0ðxÞ ¼�g; x 2 X0 � oX0

0 x 2 oX0

g; X�X0

8><>: ð18Þ

given arbitrary X0, a subset in the image domain X where oX0 is theset of points on the boundary of X0, and g > 0. For our implemen-tation g ¼ 4 is selected; however, most any constant would work.For the purposes here we use a Dirac function dðxÞ in Eq. (10) thatis slightly smoothed. We define the regularized Dirac functiond�ðxÞ as follows

d�ðxÞ ¼0; jxj > �1

2� 1þ cosðpx� Þ

� �; jxj 6 �

(ð19Þ

and utilize � ¼ 1:5 for our implementation.In implementing the proposed level set method, we carefully

selected both our time-step s and coefficient l to be safely withinthe range required for stability (sl < 1

4 as explained in Li et al.,2005), s ¼ 5 and l ¼ :04. The level set functions were initializedas the function /0 defined in Section 2.3 with g ¼ 4 using selectedregions X0. Fig. 8 shows the successful extraction of both the cristastructures and outer membrane of the 718� 763 pixel mitochon-

drion image. Identification of the interior structures was conductedseparately from the identification of the outer membrane due tothe required opposite direction of contour evolution for each. In or-der that the initial contour expand to identify the inner structuresm was chosen to be �25 and the evolution required 53 iterations,whereas for the second initial contour, used to shrink to identifythe outer membrane, m was chosen to be 10 and the evolution re-quired 38 iterations. These selections allowed accurate visualiza-tion of the boundaries of interest. Note that the algorithm wasrun on the image in Fig. 5 (lower-left) and the resulting contourshave been displayed on the original electron tomogram imageslice. Also note that even though the SED model and the segmenta-tion algorithm were run on the 3D tomogram, the contour extrac-tion is done on a slice-by-slice basis.

5. Summary and discussion

We have presented a multi-stage approach for extracting themitochondrial structures from electron tomograms. In particular,we apply the strategy to a 3D tomogram of a HeLa cellmitochondrion.

In the initial reconstruction, or noise reduction phase, we pro-pose a structure enhancing anisotropic nonlinear diffusion strat-egy: the local structure tensor Jbf is formed from the gradientinformation of a bilaterally smoothed version of the current image.In order to close gaps in structures caused by imaging limitations,the local structure tensor is further smoothed with a bilateral filter,forming a smoothed version of the structure tensor, Jbf . This helpsto preserve both orientation discontinuities and magnitude discon-tinuities. The eigenvectors vi, of the smoothed structure tensorform the basis for the diffusion tensor Dbf , where the eigenvaluesare prescribed so that there is a smooth interpolation, rather thana hard threshold switching of the diffusion characteristics betweenimage areas of differing structure properties.

After the noise reduction phase, we synthetically enhance thecontrast by applying the confidence-connected segmentation algo-rithm. The connected components of the segmented images are la-beled, and we discard the small scale components whichcorrespond to either noise or unwanted extra-mitochondrial struc-tures. Following which, structures are extracted using a variationallevel set formulation which includes a term that drives the level setfunction toward a signed distance function. This both simplifies theinitialization of the algorithm and removes the need for re-initialization.

The extracted contours are visualized in Fig. 8. The left panelshows the successful result of extracting both the outer membrane

0 5 10 15 20 25 30 35 40 45 50

0.84

0.86

0.88

0.90

0.92

0.94

0.96

0.98

1.00

Iterations

Cor

rela

tion

Correlation Behavior

corr ( f, u)corr (u0, u)

Fig. A-1. The correlation coefficient between the noise-free image and the filteredimage increases as the filtered image moves closer to the noise-free image. Whenthe measure reaches a peak it decreases as the filtered image moves slowly towardsa constant value. The correlation coefficient between the noisy image and thefiltered image decreases gradually from a value of 1.0 (perfect correlation), to aconstant value as the filtered image becomes smoother.

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 153

and the inner structures; in the right panel we show a 3D-render-ing of a ‘stack’ consisting of 25 extracted contours from 2D slices.The results are very encouraging. This computational approach ispotentially much faster, and is more robust than the hand-tracingof structures.

Acknowledgements

The authors thank Dr. Terry Frey and the San Diego State Uni-versity Mitochondria Research Group for their input and for pro-viding the images used in our computations. From this group,our special thanks go to Mei Sun and Mariam Ghochani. This workhas been supported in part by NIH Roadmap Initiative award R90DK07015.

Appendix A. Diffusion stopping criterion

Bazán and Blomgren (2007) proposed a new (very simple) diffu-sion-stopping criterion inspired by observation of the behavior ofthe correlation between the noise-free image and the filtered im-age, corrðf ;uÞ, and the correlation between the noisy image andthe filtered image, corrðu0;uÞ. Although the former measure is onlyavailable in experimental settings it helps validate the usefulnessof the latter. The nonlinear diffusion process starts from the ob-served (noisy) image, u0ðxÞ, and creates a set of filtered images,uðx; tÞ, by gradually removing noise and details from scale to scaleuntil, as t !1, the image converges to a constant value. Duringthis process the correlation between the noise-free image andthe filtered image increases as the filtered image moves closer tothe noise-free image. This behavior continues until it reaches apeak from where the measure decreases as the filtered imagemoves slowly towards a constant value. During the same processthe correlation between the noisy image and the filtered image de-creases gradually from a value of 1.0 (perfect correlation), to a con-stant value, as the filtered image becomes smoother (see Fig. A-1).By comparing both measures, we observe that as corrðf ;uÞ reachesits maximum (the best possible reconstructed image), the curva-ture of corrðu0; uÞ changes sign. This suggests that a good stoppingpoint of the diffusion process is where the second derivative ofcorrðu0;uÞ reaches a maximum. The performance of the stoppingcriterion can be observed in Fig. A-1 along with the reconstructedimages of Lena and the Clown Fig. A-2). We observe that the stop-

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Iterations

Nor

mal

ized

Cor

rela

tion

coef

fici

ent

Stopping Criterion

corr ( f, u) Catte−Lions−Morel−CollStopping Criterion

Fig. A-2. (left) Stopping criterion performance along with the reconstructed image of Lediffusion process after 18 iterations, while the proposed stopping criterion suggests to salong with the reconstructed image of the Clown using the Catté et al. (1992) model. Thethe proposed stopping criterion suggests to stop the diffusion process after 15 iteration

ping criterion is almost optimal, allowing the diffusion process tostop near the point where the three filtering methods reach theirbest possible image reconstructions.

Appendix B. Structure enhancing anisotropic nonlineardiffusion model

The proposed SED model, Eq. (17), is given by

ut �r � Dbf � ru� �

¼ 0; on X� 0;1½ Þ;u x;0ð Þ ¼ u0 xð Þ; on X;

Dbf � ru;n �

¼ 0 on oX� 0;1ð Þ:The algorithm to implement the proposed SED model is as follows:

dt ( 1=6 % time-steprd ( 1 % domain Gaussian kernel’s width

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Iterations

Nor

mal

ized

Cor

rela

tion

Coe

ffic

ient

Stopping Criterion

corr ( f, u) Catte−Lions−Morel−CollStopping Criterion

na using the Catté et al. (1992) model. The measure corrðf ; uÞ suggests stopping thetop the diffusion process after 15 iterations. (right) Stopping criterion performancemeasure corrðf ;uÞ suggests stopping the diffusion process after 16 iterations, while

s.

154 C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155

rr ( 10�2 % range Gaussian kernel’s width�rd ( 2 % orientation Gaussian kernel’s width�rr ( 10�2 % magnitude Gaussian kernel’s widthuðx;0Þ ( u0ðxÞ, on X % set initial condition�c1 ( corrðu;u0Þ % correlation measurerepeatubf ( Gbf � u % convolve image with bilateral filter kernelru( ½ux uy uz�T % estimate gradientsrubf ( ½ðubf Þx ðubf Þy ðubf Þz�

T % estimate gradients

jrubf j (ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðubf Þ2x þ ðubf Þ2y þ ðubf Þ2z

q% magnitude of the gradients

Jbf (rubf � ruTbf % structure tensor

Jbf ( Gbf � Jbf % convolve structure tensor with bilateral filter

vj;lj ( eigenðJbf Þ, j ¼ 1;2;3 % eigenvectors and eigenvalues

k( prctileðj rubf jÞ % contrast parameter

g ( 11þjrubf j2=k2 % diffusivity function

C2 ( sqtmðl1;l2Þ % coherence parameterC3 ( sqtmðl1;l3Þ % coherence parameterk1 ( g % prescribe eigenvaluek2 ( g þ ð1� gÞ exp � C2

ðl1�l2Þ2

� �% prescribe eigenvalue

k3 ( g þ ð1� gÞ exp � C3

ðl1�l3Þ2

� �% prescribe eigenvalue

Dbf ( ½v1 v2 v3�diagðk1; k2; k3Þ½v1 v2 v3�T % form diffusion tensorhDbf � ru;ni ( 0, on oX % set boundary conditions/ ¼ r � ðDbf � ruÞ % diffusion termu( uþ dt/ % evolve the image�ciþ1 ( corrðu;u0Þ % update correlation measureciþ1 ( o2�ciþ1 % stopping criterionuntil ociþ1 6 0

We are working on the implementation of a user-friendlypackage that will be distributed freely under an open source li-cense. The current code is implemented in MatLab� and ParallelMatLab� and will be ported to a faster implementation usingGPGPU. We hope to complete this phase of the project by theFall of 2009.

References

Alvarez, L., Lions, P.L., Morel, J.M., 1992. Image selective smoothing and edgedetection by nonlinear diffusion, II. SIAM Journal on Numerical Analysis 29 (3),845–866.

Aurich, V., Weule, J., 1995. Non-linear Gaussian filters performing edge preservingdiffusion. Proceedings of DAGM Symposium, vol. 17. DeutscheArbeitsgemeinschaft für Mustererkennung, pp. 538–545.

Bajaj, C., Yu, Z., 2005 (chapter 23). Geometric and Signal Processing ofReconstructed 3D Maps of Molecular Complexes. Handbook of ComputationalMolecular Biology. Chapman & Hall/CRC Press, New York, New York.

Bajaj, C., Wu, Q., Xu, G. 2003. Level-set based volumetric anisotropic diffusion for 3dimage denoising. ICES Technical Report 03-10. The University of Texas at Austin,Austin, Texas.

Bakker, P., van Vliet, L., Verbeek, P., 1999. Edge preserving orientation adaptivefiltering. In: Boasson, M., Kaandorp, J., Tonino, J., Vosselman, M. (Eds.),Proceedings of 5th Annual Conference of the Advanced School for Computingand Imaging, pp. 207–213.

Bazán, C., Blomgren, P., 2007. Image smoothing and edge detection by nonlineardiffusion and bilateral filter. Research Report CSRCR 200721. San Diego StateUniversity, San Diego, California.

Benhamouda, B. 1994. Parameter Adaptation for Nonlinear Diffusion in ImageProcessing. Master’s thesis. University of Kaiserslautern, Kaiserslautern,Germany.

Bigün, J., Granlund, G.H., 1987. Optimal orientation detection of linear symmetry.In: Proceedings of First International Conference on Computer Vision. IEEEComputer Society Press, London, England, pp. 433–438.

Brox, T., Weickert, J., 2002. Nonlinear matrix diffusion for optic flow estimation. In:van Gool, L. (Ed.), Lecture Notes in Computer Science. Springer-Verlag, Berlin,Germany, pp. 446–453.

Brox, T. van den Boomgaard, R., Lauze, F., van de Weijer, J., Weickert, J., Mrázek, P.,Kornprobst, P. 2005. Adaptive structure tensors and their applications.Technical Report 141. Universität des Saarlandes, Saarbrücken, Germany.

Canny, A., 1986. A computational approach to edge detection. IEEE Transactions onPattern Analysis and Machine Intelligence 8 (6), 679–698.

Catté, F., Lions, P.-L., Morel, J.-M., Coll, T., 1992. Image selective smoothing and edgedetection by nonlinear diffusion. SIAM Journal of Numerical Analysis 29 (1),182–193.

Chan, T.F., Vese, L.A., 2001. Active contours without edges. IEEE Transactions onImage Processing 10 (2), 266–277.

Cottet, G.-H., Germain, L., 1993. Image processing through reaction combined withnonlinear diffusion. Mathematics of Computation 61 (204), 659–673.

Fernández, J.J., Li, S., 2003. An improved algorithm for anisotropic nonlineardiffusion for denoising cryo-tomograms. Journal of Structural Biology 144 (1/2),152–161.

Fernández, J.J., Li, S., 2005. Anisotropic nonlinear filtering of cellular structures incryo-electron tomography. Computing in Science and Engineering 7 (5), 54–61.

Förstner, W., Gülch, E., 1987. A fast operator for detection and precise location ofdistinct points, corners and centres of circular features. In: Proceedings of ISPRSIntercommission Conference on Fast Processing of Photogrammetric Data, pp.281–305.

Frangakis, A.S., Hegerl, R., 2001. Noise reduction in electron tomographicreconstruction using nonlinear anisotropic diffusion. Journal of StructuralBiology 135 (3), 239–250.

Frangakis, A.S., Hegerl, R., 1999. Nonlinear anisotropic diffusion in three-dimensional electron microscopy. In: Leeuwen, G.G., van Hartmanis, J. (Eds.),Lecture Notes in Computer Science, vol. 1682. Springer-Verlag, Berlin, Germany,pp. 386–397.

Frangakis, A.S., Stoschek, A., Hegerl, R., 2001. Wavelet transform filtering andnonlinear anisotropic diffusion assessed for signal reconstruction performanceon multidimensional biomedical data. IEEE Transactions on BiomedicalEngineering 48 (2), 213–222.

Frank, S., Gaume, B., Bergmann-Leitner, E., Leitner, W., Robert, E., Catez, F., Smith, C.,Youle, R., 2001. The role of dynamin-related protein 1, a mediator ofmitochondrial fission, in apoptosis. Developmental Cell 1 (4), 515–525.

Frey, T.G., Renken, C.W., Perkins, G.A., 2002. Insight into mitochondrial structureand function from electron tomography. Biochimica et Biophysica Acta 1555(13), 196–203.

Frey, T.G., Perkins, G.A., Ellisman, M.H., 2006. Electron tomography of membrane-bound cellular organelles. The Annual Review of Biophysics and BiomolecularStructure 35, 199–224.

Fröhlich, J., Weickert, J., 1994. Image processing using a wavelet algorithm fornonlinear diffusion. Report 104. Laboratory of Technomathematics, Universityof Kaiserslautern, Kaiserslautern, Germany.

Green, D.R., Reed, J.C., 1998. Mitochondria and apoptosis. Science 281 (5381), 1309–1312.

Jiang, W., Baker, M.L., Wu, Q., Bajaj, C., Chiu, W., 2003. Applications of a bilateraldenoising filter in biological electron microscopy. Journal of Structural Biology144 (1/2), 114–122.

Kichenassamy, S., 1997. The Perona–Malik paradox. SIAM Journal of AppliedMathematics 57 (5), 1328–1342.

Kremer, J.R., Mastronarde, D.N., McIntosh, J.R., 1996. Computer visualization ofthree-dimensional image data using IMOD. Journal of Structural Biology 116(1), 71–76.

Kuwahara, M., Hachimura, K., Eiho, S., Kinoshita, M., 1976. Processing ofriangiocardiographic images. In: Preston, K., Onoe, M. (Eds.), DigitalProcessing of Biomedical Images. Plenum Press, New York, New York, pp.187–202.

Li, C., Xu, C., Gui, C., Fox, M.D., 2005. Level set evolution without re-initialization: anew variational formulation. In: Proceedings of IEEE International Conferenceon Computer Vision and Pattern Recognition, pp. 430–436.

Malladi, R., Sethian, J.A., Vemuri, B.C., 1996. A fast level set based algorithm fortopology-independent shape modeling. Journal of Mathematical Imaging andVision 6 (2/3), 269–289.

Marco, S., Bouldier, T., Messaoudi, C., Rigaud, J.-L., 2004. Electron tomography ofbiological samples. Biochemistry 69 (11), 1219–1225.

Meier, T., Ngan, K.N., Crebbin, G., 1997. A robust markovian segmentation based onhighest confidence first. Proceedings of IEEE International Conference on ImageProcessing, vol. 1. IEEE, pp. 216–219.

Mrázek, P., 2001. Nonlinear Diffusion for Image Filtering and MonotonicityEnhancement. PhD thesis. Czech Technical University, Prague, Czech Republic.

Munnich, A., Rustin, P., 2001. Clinical spectrum and diagnosis of mitochondrialdisorders. American Journal of Medical Genetics 106, 4–17.

Nagao, M., Matsuyama, T., 1979. Edge preserving smoothing. Computer Graphicsand Image Processing 9 (4), 394–407.

Nitzberg, M., Shiota, T., 1992. Nonlinear image filtering with edge and cornerenhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence14 (8), 826–833.

Obeid, M., Tesniere, A., Ghiringhelli, F., Fimia, G.M., Apetoh, L., Perfettini, J.L.,Castedo, M., Mignot, G., Panaretakis, T., Casares, N., Metivier, D., Larochette, N.,van Endert, P., Ciccosanti, F., Piacentini, M., Zitvogel, L., Kroemer, G., 2007.Calreticulin exposure dictates the immunogenicity of cancer cell death. NatureMedicine 13 (1), 54–61.

Osher, S., Fedkiw, R., 2003. Level Set Methods and Dynamic Implicit Surfaces, vol.153. Springer, New York, New York.

Osher, S., Sethian, J., 1988. Fronts propagating with curvature-dependent speed:algorithms based on the hamilton-jacobi formulation. Journal of ComputationalPhysics 79, 12–49.

Perkins, G., Frey, T., 1999. Electron tomography. In: Creighton, T. (Ed.), Encyclopediaof Molecular Biology. John Wiley & Sons, New York, New York, p. 796.

C. Bazán et al. / Journal of Structural Biology 166 (2009) 144–155 155

Perkins, G.A., Renken, C.W., Song, J.Y., Frey, T.G., Young, S.J., Lamont, S., Martone,M.E., Lindsey, S., Ellisman, M.H., 1997. Electron tomography of large,multicomponent biological structures. Journal of Structural Biology 120 (3),219–227.

Mannella, C.A., Marko, M., Penczek, P., Frank, J., 1994. The internalcompartmentation of rat-liver mitochondria: tomographic study using thehigh-voltage transmission electron microscope. Microscopical Research andTechnology 27, 278–283.

Perona, P., Malik, J., 1990. Scale space and edge detection using anisotropicdiffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(7), 629–639.

Pon, L.A., Schon, E.A., 2007. Preface. In: Pon, L.A., Schon, E.A. (Eds.), Mitochondria.Academic Press, San Diego, California, pp. xxiii–xxiv.

Sethian, J.A., 1999. Level Set Methods and Fast Marching Methods. CambridgeUniversity Press, Cambridge, Massachusetts.

Smithm, S.M., Brady, J.M., 1997. SUSAN—a new approach to low level imageprocessing. International Journal of Computer Vision 23 (1), 45–78.

Tandler, B., Dunlap, M., Hoppel, C.L., Hassan, M., 2002. Giant mitochondria in acardiomyopathic heart. Ultrastructural Pathology 26, 177–183.

Tomasi, C., Manduchi, R., 1998. Bilateral filtering for gray and color images. In:Proceedings of IEEE International Conference on Computer Vision. IEEE, pp.839–846.

van den Boomgaard, R., 2002. The Kuwahara–Nagao operator decomposed in termsof a linear smoothing and a morphological sharpening. In: Talbot, H., Beare, R.(Eds.), Proceedings of 6th International Symposium on MathematicalMorphology. CSIRO Publishing, Sydney, Australia, pp. 283–292.

van den Boomgaard, R., van de Weijer, J., 2002. Robust estimation of orientation fortexture analysis. In: Proceedings of Texture 2002, 2nd International Workshopon Texture Analysis and Synthesis. Copenhagen, Denmark.

Volkmann, N., 2002. A novel three-dimensional variant of the watershed transformfor segmentation of electron density maps. Journal of Structural Biology 138(12), 123–129.

Weickert, J., 2001. Efficient image segmentation using partial differential equationsand morphology. Pattern Recognition 34 (9), 1813–1824.

Weickert, J., 1994a. Theoretical foundations of anisotropic diffusion in imageprocessing. In: Proceedings of 7th TFCV on Theoretical Foundations of ComputerVision. Computing Supplement, vol. 11. Springer-Verlag, London, pp. 221–236.

Weickert, J., 1994b. Anisotropic diffusion filters for image processing based qualitycontrol. In: Fasano, A., Primicerio, M. (Eds.), Proceedings of Seventh EuropeanConference on Mathematics in Industry. Teubner, Stuttgart, Germany, pp. 355–362.

Weickert, J., 1995. Multiscale texture enhancement. In: Hlavác, V., Šára (Eds.),Computer Analysis of Images and Patterns, Lecture Notes in Computer Science.Springer, Berlin, Germany, pp. 230–237.

Weickert, J., 1996a. Theoretical foundations of anisotropic diffusion in imageprocessing. Computing Supplement 11, 221–236.

Weickert, J., 1996b. Anisotropic Diffusion in Image Processing. PhD thesis.Universität Kaiserslautern, Kaiserslautern, Germany.

Weickert, J., 1998. Anisotropic Diffusion in Image Processing. ECMI Series. Teubner-Verlag, Stuttgart.

Weickert, J., 1999a. Coherence-enhancing diffusion of colour images. Image andVision Computing 17 (3), 201–212.

Weickert, J., 1999b. Coherence-enhancing diffusion filtering. International Journalof Computer Vision 31 (2/3), 111–127.

Weickert, J., Schnörr, C., 2000. PDE-based preprocessing of medical images.Kunstliche Intelligenz 3, 5–10.

Weickert, J., Romeny, B.M.t.H., Viergever, M.A., 1998. Efficient and reliable schemesfor nonlinear diffusion filtering. IEEE Transactions on Image Processing 7 (3),398–410.

Whitaker, R.T., Pizer, S.M., 1993. A multi-scale approach to non-uniform diffusion.Computer Vision, Graphics, and Image Processing: Image Understanding 57 (1),99–110.

Zamzami, N., Maisse, C., Métivier, D., Kroemer, G., 2007. Measurement of membranepermeability and the permeability transition of mitochondria. In: Pon, L.A.,Schon, E.A. (Eds.), Mitochondria, Methods in Cell Biology, vol. 80. AcademicPress, San Diego, California, pp. 327–340 (chapter 16).